Approximation of elliptic PDEs with random diffusion coefficients typically requires an expansion of the diffusion field, which is then used to define approximations such as polynomials in the scalar variables appearing in the expansion. Although Karhunen-Loeve representations are commonly used to define such expansions, it was recently shown, in the relevant case of lognormal diffusion fields, that they do not generally yield optimal approximation rates. Motivated by these results, we construct wavelet-type expansions of stationary Gaussian random fields defined on bounded domains. The size and localization properties of these wavelets are studied, and used to obtain polynomial approximation results for the related elliptic PDE which outperform those achievable when using Karhunen-Loeve representations. Our construction is based on a periodic extension of the random field, and the expansion on the domain is then obtained by simple restriction. This makes the approach easily applicable even when the computational domain of the PDE has a complicated geometry. In particular, we apply this construction to the class of Gaussian processes defined by the family of Matern covariances.