In this letter, we learned overcomplete filters to model rich priors of nature images. Our approach extends the Gaussian Scale Mixture Fields of Experts (GSM FOE), which is a fast approximate model based on Fields of Experts (FOE). In these previous image prior model, the overcomplete case is not considered because of the heavy computation. We introduce the assumption of quasi-orthogonality to the GSM FOE, which allows us to learn overcomplete filters of nature images fast and efficiently. Simulations show these obtained overcomplete filters have properties similar with those of Fields of Experts', and denoising experiments also show the superiority of our model.