In the self-similarity super resolution (SR) approach, similar examples are searched across down-scales in the image pyramid, and the computations of searching similar examples are very heavy. This makes it difficult to work in a real-time way under common software implementation. Therefore, the search process should be further accelerated at an algorithm level. Cauchy-Schwarz inequality has been used previously for fast vector quantization (VQ) encoding. The candidate patches in the search region of SR are analogous to the code-words in the VQ, and Cauchy-Schwarz inequality is exploited to exclude implausible candidate patches early. Consequently, significant acceleration of the similar patch search process is achieved. The proposed method can easily make an optimal trade-off between running speed and visual quality by appropriately configuring the bypass-threshold.