In a patch-based super-resolution algorithm, a low-resolution patch is influenced by surrounding patches due to blurring. We propose to remove this boundary effect by subtracting the blur from the surrounding high-resolution patches, which enables more accurate sparse representation. We demonstrate improved performance through experimentation. The proposed algorithm can be applied to most of patch-based super-resolution algorithms to achieve additional improvement.