We propose a retrieval method using scale invariant visual phrases (SIVPs). Our method encodes spatial information into the SIVPs which capture translation, rotation and scale invariance, and employs the SIVPs to determine the spatial correspondences between query image and database image. To compute the spatial correspondences efficiently, the SIVPs are introduced into the inverted index, and SIVP verification is investigated to refine the candidate images returned from inverted index. Experimental results demonstrate that our method improves the retrieval accuracy while increasing the retrieval efficiency.