This paper is concerned with the data compression and interpolation of multi-view image set. In this paper, we propose a novel disparity compensation scheme based on geometric relationship. We first investigate the geometric relationship between a point in the object space and its projection onto view images. Then, we propose the disparity compensation scheme which utilize the geometric constraints between view images. This scheme is used to compress the multi-view image into the structure of the triangular patches and the texture data on the surface of patches. This scheme not only compresses the multi-view image but also synthesize the view images from any viewpoints in the viewing zone. Also, this scheme is fast and have compatibility with 2-D interframe coding. Finally, we report the experiment, where two sets multi-view image were used as original images and the amount of data was reduced to 1/19 and 1/20 with SNR 34 dB and 20 dB, respectively.