In image-based relighting (IBR), users are allowed to control the illumination condition of a scene or an object. A relighting data set (RDS) contains a large number of reference images captured under various directional light sources. This paper proposes a principal component analysis (PCA) based compression scheme that effectively reduces the data volume. Since the size of images is very large, a tiling recursive least square PCA (RLS-PCA) is used. The output of RLS-PCA is a set of eigenimages and the corresponding eigen coefficients. To further compress the data, extracted eigenimages are compressed using transform coding while extracted eigen coefficients are compressed using uniform quantization with entropy coding. Our simulation shows that the proposed approach is superior to compressing reference images with JPEG and MPEG2.