To implement the parallel acceleration of convolution operation of Convolutional Neural Networks (CNNs) on field programmable gate array (FPGA), large quantities of the logic resources will be consumed, expecially DSP cores. Many previous researches fail to make a well balance between DSP and LUT6. For better resource efficiency, a typical convolution structure is implemented with LUT6s in this paper. Besides, a novel convolution structure is proposed to further reduce the LUT6 resource consumption by modifying the typical convolution structure. The equations to evaluate the LUT6 resource consumptions of both structures are presented and validated. The theoretical evaluation and experimental results show that the novel structure can save 3.5-8% of LUT6s compared with the typical structure.