Real-time object detection or recognition technology becomes more important for various intelligent vision systems. Processing models for object detection or recognition from natural images should tolerate pattern deformations and pattern position shifts. The hierarchical convolutional neural networks are considered as a promising model for robust object detection/recognition. This model requires huge computational power for a large number of multiply-and-accumulation operations. In order to apply this model to robot vision or various intelligent real-time vision systems, its LSI implementation is essential. This paper proposes a new algorithm for reducing multiply-and-accumulation operation by sorting neuron outputs by magnitude. We also propose an LSI architecture based on this algorithm. As a proof of concept for our LSI architecture, we have designed, fabricated and tested two test LSIs: a sorting LSI and an image-filtering LSI. The sorting LSI is designed based on the content addressable memory (CAM) circuit technology. The image-filtering LSI is designed for parallel processing by analog circuit array based on the merged/mixed analog-digital approach. We have verified the validity of our LSI architecture by measuring the LSIs.