In this article we present a survey of medical image processing with the stress on applications of image generation and pattern recognition / understanding to computer aided diagnosis (CAD) and surgery (CAS). First, topics and fields of research in medical image processing are summarized. Second the importance of the 3D image processing and the use of virtualized human body (VHB) is pointed out. Thirdly the visualization and the observation methods of the VHB are introduced. In the forth section the virtualized endoscope system is presented from the viewpoint of the observation of the VHB with the moving viewpoints. The fifth topic is the use of VHB with deformation such as the simulation of surgical operation, intra-operative aids and image overlay. In the seventh section several topics on image processing methodologies are introduced including model generation, registration, segmentation, rendering and the use of knowledge processing.