In this paper, we propose a new computer-aided diagnosis system which can extract specific features from hematoxylin and eosin (HE)-stained breast tumor images and evaluate the type of tumor using artificial organisms. The gene of the artificial organisms is defined by three kinds of texture features, which can evaluate the specific features of the tumor region in the image. The artificial organisms move around in the image and investigate their environmental conditions during the searching process. When the target pixel is regarded as a tumor region, the organism obtains energy and produces offspring; organisms in other regions lose energy and die. The searching process is iterated until the 30th generation; as a result, tumor regions are filled with artificial organisms. Whether the detected tumor is benign or malignant is evaluated based on the combination of selected genes. The method developed was applied to 27 test cases and the distinction between benign and malignant tumors by the artificial organisms was successful in about 90% of tumor images. In this diagnosis support system, the combination of genes, which represents specific features of detected tumor region, is selected automatically for each tumor image during the searching process.