This paper deals with the use of neural network rule extraction techniques based on the Genetic Programming (GP) to build intelligent and explanatory evaluation systems. Recent development in algorithms that extract rules from trained neural networks enable us to generate classification rules in spite of their intrinsically black-box nature. However, in the original decompositional method looking at the internal structure of the networks, the comprehensive methods combining the output to the inputs using parameters are complicated. Then, in our paper, we utilized the GP to automatize the rule extraction process in the trained neural networks where the statements changed into a binary classification. Even though the production (classification) rule generation based on the GP alone are applicable straightforward to the underlying problems for decision making, but in the original GP method production rules include many statements described by arithmetic expressions as well as basic logical expressions, and it makes the rule generation process very complicated. Therefore, we utilize the neural network and binary classification to obtain simple and relevant classification rules in real applications by avoiding straightforward applications of the GP procedure to the arithmetic expressions. At first, the pruning process of weight among neurons is applied to obtain simple but substantial binary expressions which are used as statements is classification rules. Then, the GP is applied to generate ultimate rules. As applications, we generate rules to prediction of bankruptcy and creditworthiness for binary classifications, and the apply the method to multi-level classification of corporate bonds (rating) by using the financial indicators.