In this paper, we show that the neural network can approximate the chaotic behavior in nonlinear dynamical system by experimental study. Chaotic neural activities have been reported in many respects including neural network field. On the contrary, can the neural network learn the chaotic behavior? There have been explored the neural network architecture for predicting successive elements of a sequence. Also there have been several studies related to learning algorithms for general recurrent neural networks. But they often require complicated procedure in time calculation. We use simple standard backpropagation for a kind of simple recurrent neural network. Two types of chaotic system, differential equation and difference equation, are examined to compare characteristics. In the experiments, Lorenz equation is used as an example of differential equation. One-dimensional logistic equation and Henon equation are used as examples of difference equation. As a result, we show the approximation ability of chaotic dynamics in difference equation, which is logistic equation and Henon equation, by neural network. To indicate the chaotic state, we use Lyapunov exponent which represents chaotic activity.