The hardware implementation of a neural network model using stochastic logic has been able to integrate numerous neuron units on a chip. However, the limitation of applications occurred since the stochastic neurosystem could execute only discrete-time dynamics. We have contrived a neuron model with continuous-time dynamics by using stochastic calculations. In this paper, we propose the circuit design of a new neuron circuit, and show the fabricated neurochip comprising 64 neurons with experimental results. Furthermore, a new asynchronous updating method and a new activation function circuit are proposed. These improvements enhance the performance of the neurochip greatly.