A dynamic neural network has ternary connection parameters and can generate various binary periodic orbits. In order to analyze the dynamics, we present two feature quantities which characterize stability and transient phenomenon of a periodic orbit. Calculating the feature quantities, we investigate influence of connection sparsity on stability of a target periodic orbit corresponding to a circuit control signal. As the sparsity increases, at first, stability of a target periodic orbit tends to be stronger. In the next, the stability tends to be weakened and various transient phenomena exist. In the most sparse case, the network has many periodic orbits without transient phenomenon.