In this paper, we investigated the overfitting characteristics of nonlinear equalizers based on an artificial neural network (ANN) and the Volterra series transfer function (VSTF), which were designed to compensate for optical nonlinear waveform distortion in optical fiber communication systems. Linear waveform distortion caused by, e.g., chromatic dispersion (CD) is commonly compensated by linear equalizers using digital signal processing (DSP) in digital coherent receivers. However, mitigation of nonlinear waveform distortion is considered to be one of the next important issues. An ANN-based nonlinear equalizer is one possible candidate for solving this problem. However, the risk of overfitting of ANNs is one obstacle in using the technology in practical applications. We evaluated and compared the overfitting of ANN- and conventional VSTF-based nonlinear equalizers used to compensate for optical nonlinear distortion. The equalizers were trained on repeated random bit sequences (RRBSs), while varying the length of the bit sequences. When the number of hidden-layer units of the ANN was as large as 100 or 1000, the overfitting characteristics were comparable to those of the VSTF. However, when the number of hidden-layer units was 10, which is usually enough to compensate for optical nonlinear distortion, the overfitting was weaker than that of the VSTF. Furthermore, we confirmed that even commonly used finite impulse response (FIR) filters showed overfitting to the RRBS when the length of the RRBS was equal to or shorter than the length of the tapped delay line of the filters. Conversely, when the RRBS used for the training was sufficiently longer than the tapped delay line, the overfitting could be suppressed, even when using an ANN-based nonlinear equalizer with 10 hidden-layer units.

With the continued innovation of deep neural networks, spiking neural networks (SNNs) that more closely resemble biological brain synapses have attracted attention because of their low power consumption. Unlike artificial neural networks (ANNs), for continuous data values, they must employ an encoding process to convert the values to spike trains, suppressing the SNN's performance. To avoid this degradation, the incoming analog signal must be regulated prior to the encoding process, which is also realized in living things eg, the basement membranes of humans mechanically perform the Fourier transform. To this end, we combine an ANN and an SNN to build ANN-to-SNN hybrid neural networks (HNNs) that improve the concerned performance. To qualify this performance and robustness, MNIST and CIFAR-10 image datasets are used for various classification tasks in which the training and encoding methods changes. In addition, we present simultaneous and separate training methods for the artificial and spiking layers, considering the encoding methods of each. We find that increasing the number of artificial layers at the expense of spiking layers improves the HNN performance. For straightforward datasets such as MNIST, similar performances as ANN's are achieved by using duplicate coding and separate learning. However, for more complex tasks, the use of Gaussian coding and simultaneous learning is found to improve the accuracy of the HNN while lower power consumption.

A feedforward (FF) network using ΔΣ modulators is investigated to implement a non-binary analog-to-digital (A/D) converter. Weighting coefficients in the network are determined to suppress the generation of quantization noise. A moving average is adopted to prevent the analog signal amplitude from increasing beyond the allowable input range of the modulators. The noise transfer function is derived and used to estimate the signal-to-noise ratio (SNR). The FF network output is a non-uniformly distributed multi-level signal, which results in a better SNR than a uniformly distributed one. Also, the effect of the characteristic mismatch in analog components on the SNR is analyzed. Our behavioral simulations show that the SNR is improved by more than 30 dB, or equivalently a bit resolution of 5 bits, compared with a conventional first-order ΔΣ modulator.

This paper describes the advantages and disadvantages observed when describing complex parameterizable Artificial Neural Networks (ANNs) at the behavioral level using SystemC and at the Register Transfer Level (RTL) using VHDL. ANNs are complex to parameterize because they have a configurable number of layers, and each one of them has a unique configuration. This kind of structure makes ANNs, a priori, challenging to parameterize using Hardware Description Languages (HDL). Thus, it seems intuitively that ANNs would benefit from the raise in level of abstraction from RTL to behavioral level. This paper presents the results of implementing an ANN using both levels of abstractions. Results surprisingly show that VHDL leads to better results and allows a much higher degree of parameterization than SystemC. The implementation of these parameterizable ANNs are made open source and are freely available online. Finally, at the end of the paper we make some recommendation for future HLS tools to improve their parameterization capabilities.

With the fast growth of the international tourism industry, it has been a challenge to forecast the tourism demand in the international tourism market. Traditional forecasting methods usually suffer from the prediction accuracy problem due to the high volatility, irregular movements and non-stationarity of the tourist time series. In this study, a novel single dendritic neuron model (SDNM) is proposed to perform the tourism demand forecasting. First, we use a phase space reconstruction to analyze the characteristics of the tourism and reconstruct the time series into proper phase space points. Then, the maximum Lyapunov exponent is employed to identify the chaotic properties of time series which is used to determine the limit of prediction. Finally, we use SDNM to make a short-term prediction. Experimental results of the forecasting of the monthly foreign tourist arrivals to Japan indicate that the proposed SDNM is more efficient and accurate than other neural networks including the multi-layered perceptron, the neuro-fuzzy inference system, the Elman network, and the single multiplicative neuron model.

DDoS remains a major threat to Software Defined Networks. To keep SDN secure, effective detection techniques for DDoS are indispensable. Most of the newly proposed schemes for detecting such attacks on SDN make the SDN controller act as the IDS or the central server of a collaborative IDS. The controller consequently becomes a target of the attacks and a heavy loaded point of collecting traffic. A collaborative intrusion detection system is proposed in this paper without the need for the controller to play a central role. It is deployed as a modified artificial neural network distributed over the entire substrate of SDN. It disperses its computation power over the network that requires every participating switch to perform like a neuron. The system is robust without individual targets and has a global view on a large-scale distributed attack without aggregating traffic over the network. Emulation results demonstrate its effectiveness.

Prediction of seasonal infectious disease spread is traditionally dealt with as a function of time. Typical methods are time series analysis such as ARIMA (autoregressive, integrated, and moving average) or ANN (artificial neural networks). However, if we regard the time series data as the matrix form, e.g., consisting of yearly magnitude in row and weekly trend in column, we may expect to use a different method (matrix approach) to predict the disease spread when seasonality is dominant. The MD (matrix decomposition) method is the one method which is used in recommendation systems. The other is the IRT (item response theory) used in ability evaluation systems. In this paper, we apply these two methods to predict the disease spread in the case of infectious gastroenteritis caused by norovirus in Japan, and compare the results obtained by using two conventional methods in forecasting, ARIMA and ANN. We have found that the matrix approach is simple and useful in prediction for the seasonal infectious disease spread.

Regular on-site testing is an elementary means to obtain real-time data and state of Electromagnetic Compatibility (EMC) of electronics systems. Nowadays, there is a lot of measured EMC data while the application of the data is insufficient. So we put forward the concept of EMC model synthesis. To carry out EMC data mining with measured electromagnetic data, we can build or modify models and synthesize variation rules of electromagnetic parameters of equipment and EMC performance of systems and platforms, then realize the information synthesis and state prediction. The concept of EMC reliability is brought forward together with the definition and description of parameters such as invalidation rate and EMC lifetime. We studied the application of statistical algorithms and Artificial Neural Network (ANN) in model synthesis. Operating flows and simulation results as well as measured data are presented. Relative research can support special measurement, active management and predictive maintenance and replenishment in the area of EMC.

In next generation mobile multimedia communications, different wireless access networks are expected to cooperate. However, it is a challenging task to choose an optimal transmission path in this scenario. This paper focuses on the problem of selecting the optimal access network for multicast services in the cooperative mobile and broadcasting networks. An algorithm is proposed, which considers multiple decision factors and multiple optimization objectives. An analytic hierarchy process (AHP) method is applied to schedule the service queue and an artificial neural network (ANN) is used to improve the flexibility of the algorithm. Simulation results show that by applying the AHP method, a group of weight ratios can be obtained to improve the performance of multiple objectives. And ANN method is effective to adaptively adjust weight ratios when users' new waiting threshold is generated.

A new type of algorithm for predicting the mechanical faults of a vacuum circuit breaker (VCB) based on an artificial neural network (ANN) is proposed in this paper. There are two types of mechanical faults in a VCB: operation mechanism faults and tripping circuit faults. An angle displacement sensor is used to measure the main axle angle displacement which reflects the displacement of the moving contact, to obtain the state of the operation mechanism in the VCB, while a Hall current sensor is used to measure the trip coil current, which reflects the operation state of the tripping circuit. Then an ANN prediction algorithm based on a sliding time window is proposed in this paper and successfully used to predict mechanical faults in a VCB. The research results in this paper provide a theoretical basis for the realization of online monitoring and fault diagnosis of a VCB.

We investigated relations between torque and elbow joint angle for constant muscle activations in isovelocity flexion movements of the forearm in three normal subjects. The reference angular velocity was from 0 to 90°/s and the applied torque from 0 to 15% of maximum voluntary contraction. Integrated surface electromyograms (IEMGs) of six muscles, torque, angle and angular velocity of the elbow joint were measured. A mathematical model describing the relationship between these variables was constructed with an artificial neural network. We estimated elbow joint torque by presenting different elbow joint angles, constant IEMGs and constant angular velocity to the model. For elbow joint angles greater than 60°, the slope, which was defined as the rate of torque increase with respect to elbow joint angle, was negative. For elbow joint angles less than 50°, the slope changed from positive to negative when the angular velocity increased. This implied that the flexor muscle-elbow joint system could change from unstable to stable when the angular velocity increased.

In this paper, we present a learning approach, positive correlation learning (PCL), that creates a multilayer neural network with good generalization ability. A correlation function is added to the standard error function of back propagation learning, and the error function is minimized by a steepest-descent method. During training, all the unnecessary units in the hidden layer are correlated with necessary ones in a positive sense. PCL can therefore create positively correlated activities of hidden units in response to input patterns. We show that PCL can reduce the information on the input patterns and decay the weights, which lead to improved generalization ability. Here, the information is defined with respect to hidden unit activity since the hidden unit plays a crucial role in storing the information on the input patterns. That is, as previously proposed, the information is defined by the difference between the uncertainty of the hidden unit at the initial stage of learning and the uncertainty of the hidden unit at the final stage of learning. After deriving new weight update rules for the PCL, we applied this method to several standard benchmark classification problems such as breast cancer, diabetes and glass identification problems. Experimental results confirmed that the PCL produces positively correlated hidden units and reduces significantly the amount of information, resulting improved generalization ability.

Using the Vibration signatures obtained during the operations as the original data, a mechanical condition monitoring method for vacuum circuit breaker is developed in this paper. The method combined the time-frequency analysis and the condition recognition based on artificial neural network. During preprocessing, the vibration signature was decomposed into individual frequency bands using the arithmetic of wavelet packets. The signal energy in the main frequency bands was used to form the condition feature vector, which was input to the artificial neural network for condition recognition. By introducing the parameter of approximation degree, a new recognition arithmetic based on Radial Basis Function was constructed. This approach could not only distinguish these conditions that belong to different known condition modes but also distinguish new condition modes.

This paper presents the design of a novel method for improvement of the operation of distance relays during capacitive voltage transformer transients using artificial neural network. The proposed module uses voltage and current signals to learn the hidden relationship existing in the input patterns. Simulation studies are preformed and the influence of changing system parameters, such as fault resistance and source impedance is studied. Details of the design procedure and the results of performance studies with the proposed relay are given in the paper. Performance studies results show that the proposed algorithm decreases the effects of CVT transients and is fast and accurate.

The paper first researches the properties of neural networks in the framework of the dual linear programming theory, then discusses the variation range of a Hessian matrix associated to dual linear programming problems. By means of eigenvalues method, a Lipschitz constant based formula for determining the algorithm step-size is presented. Two examples are given to show that the proposed formula is efficacious.

Single electron devices are ultra low power and extremely small devices, and suitable for implementation of large scale integrated circuits. Artificial neural networks (ANNs), which require a large number of transistors for being to be applied to practical use, is one of the possible applications of single electron devices. In order to simplify a single electron circuit configuration, we apply stochastic logic in which various complex operations can be done with basic logic gates. We design basic subcircuits of a single electron stochastic neural network, and confirm that backgate bias control and a redundant configuration are necessary for a feedback loop configuration by computer simulation based on Monte Carlo method. The proposed single electron circuit is well-suited for hardware implementation of a stochastic neural network because we can save circuit area and power consumption by using a single electron random number generator (RNG) instead of a conventional complementary metal oxide semiconductor (CMOS) RNG.

Scientific computations for diffusion equations and ANNs (Artificial Neural Networks) are data intensive tasks accompanied by heavy memory access; on the other hand, their computational complexities are relatively low. Thus, this type of tasks naturally maps onto SIMD (Single Instruction Multiple Data stream) parallel processing with distributed memory. This paper proposes a high performance acceleration processor of which architecture is optimized for scientific computing using diffusion equations and ANNs. The proposed architecture includes a customized instruction set and specific hardware resources which consist of a control unit (CU), 16 processing units (PUs), and a non-linear function unit (NFU) on chip. They are effectively connected with dedicated ring and global bus structure. Each PU is equipped with an address modifier (AM) and 16-bit 1.5 k-word local memory (LM). The proposed processor can be easily expanded by multi-chip expansion mode to accommodate to a large scale parallel computation. The prototype chip is implemented with FPGA. The total gate count is about 1 million with 530, 432-bit embedded memory cells and it operates at 15 MHz. The functionality and performance of the proposed processor is verified with simulation of oil reservoir problem using diffusion equations and character recognition application using ANNs. The execution times of two applications are compared with software realizations on 1.7 GHz Pentium IV personal computer. Though the proposed processor architecture and the instruction set are optimized for diffusion equations and ANNs, it provides flexibility to program for many other scientific computation algorithms.

A brain-computer interface using an electroencephalogram as input into an artificial neural network is investigated as a potentially general control system applicable to all subjects and time frames. Using the intent and imagination of bending the left or right elbow, the left and right desired movements are successfully distinguished using event-related desynchronization resolved by fast Fourier transformation of the electroencephalogram and analysis of the power spectrum using the artificial neural network. The influence of age was identified and eliminated through the use of a frequency distribution in the α band, and the recognition rate was further improved by confirmation based on forced excitement of the β band in the case of an error. The proposed system was effectively trained for general use by using the combined data of a cross-section of subjects.

A method of learning for multi-layer artificial neural networks is proposed. The learning model is designed to provide an effective means of escape from the Backpropagation local minima. The system is shown to escape from the Backpropagation local minima and be of much faster convergence than simulated annealing techniques by simulations on the exclusive-or problem and the Arabic numerals recognition problem.

Investigations into the suitability of artificial neural network for the prediction of rain attenuation based on radio, meteorological and geographical data from ITU-R data bank are presented. First successful steps towards a prediction model of rain attenuation for radio communication based on adaptive learning from the measurement are made. Rain attenuation prediction with the model based on artificial neural network shows good conformity with the measurement. Moreover, a new evolutionary system, EPNet is used to evolve the artificial neural network rain attenuation model obtained both in architecture and weight, and an optimal rain attenuation model with simpler architecture and better prediction accuracy based on EPNet-evolved artificial neural network is obtained. Compared with the ITU-R model, the EPNet-evolved artificial neural network model of rain attenuation proposed in this paper improves the accuracy of rain attenuation prediction and creates a novel way to predict rain attenuation.