Multilayer perceptron (MLP) is a basic neural network model that is used in practical industrial applications, such as network intrusion detection (NID) systems. It is also used as a building block in newer models, such as gMLP. Currently, there is a demand for fast training in NID and other areas. However, in training with numerous GPUs, the problems of power consumption and long training times arise. Many of the latest deep neural network (DNN) models and MLPs are trained using a backpropagation algorithm which transmits an error gradient from the output layer to the input layer such that in the sequential computation, the next input cannot be processed until the weights of all layers are updated from the last layer. This is known as backward locking. In this study, a weight parameter update mechanism is proposed with time delays that can accommodate the weight update delay to allow simultaneous forward and backward computation. To this end, a one-dimensional systolic array structure was designed on a Xilinx U50 Alveo FPGA card in which each layer of the MLP is assigned to a processing element (PE). The time-delay backpropagation algorithm executes all layers in parallel, and transfers data between layers in a pipeline. Compared to the Intel Core i9 CPU and NVIDIA RTX 3090 GPU, it is 3 times faster than the CPU and 2.5 times faster than the GPU. The processing speed per power consumption is 11.5 times better than that of the CPU and 21.4 times better than that of the GPU. From these results, it is concluded that a training accelerator on an FPGA can achieve high speed and energy efficiency.

With the rapid progress of the integrated nanophotonics technology, the optical neural network architecture has been widely investigated. Since the optical neural network can complete the inference processing just by propagating the optical signal in the network, it is expected more than one order of magnitude faster than the electronics-only implementation of artificial neural networks (ANN). In this paper, we first propose an optical vector-matrix multiplication (VMM) circuit using wavelength division multiplexing, which enables inference processing at the speed of light with ultra-wideband. This paper next proposes optoelectronic circuit implementation for batch normalization and activation function, which significantly improves the accuracy of the inference processing without sacrificing the speed performance. Finally, using a virtual environment for machine learning and an optoelectronic circuit simulator, we demonstrate the ultra-fast and accurate operation of the optical-electronic ANN circuit.

MathML is a standard markup language for describing math expressions. MathML consists of two sets of elements: Presentation Markup and Content Markup. The former is widely used to display math expressions in Web pages, while the latter is more suited to the calculation of math expressions. In this letter, we focus on the former and consider classifying Presentation MathML expressions. Identifying the classes of given Presentation MathML expressions is helpful for several applications, e.g., Presentation to Content MathML conversion, text-to-speech, and so on. We propose a method for classifying Presentation MathML expressions by using multilayer perceptron. Experimental results show that our method classifies MathML expressions with high accuracy.

Arrhythmia classification based on electrocardiogram (ECG) is crucial in automatic cardiovascular disease diagnosis. The classification methods used in the current practice largely depend on hand-crafted manual features. However, extracting hand-crafted manual features may introduce significant computational complexity, especially in the transform domains. In this study, an accurate method for patient-specific ECG beat classification is proposed, which adopts morphological features and timing information. As to the morphological features of heartbeat, an attention-based two-level 1-D CNN is incorporated in the proposed method to extract different grained features automatically by focusing on various parts of a heartbeat. As to the timing information, the difference between previous and post RR intervels is computed as a dynamic feature. Both the extracted morphological features and the interval difference are used by multi-layer perceptron (MLP) for classifing ECG signals. In addition, to reduce memory storage of ECG data and denoise to some extent, an adaptive heartbeat normalization technique is adopted which includes amplitude unification, resolution modification, and signal difference. Based on the MIT-BIH arrhythmia database, the proposed classification method achieved sensitivity Sen=93.4% and positive predictivity Ppr=94.9% in ventricular ectopic beat (VEB) detection, sensitivity Sen=86.3% and positive predictivity Ppr=80.0% in supraventricular ectopic beat (SVEB) detection, and overall accuracy OA=97.8% under 6-bit ECG signal resolution. Compared with the state-of-the-art automatic ECG classification methods, these results show that the proposed method acquires comparable accuracy of heartbeat classification though ECG signals are represented by lower resolution.

In this paper, we present several cryptanalyses of Hierocrypt-L1 block cipher, which was selected as one of the CRYPTREC recommended ciphers in Japan in 2003. We present a differential attack and an impossible differential attack on 8 S-function layers in a related-key setting. We first show that there exist the key scheduling differential characteristics which always hold, then we search for differential paths for the data randomizing part with the minimum active S-boxes using the above key differentials. We also show that our impossible differential attack is a new type.

Glial cells include several types of cells such as astrocytes, and oligodendrocytes apart from the neurons in the brain. In particular, astrocytes are known to be important in higher brain function and are therefore sometimes simply called glial cells. An astrocyte can transmit signals to other astrocytes and neurons using ion concentrations. Thus, we expect that the functions of an astrocyte can be applied to an artificial neural network. In this study, we propose a multi-layer perceptron (MLP) with a pulse glial chain. The proposed MLP contains glia (astrocytes) in a hidden layer. The glia are connected to neurons and are excited by the outputs of the neurons. The excited glia generate pulses that affect the excitation thresholds of the neurons and their neighboring glia. The glial network provides a type of positional relationship between the neurons in the hidden layer, which can enhance the performance of MLP learning. We confirm through computer simulations that the proposed MLP has better learning performance than a conventional MLP.

This letter studies the simple dynamic binary neural network characterized by signum activation function and ternary connection parameters. In order to control the sparsity of the connections and the stability of the stored signal, a simple evolutionary algorithm is presented. As a basic example of teacher signals, we consider a binary periodic orbit which corresponds to a control signal of ac-dc regulators. In the numerical experiment, applying the correlation-based learning, the periodic orbit can be stored. The sparsification can be effective to reinforce the stability of the periodic orbit.

This paper presents a new span-based dependency chart parsing algorithm that models the relations between the left and right dependents of a head. Such relations cannot be modeled in existing span-based algorithms, despite their popularity in dependency corpora. We address this problem through ternary-span combination during the subtree derivation. By modeling the relations between the left and right dependents of a head, our proposed algorithm provides a better capability of coordination disambiguation when the conjunction is annotated as the head of the left and right conjuncts. This eventually leads to state-of-the-art performance of dependency parsing on the Chinese data of the CoNLL shared task.

MISTY1 is a 64-bit block cipher that has provable security against differential and linear cryptanalysis. MISTY1 is one of the algorithms selected in the European NESSIE project, and it is recommended for Japanese e-Government ciphers by the CRYPTREC project. In this paper, we report on 12th order differentials in 3-round MISTY1 with FL functions and 44th order differentials in 4-round MISTY1 with FL functions both previously unknown. We also report that both data complexity and computational complexity of higher order differential attacks on 6-round MISTY1 with FL functions and 7-round MISTY1 with FL functions using the 46th order differential can be reduced to as much as 1/22 of the previous values by using multiple 44th order differentials simultaneously.

We consider a model composed of nonlinear perceptrons and analytically investigate its generalization performance using correlated examples in the framework of on-line learning by a statistical mechanical method. In Hebbian and AdaTron learning, the larger the number of examples used in an update, the slower the learning. In contrast, Perceptron learning does not exhibit such behaviors, and the learning becomes fast in some time region.

In this study, we propose a multi-layer perceptron with a glial network which is inspired from the features of glias in the brain. All glias in the proposed network generate independent oscillations, and the oscillations propagate through the glial network with attenuation. We apply the proposed network to the two-spiral problem. Computer simulations show that the proposed network gains a better performance than the conventional multi-layer perceptron.

MISTY1 is a 64-bit block cipher that has provable security against differential and linear cryptanalysis. MISTY1 is one of the algorithms selected in the European NESSIE project, and it has been recommended for Japanese e-Government ciphers by the CRYPTREC project. This paper shows that higher order differential attacks can be successful against 7-round versions of MISTY1 with FL functions. The attack on 7-round MISTY1 can recover a partial subkey with a data complexity of 254.1 and a computational complexity of 2120.8, which signifies the first successful attack on 7-round MISTY1 with no limitation such as a weak key. This paper also evaluates the complexity of this higher order differential attack on MISTY1 in which the key schedule is replaced by a pseudorandom function. It is shown that resistance to the higher order differential attack is not substantially improved even in 7-round MISTY1 in which the key schedule is replaced by a pseudorandom function.

MISTY1 is a 64-bit block cipher that has provable security against differential and linear cryptanalysis. MISTY1 is one of the algorithms selected in the European NESSIE project, and it has been recommended for Japanese e-Government ciphers by the CRYPTREC project. This paper reports a previously unknown higher order differential characteristic of 4-round MISTY1 with the FL functions. It also shows that a higher order differential attack that utilizes this newly discovered characteristic is successful against 6-round MISTY1 with the FL functions. This attack can recover a partial subkey with a data complexity of 253.7 and a computational complexity of 264.4, which is better than any previous cryptanalysis of MISTY1.

In this paper we analytically investigate the generalization performance of learning using correlated inputs in the framework of on-line learning with a statistical mechanical method. We consider a model composed of linear perceptrons with Gaussian noise. First, we analyze the case of the gradient method. We analytically clarify that the larger the correlation among inputs is or the larger the number of inputs is, the stricter the condition the learning rate should satisfy is, and the slower the learning speed is. Second, we treat the block orthogonal projection learning as an alternative learning rule and derive the theory. In a noiseless case, the learning speed does not depend on the correlation and is proportional to the number of inputs used in an update. The learning speed is identical to that of the gradient method with uncorrelated inputs. On the other hand, when there is noise, the larger the correlation among inputs is, the slower the learning speed is and the larger the residual generalization error is.

A method was developed for automatic recognition of syllable tone types in continuous speech of Mandarin by integrating two techniques, tone nucleus modeling and neural network classifier. The tone nucleus modeling considers a syllable F0 contour as consisting of three parts: onset course, tone nucleus, and offset course. Two courses are transitions from/to neighboring syllable F0 contours, while the tone nucleus is intrinsic part of the F0 contour. By viewing only the tone nucleus, acoustic features less affected by neighboring syllables are obtained. When using the tone nucleus modeling, automatic detection of tone nucleus comes crucial. An improvement was added to the original detection method. Distinctive acoustic features for tone types are not limited to F0 contours. Other prosodic features, such as waveform power and syllable duration, are also useful for tone recognition. Their heterogeneous features are rather difficult to be handled simultaneously in hidden Markov models (HMM), but are easy in neural networks. We adopted multi-layer perceptron (MLP) as a neural network. Tone recognition experiments were conducted for speaker dependent and independent cases. In order to show the effect of integration, experiments were conducted also for two baselines: HMM classifier with tone nucleus modeling, and MLP classifier viewing entire syllable instead of tone nucleus. The integrated method showed 87.1% of tone recognition rate in speaker dependent case, and 80.9% in speaker independent case, which was about 10% relative error reduction as compared to the baselines.

For more efficient data transmissions, a new MLP/BP-based channel equalizer is proposed to compensate for multi-path fading in wireless applications. In this work, for better system performance, we apply the soft output and the soft feedback structure as well as the soft decision channel decoding. Moreover, to improve packet error rate (PER) and bit error rate (BER), we search for the optimal scaling factor of the transfer function in the output layer of the MLP/BP neural networks and add small random disturbances to the training data. As compared with the conventional MLP/BP-based DFEs and the soft output MLP/BP-based DFEs, the proposed MLP/BP-based soft DFEs under multi-path fading channels can improve over 3-0.6 dB at PER=10-1 and over 3.3-0.8 dB at BER=10-3.

The family of Quasi-Additive (QA) algorithms is a natural generalization of the perceptron learning, which is a kind of on-line learning having two parameter vectors: One is an accumulation of input vectors and the other is a weight vector for prediction associated with the former by a nonlinear function. We show that the vectors have a dually-flat structure from the information-geometric point of view, and this representation makes it easier to discuss the convergence properties.

This paper proposes a calibration method for catadioptric camera systems consisting of a mirror whose reflecting surface is the surface of revolution and a perspective camera as typified by HyperOmni Vision. The proposed method is based on conventional camera calibration and mirror posture estimation. Many methods for camera calibration have been proposed and during the last decade, methods for catadioptric camera calibration have also been proposed. The main problem with catadioptric camera calibration is that the degree of freedom of mirror posture is limited or the accuracy of the estimated parameters is inadequate due to nonlinear optimization. On the other hand, our method can estimate five degrees of freedom of mirror posture and is free from the volatility of nonlinear optimization. The mirror posture has five degrees of freedom, because the mirror surface has a surface of revolution. Our method uses the mirror boundary and can estimate up to four mirror postures. We apply an extrinsic parameter calibration method based on conic fitting for this estimation method. Because an estimate of the mirror posture is not unique, we also propose a selection method for finding the best one. By using the conic-based analytical method we can avoid the initial value problem arising from nonlinear optimization. We conducted experiments on synthesized images and real images to evaluate the performance of our method, and discuss its accuracy.

We address issues for improving hands-free speech enhancement and speech recognition performance in different car environments using a single distant microphone. This paper describes a new single-channel in-car speech enhancement method that estimates the log spectra of speech at a close-talking microphone based on the nonlinear regression of the log spectra of noisy signal captured by a distant microphone and the estimated noise. The proposed method provides significant overall quality improvements in our subjective evaluation on the regression-enhanced speech, and performed best in most objective measures. Based on our isolated word recognition experiments conducted under 15 real car environments, the proposed adaptive nonlinear regression approach shows an advantage in average relative word error rate (WER) reductions of 50.8% and 13.1%, respectively, compared to original noisy speech and ETSI advanced front-end (ETSI ES 202 050).

An alaryngeal speech enhancement system is proposed to improve the intelligibility and quality of speech signals generated by an artificial larynx transducer (ALT). Proposed system identifies the voiced segments of alaryngeal speech signal, by using pattern recognition methods, and replaces these by their equivalent voiced segments of normal speech. Evaluation results show that proposed system provides a fairly good improvement of the quality and intelligibility of ALT generated speech.