In this paper, we address issues in improving hands-free speech recognition performance in different car environments using multiple spatially distributed microphones. In the previous work, we proposed the multiple linear regression of the log spectra (MRLS) for estimating the log spectra of speech at a close-talking microphone. In this paper, the concept is extended to nonlinear regressions. Regressions in the cepstrum domain are also investigated. An effective algorithm is developed to adapt the regression weights automatically to different noise environments. Compared to the nearest distant microphone and adaptive beamformer (Generalized Sidelobe Canceller), the proposed adaptive nonlinear regression approach shows an advantage in the average relative word error rate (WER) reductions of 58.5% and 10.3%, respectively, for isolated word recognition under 15 real car environments.

In this paper we present the analog architecture and the implementation of an on-chip learning Multi Layer Perceptron network. The learning algorithm is based on Back Propagation but it exhibits increased capabilities due to local learning rate management. A prototype chip (SLANP, Self-Learning Neural Processor) has been designed and fabricated in a CMOS 0.7 µm minimum channel length technology. We report the experimental results that confirm the functionality of the chip and the soundness of the approach. The SLANP performance compare favourably with those reported in the literature.

A new algorithm for synthesizing binary threshold neural networks (BTNNs) is proposed. A binary (Boolean) input-output mapping that can be represented by minimal sum-of-product (MSP) terms is initially obtained from training data. The BTNN is then synthesized based on an MSP term grouping method. As a result, a fast and optimal realization of a BTNN can be obtained. Examples of both feedforward and recurrent BTNN synthesis used in a parallel processing architecture are given and compared with other existing methods.

Associative memories composed of sparsely interconnected neural networks (SINNs) are suitable for analog hardware implementation. However, the sparsely interconnected structure also gives rise to a decrease in the capability of SINNs for associative memories. Although this problem can be solved by increasing the number of interconnections, the hardware cost goes up rapidly. Therefore, we propose associative memories consisting of multilayer perceptrons (MLPs) with 3-valued weights and SINNs. It is expected that such MLPs can be realized at a lower cost than increasing interconnections in SINNs and can give each neuron in SINNs the global information of an input pattern to improve the storage capacity. Finally, it is confirmed by simulations that our proposed associative memories have good performance.

In this article, a hierarchical classifier is proposed for classification of ground-cover types of a satellite image of Kangaroo Island, South Australia. The image contains seven ground-cover types, which are categorized into three groups using principal component analysis. The first group contains clouds only, the second consists of sea and cloud shadow over land, and the third contains land and three types of forest. The sea and shadow over land classes are classified with 99% accuracy using a network of threshold logic units. The land and forest classes are classified by multilayer perceptrons (MLPs) using texture features and intensity values. The average performance achieved by six trained MLPs is 91%. In order to improve the classification accuracy even further, the outputs of the six MLPs were combined using several committee machines. All committee machines achieved significant improvement in performance over the multilayer perceptron classifiers, with the best machine achieving over 92% correct classification.

This paper proposes a new method of realizing internationlized domain names (iDN) and has been discussed at IETF (Internet Engineering Task Force). iDN allows a user to specify multi-lingual domain names, such as Japanese, Chinese, and Korean. iDN is a proper extension of the current domain name system. We have already developed an iDN implementation, named Global Domain Name System (GDNS). GDNS extends the usage of alias records, and gives reverse mapping information for multi-lingual domain names. This paper presents yet another method which introduces new Resource Record (RR) types to cover multi-lingual domain names. We have two new RR (Resource Record) types. The first new record is INAME and the other is IPTR. These two RR types can cover multi-lingual domain names. This paper also discusses the efficiency of DNS. Since DNS is a distributed database system, the performance depends on the method of retrieving data. This paper suggests a new retrieving method that can improve the performance of DNS remarkably.

Information geometry is applied to the manifold of neural networks called multilayer perceptrons. It is important to study a total family of networks as a geometrical manifold, because learning is represented by a trajectory in such a space. The manifold of perceptrons has a rich differential-geometrical structure represented by a Riemannian metric and singularities. An efficient learning method is proposed by using it. The parameter space of perceptrons includes a lot of algebraic singularities, which affect trajectories of learning. Such singularities are studied by using simple models. This poses an interesting problem of statistical inference and learning in hierarchical models including singularities.

A multilayer perceptron is usually considered a passive learner that only receives given training data. However, if a multilayer perceptron actively gathers training data that resolve its uncertainty about a problem being learnt, sufficiently accurate classification is attained with fewer training data. Recently, such active learning has been receiving an increasing interest. In this paper, we propose a novel active learning strategy. The strategy attempts to produce only useful training data for multilayer perceptrons to achieve accurate classification, and avoids generating redundant training data. Furthermore, the strategy attempts to avoid generating temporarily useful training data that will become redundant in the future. As a result, the strategy can allow multilayer perceptrons to achieve accurate classification with fewer training data. To demonstrate the performance of the strategy in comparison with other active learning strategies, we also propose an empirical active learning algorithm as an implementation of the strategy, which does not require expensive computations. Experimental results show that the proposed algorithm improves the classification accuracy of a multilayer perceptron with fewer training data than that for a conventional random selection algorithm that constructs a training data set without explicit strategies. Moreover, the algorithm outperforms typical active learning algorithms in the experiments. Those results show that the algorithm can construct an appropriate training data set at lower computational cost, because training data generation is usually costly. Accordingly, the algorithm proves the effectiveness of the strategy through the experiments. We also discuss some drawbacks of the algorithm.

In order to improve the efficiency of the feature extraction of backpropagation (BP) learning in layered neural networks, model switching for changing the function model without altering the map is proposed. Model switching involves map preserving reduction of units by channel fusion, or addition of units by channel installation. For reducing the model size by channel fusion, two criteria for detection of the redundant channels are addressed, and the local link weight compensations for map preservation are formulated. The upper limits of the discrepancies between the maps of the switched models are derived for use as the unified criterion in selecting the switching model candidate. In the experiments, model switching is used during the BP training of a layered network model for image texture classification, to aid its inefficiency of feature extraction. The results showed that fusion and re-installation of redundant channels, weight compensations on channel fusion for map preservation, and the use of the unified criterion for model selection are all effective for improved generalization ability and quick learning. Further, the possibility of using model switching for concurrent optimization of the model and the map will be discussed.

We apply two acceleration techniques for the backpropagation algorithm to an iterative gradient descent algorithm called the network inversion algorithm. Experimental results show that these techniques are also quite effective to decrease the number of iterations required for the detection of input vectors on the classification boundary of a multilayer perceptron.

This paper proposes a simple learning algorithm that can realize any boolean function using the three-layer binary neural networks. The algorithm has flexible learning functions. 1) moving "core" for the inputs separations,2) "don't care" settings of the separated inputs. The "don't care" inputs do not affect the successive separations. Performing numerical simulations on some typical examples, we have verified that our algorithm can give less number of hidden layer neurons than those by conventional ones.

This paper presents a neural network-based control system for Adaptive Noise Control (ANC). The control system derives a secondary signal to destructively interfere with the original noise to cut down the noise power. This paper begins with an introduction to feedback ANC systems and then describes our adaptive algorithm in detail. Three types of noise signals, recorded in destroyer, F16 airplane and MR imaging room respectively, were then applied to our noise control system which was implemented by software. We obtained an average noise power attenuation of about 20 dB. It was shown that our system performed as well as traditional DSP controllers for narrow-band noise and achieved better results for nonlinear broadband noise problems. In this paper we also present a hardware implementation method for the proposed algorithm. This hardware architecture allows fast and efficient field training in new environments and makes real-time real-life applications possible.

It is well known that offset errors in the multipliers of neural LSIs can have fatal effects on performance. The aim of this study is to understand theoretically how offset errors affect performance of neural LSIs. We have used a single-layer perceptron as an example, and compare our theoretically derived results with computer simulations. We have found that offset errors in the multipliers for the forward process can be canceled out through learning, but those for the updating process cannot be. We have examined the asymptotic behavior of learning for the updating process and derived a mathematical expression for dL, the excess of the averaged loss function L. The derived expression gives us a basis for estimating robustness with respect to the offset errors. Our analysis indicates that dL can be expressed in the form of a quadratic form of offset errors and the inverse of the Hessian matrix of L. We have found that increasing the number of synapses degrades the performacne. We have also learned that enlarging the input signal level and reducing the signal level of the desired response can be effective techniques for reducing the effects of offset errors of the updating process.

In fully connected Multilayer perceptron (MLP), all the hidden units are activated by samples from the whole input space. For complex problems, due to interference and cross coupling of hidden units' activations, the network needs many hidden units to represent the problem and the error surface becomes highly non-linear. Searching for the minimum is then complex and computationally expensive, and simple gradient descent algorithms usually fail. We propose a network, where the input space is partitioned into local sub-regions. Subsequently, a number of smaller networks are simultaneously trained by overlapping subsets of the input samples. Remarkable improvement of training efficiency as well as generalization performance of this combined network are observed through various simulations.

A sparse memory access architecture which is proposed to achieve a high-computational-speed neural-network LSI is described in detail. This architecture uses two key techniques, compressible synapse-weight neuron calculation and differential neuron operation, to reduce the number of accesses to synapse weight memories and the number of neuron calculations without incurring an accuracy penalty. The test chip based on this architecture has 96 parallel data-driven processing units and enough memory for 12,288 synapse weights. In a pattern recognition example, the number of memory accesses and neuron calculations was reduced to 0.87% that needed in the conventional method and the practical performance was 18 GCPS. The sparse memory access architecture is also effective when the synapse weights are stored in off-chip memory.

A multiple classifier system can be a powerful solution for robust pattern recognition. It is expected that the appropriate combination of multiple classifiers may reduce errors, provide robustness, and achieve higher performance. In this paper, high performance Chinese syllable recognition is presented using combinations of multiple classifiers. Chinese syllable recognition is divided into base syllable recognition (disregarding the tones) and recognition of 4 tones. For base syllable recognition, we used a combination of two multisegment vector quantization (MSVQ) classifiers based on different features (instantaneous and transitional features of speech). For tone recognition, vector quantization (VQ) classifier was first used, and was comparable to multilayer perceptron (MLP) classifier. To get robust or better performance, a combination of distortion-based classifier (VQ) and discriminant-based classifier (MLP) is proposed. The evaluations have been carried out using standard syllable database CRDB in China, and experimental results have shown that combination of multiple classifiers with different features or different methodologies can improve recognition performance. Recognition accuracy for base syllable, tone, and tonal syllable is 96.79%, 99.82% and 96.24% respectively. Since these results were evaluated on a standard database, they can be used as a benchmark that allows direct comparison against other approaches.

The capability of generalization is the most desirable property of a learning system. It is well known that to achieve good generalization, the complexity of the system should match the intrinsic complexity of the problem to be learned. In this work, introduction of fractal connection structure in nonlinear learning systems like multilayer perceptrons as a means of improving its generalization capability in classification problems has been investigated via simulation on sonar data set in underwater target classification problem. It has been found that fractally connected net has better generalization capability compared to the fully connected net and a randomly connected net of same average connectivity for proper choice of fractal dimension which controlls the average connectivity of the net.

In general, a learning machine will behave better as the number of training examples increases. It is important to know how fast and how well the behavior is improved. The average prediction error, the average of the probability that the trained machine mispredicts the output signal, is one of the most popular criteria to see the behavior. However, it is not easy to evaluate the average prediction error even in the simplest case, that is, the linear dichotomy (perceptron) case. When a continuous deterministic dichotomy machine is trained by t examples of input-output pairs produced from a realizable teacher, these examples limits the region of the parameter space which includes the true parameter. Any parameter in the region can explain the input-output behaviors of the examples. Such a region, celled the admissible region, forms in general a (curved) polyhedron in the parameter space, and it becomes smaller and smaller as the number of examples increases. The present paper studies the shape and volume of the admissible region. We use the stochastic geometrical approach to this problem. We have studied the stochastic geometrical features of the admissible region using the fact that it is dual to the convex hull the examples make in the example space. Since the admissible region is related to the average prediction error of the linear dichotomy, we derived the new upper and lower bounds of the average prediction error.

In this paper approaches to language identification based on the sequential information of phonemes are described. These approaches assume that each language can be identified from its own phoneme structure, or phonotactics. To extract this phoneme structure, we use phoneme classifiers and grammars for each language. The phoneme classifier for each language is implemented as a multi-layer perceptron trained on quasi-phonetic hand-labeled transcriptions. After training the phoneme classifiers, the grammars for each language are calculated as a set of transition probabilities for each phoneme pair. Because of the interest in automatic language identification for worldwide voice communication, we decided to use telephone speech for this study. The data for this study were drawn from the OGI (Oregon Graduate Institute)-TS (telephone speech) corpus, a standard corpus for this type of research. To investigate the basic issues of this approach, two languages, Japanese and English, were selected. The language classification algorithms are based on Viterbi search constrained by a bigram grammar and by minimum and maximum durations. Using a phoneme classifier trained only on English phonemes, we achieved 81.1% accuracy. We achieved 79.3% accuracy using a phoneme classifier trained on Japanese phonemes. Using both the English and the Japanese phoneme classifiers together, we obtained our best result: 83.3%. Our results were comparable to those obtained by other methods such as that based on the hidden Markov model.

A multi-layer perceptron (MLP) acting directly in the time-domain is applied as a speech signal enhancer, and the performance examined in the context of three common classes of degradation, namely low bit-rate CELP degradation is non-linear system degradation, additive noise, and convolution by a linear system. The investigation focuses on two topics: (i) the influence of non-linearities within the network and (ii) network topology, comparing single and multiple output structures. The objective is to examine how these characteristics influence network performance and whether this depends on the class of degradation. Experimental results show the importance of matching the enhancer to the class of degradation. In the case of the CELP coder the standard MLP with its inherently non-linear characteristics is shown to be consistently better than any equivalent linear structure (up to 3.2 dB compared with 1.6 dB SNR improvement). In contrast, when the degradation is from additive noise, a linear enhancer is always, superior.