This paper proposes an alternative learning algorithm for a stereophonic acoustic echo canceller without pre-processing which can identify the correct echo-paths. By dividing the filter coefficients into the former/latter parts and updating them alternatively, conditions both for unique solution and for perfect echo cancellation are satisfied. The learning for each part is switched from one part to the other when that part converges. Convergence analysis clarifies the condition for correct echo-path identification. For fast and stable convergence, a convergence detection and an adaptive step-size are introduced. The modification amount of the filter coefficients determines the convergence state and the step-size. Computer simulations show 10 dB smaller filter coefficient error than those of the conventional algorithms without pre-processing.

FeedForward (FF-) Blind Source Separation (BSS) systems have some degree of freedom in the solution space. Therefore, signal distortion is likely to occur. First, a criterion for the signal distortion is discussed. Properties of conventional methods proposed to suppress the signal distortion are analyzed. Next, a general condition for complete separation and distortion-free is derived for multi-channel FF-BSS systems. This condition is incorporated in learning algorithms as a distortion-free constraint. Computer simulations using speech signals and stationary colored signals are performed for the conventional methods and for the new learning algorithms employing the proposed distortion-free constraint. The proposed method can well suppress signal distortion, while maintaining a high source separation performance.

This paper proposes a new combined fast algorithm for transversal adaptive filters. The fast transversal filter (FTF) algorithm and the normalized LMS (NLMS) are combined in the following way. In the initialization period, the FTF is used to obtain fast convergence. After converging, the algorithm is switched to the NLMS algorithm because the FTF cannot be used for a long time due to its numerical instability. Nonstationary environment, that is, time varying unknown system for instance, is classified into three categories: slow time varying, fast time varying and sudden time varying systems. The NLMS algorithm is applied to the first situation. In the latter two cases, however, the NLMS algorithm cannot provide a good performance. So, the FTF algorithm is selected. Switching between the two algorithms is automatically controlled by using the difference of the MSE sequence. If the difference exceeds a threshold, then the FTF is selected. Other wise, the NLMS is selected. Compared with the RLS algorithm, the proposed combined algorithm needs less computation, while maintaining the same performance. Furthermore, compared with the FTF algorithm, it provides numerically stable operation.

A nonlinear time series predictor was proposed, in which a nonlinear sub-predictor (NSP) and a linear sub-predictor (LSP) are combined in a cascade form. This model is called "hybrid predictor" here. The nonlinearity analysis method of the input time series was also proposed to estimate the network size. We have considered the nonlinear prediction problem as a pattern mapping one. A multi-layer neural network, which consists of sigmoidal hidden neurons and a single linear output neuron, has been employed as a nonlinear sub-predictor. Since the NSP includes nonlinear functions, it can predict the nonlinearity of the input time series. However, the prediction is not complete in some cases. Therefore, the NSP prediction error is further compensated for by employing a linear sub-predictor after the NSP. In this paper, the prediction mechanism and a role of the NSP and the LSP are theoretically and experimentally analyzed. The role of the NSP is to predict the nonlinear and some part of the linear property of the time series. The LSP works to predict the NSP prediction error. Furthermore, predictability of the hybrid predictor for noisy time series is investigated. The sigmoidal functions used in the NSP can suppress the noise effects by using their saturation regions. Computer simulations, using several kinds of nonlinear time series and other conventional predictor models, are demonstrated. The theoretical analysis of the predictor mechanism is confirmed through these simulations. Furthermore, predictability is improved by slightly expanding or shifting the input potential of the hidden neurons toward the saturation regions in the learning process.

This paper compares signal classification performance of multilayer neural networks (MLNNs) and linear filters (LFs). The MLNNs are useful for arbitrary waveform signal classification. On the other hand, LFS are useful for the signals, which are specified with frequency components. In this paper, both methods are compared based on frequency selective performance. The signals to be classified contain several frequency components. Furthermore, effects of the number of the signal samples are investigated. In this case, the frequency information may be lost to some extent. This makes the classification problems difficult. From practical viewpoint, computational complexity is also limited to the same level in both methods.IIR and FIR filters are compared. FIR filters with a direct form can save computations, which is independent of the filter order. IIR filters, on the other hand, cannot provide good signal classification deu to their phase distortion, and require a large amount of computations due to their recursive structure. When the number of the input samples is strictly limited, the signal vectors are widely distributed in the multi-dimensional signal space. In this case, signal classification by the LF method cannot provide a good performance. Because, they are designed to extract the frequency components. On the other hand, the MLNN method can form class regions in the signal vector space with high degree of freedom.

A training data selection method is proposed for multilayer neural networks (MLNNs). This method selects a small number of the training data, which guarantee both generalization and fast training of the MLNNs applied to pattern classification. The generalization will be satisfied using the data locate close to the boundary of the pattern classes. However, if these data are only used in the training, convergence is slow. This phenomenon is analyzed in this paper. Therefore, in the proposed method, the MLNN is first trained using some number of the data, which are randomly selected (Step 1). The data, for which the output error is relatively large, are selected. Furthermore, they are paired with the nearest data belong to the different class. The newly selected data are further paired with the nearest data. Finally, pairs of the data, which locate close to the boundary, can be found. Using these pairs of the data, the MLNNs are further trained (Step 2). Since, there are some variations to combine Steps 1 and 2, the proposed method can be applied to both off-line and on-line training. The proposed method can reduce the number of the training data, at the same time, can hasten the training. Usefulness is confirmed through computer simulation.

In this letter, we introduce a predictor based least square (PLS) algorithm. By involving both order- and time-update recursions, the PLS algorithm is found to have a more stable performance compared with the stable version (Version II) of the RLS algorithm shown in Ref.[1]. Nevertheless, the computational requirement is about 50% of that of the RLS algorithm. As an application, the PLS algorithm can be applied to the fast Newton transversal filters (FNTF). The FNTF algorithms suffer from the numerical instability problem if the quantities used for extending the gain vector are computed by using the fast RLS algorithms. By combing the PLS and the FNTF algorithms, we obtain a much more stable performance and a simple algorithm formulation.

Over the years, many improvements and refinements to the backpropagation learning algorithm have been reported. In this paper, a new adaptive penalty-based learning extension for the backpropagation learning algorithm and its variants is proposed. The new method initially puts pressure on artificial neural networks in order to get all outputs for all training patterns into the correct half of the output range, instead of mainly focusing on minimizing the difference between the target and actual output values. The upper bound of the penalty values is also controlled. The technique is easy to implement and computationally inexpensive. In this study, the new approach is applied to the backpropagation learning algorithm as well as the RPROP learning algorithm. The superiority of the new proposed method is demonstrated though many simulations. By applying the extension, the percentage of successful runs can be greatly increased and the average number of epochs to convergence can be well reduced on various problem instances. The behavior of the penalty values during training is also analyzed and their active role within the learning process is confirmed.

An automatic tap assignment method in sub-band adaptive filter is proposed in this letter. The number of taps of the adaptive filter in each band is controlled by the mean-squared error. The numbers of taps increase in the bands which have large errors, while they decrease in the bands having small errors, until residual errors in all the bands become the same. In this way, the number of taps in a band is roughly proportional to the length of the impulse response of the unknown system in this band. The convergence rate and the residual error are improved, in comparison with existing uniform tap assignment. Effectiveness of the proposed method has been confirmed through computer simulation.

Time series prediction is very important technology in a wide variety of fields. The actual time series contains both linear and nonlinear properties. The amplitude of the time series to be predicted is usually continuous value. For these reasons, we combine nonlinear and linear predictors in a cascade form. The nonlinear prediction problem is reduced to a pattern classification. A set of the past samples x(n-1),. . . ,x(n-N) is transformed into the output, which is the prediction of the next coming sample x(n). So, we employ a multi-layer neural network with a sigmoidal hidden layer and a single linear output neuron for the nonlinear prediction. It is called a Nonlinear Sub-Predictor (NSP). The NSP is trained by the supervised learning algorithm using the sample x(n) as a target. However, it is rather difficult to generate the continuous amplitude and to predict linear property. So, we employ a linear predictor after the NSP. An FIR filter is used for this purpose, which is called a Linear Sub-Predictor (LSP). The LSP is trained by the supervised learning algorithm using also x(n) as a target. In order to estimate the minimum size of the proposed predictor, we analyze the nonlinearity of the time series of interest. The prediction is equal to mapping a set of past samples to the next coming sample. The multi-layer neural network is good for this kind of pattern mapping. Still, difficult mappings may exist when several sets of very similar patterns are mapped onto very different samples. The degree of difficulty of the mapping is closely related to the nonlinearity. The necessary number of the past samples used for prediction is determined by this nonlinearity. The difficult mapping requires a large number of the past samples. Computer simulations using the sunspot data and the artificially generated discrete amplitude data have demonstrated the efficiency of the proposed predictor and the nonlinearity analysis.

This paper presents a new structure for noise and echo cancelers based on a combined fast abaptive algorithm. The main purpose of the new structure is to detect both the double-talk and the unknown path change. This goal is accomplished by using two adaptive filters. A main adaptive filter Fn, adjusted only in the non-double-talk period by the normalized LMS algorithm, is used for providing the canceler output. An auxiliary adaptive filter Ff, adjusted by the fast RLS algorithm, is used for detecting the double-talk and obtaining a near optimum tap-weight vector for Fn in the initialization period and whenever the unknown path has a sudden or fast change. The proposed structure is examined through computer simulation on a noise cancellation problem. Good cancellation performance and stable operation are obtained when signal is a speech corrupted by a white noise, a colored noise and another speech signal. Simulation results also show that the proposed structure is capable of distinguishing the near-end signal from the noise path change and quickly tracking this change.

This paper proposes a low-distortion noise canceller and its learning algorithm which is robust against crosstalk and is applicable for continuous sounds. The proposed canceller consists of two stages: cancellation of the crosstalk and cancellation of the noise. A recursive filter reduces the number of computations for noise cancellation stage. Separate filters for the adaptation and the filtering are introduced for crosstalk cancellation. Computer simulations show 10 dB improvement of the error power.

This paper proposes a new discrte optimization method which is mainly directed toward saving computing time for high-order FIR filters. In the proposed method, a transfer function is first approximated in a cascade form of a low-order function W(z) with pre-rounded coefficients and a high-order function F(z) with infinite precision coefficients. Second, rounded F(z) coefficients are discretely optimized so as to minimize the mean square error of the amplitude response. In other words, the roundoff error spectrum is shaped so as to be suppressed by a weighting function W(z). In order to save computing time, the error is equivalently evaluated in a time domain, and the F(z) coefficients are divided into small groups in the discrete optimization procedure. Design examples for 200 tap FIR filters demonstrate practical usefullness.

Analog signal processing is important for the following reasons. There exist many analog environments, and integrated analog circuits have several advantages over digital circuits. On the other hand, a digital approach can provide another features, such as accurate operation and programmability. Therefore, both circuits are effectively combined, resulting in high performance LSIs. This tutorial paper provides an overview for the recent and future trends in design and applications of integrated analog signal processing circuits. First, design techniques are reviewed for operational amplifier (Op-Amp), monolithic bipolar active RC circuits, switched-capacitor (SC) circuits, continuous-time MOS circuits, and analog-to-digital converter (ADC). High frequency filter realization, up to 100 MHz, has been tried by bipolar active RC circuits and GaAs circuits. Improved design techniques for SC circuits have been proposed. They include noise cancellation and building blocks with reduced sensitivity to nonideal Op-Amp performance. In order to overcome some SC circuit drawbacks due to a sampled data circuit, continuous-time MOS circuits have been proposed. Successful results have been obtained by using an automatic tuning method. A multi-stage noise shaping ADC is very useful to integrate an accurate ADC. A high signal-to-noise ratio (SNR), more than 91 dB, was obtained by the three-stage ADC, which can be applied to digital audio system. Automatic design and fabrication processes are also important aspects. Silicon compilers for SC circuits are overviewed. Systematic design rule, by which a globally optimum solution can be obtained, requires further investigation. A mixed analog/digital master slice LSI has been proposed to simplify an LSI customizing process. A voice-band MODEM LSI has been developed, resulting in good filter responses and SNR. Finally, promising applications of integrated analog circuits are briefly reviewed. Analog circuits are superior to a digital version in operating speed, power dissipation and integration density. In actuality, however, both approaches will be combined, resulting in mixed analog/digital LSIs where both circuits supplement each other's excellent features and negate drawbacks.

For system identification problems, such as noise and echo cancellation, FIR adaptive filters are mainly used for their simple adaptation and numerical stability. When the unknown system is a high-Q resonant system, having a very long impulse response, IIR adaptive filters are more efficient for reduction in the order of a transfer function. One way to realize the IIR adaptive filter is a separate form, in which the numerator and the denominator are separately realized and adjusted. In the actual applications, the order of the unknown system is not known. In this case, it is very important to estimate the total order and the order assignment on the numerator and the denominator. In this paper, effects of the order estimation error on the residual error are investigated. In this form, indirect error evaluation called "equation error" is used. Through theoretical and numerical investigation, the following results are obtained. First, under estimation of the order of the denominator causes large degradation. Second, over estimation can improve the performance. However, this improvement is saturated to some extent due to cancellation of the redundant poles and zeros. Third, the system identification error is proportional to the equation error as the adaptive filter approaching the optimum. Finally, there is possibility of recovering from the unstable state as the order assignment approaches to the optimum in an adaptive process using the equation error. Computer solutions are provided to aid in gaining insight of the order assignment and stability problem.

Single-reference and multi-reference noise canceller (SRNC and MRNC) performances are investigated based on correlation between signal and noise. Exact relations between these noise canceller performances and signal-noise correlation have not been well discussed yet. In this paper, the above relations are investigated based on theoretical, analysis and computer simulation. The normalized LMS (NLMS) algorithm is employed. Uncorrelate, partially correlated, and correlated signal and noise combinations are taken into account. Computer simulation is carried out, using real speech, white noise, real noise sound, sine wave signals, and their combinations. In the SRNC problem, spectral analysis is applied to derive the canceller output power spectrum. From the simulation results, it is proven that the SRNC performance is inversely proportional to the signal-noise correlation as expected by the theoretical analysis. From the simulation results, the MRNC performance is more sensitive to the signal-noise correlation than that of SRNC. When the signal-noise correlation is high, by using a larger number of adaptive filter taps, the noise is reduced more, and, the signal distortion is increased. This means the signal components included in the noise are canceled exactly.

The numerical properties of the recursive least squares (RLS) algorithm and its fast versions have been extensively studied. However, very few investigations are reported concerning the numerical behavior of the predictor based least squares (PLS) algorithms that provide the same least squares solutions as the RLS algorithm. This paper presents a comparative study on the numerical performances of the RLS and the backward PLS (BPLS) algorithms. Theoretical analysis of three main instability sources reported in the literature, including the overrange of the conversion factor, the loss of symmetry and the loss of positive definiteness of the inverse correlation matrix, has been done under a finite-precision arithmetic. Simulation results have confirmed the validity of our analysis. The results show that three main instability sources encountered in the RLS algorithm do not exist in the BPLS algorithm. Consequently, the BPLS algorithm provides a much more stable and robust numerical performance compared with the RLS algorithm.

The numerical property of the recursive least squares (RLS) algorithm has been extensively, studied. However, very few investigations are reported concerning the numerical behavior of the predictor-based least squares (PLS) algorithms which provide the same least squares solutions as the RLS algorithm. In Ref. [9], we gave a comparative study on the numerical performances of the RLS and the backward PLS (BPLS) algorithms. It was shown that the numerical property of the BPLS algorithm is much superior to that of the RLS algorithm under a finite-precision arithmetic because several main instability sources encountered in the RLS algorithm do not appear in the BPLS algorithm. This paper theoretically shows the stability of the BPLS algorithm by error propagation analysis. Since the time-variant nature of the BPLS algorithm, we prove the stability of the BPLS algorithm by using the method as shown in Ref. [6]. The expectation of the transition matrix in the BPLS algorithm is analyzed and its eigenvalues are shown to have values within the unit circle. Therefore we can say that the BPLS algorithm is numerically stable.