This paper presents a unified treatment of the tracking analysis of adaptive filters with data normalization and error nonlinearities. The approach we develop is based on the celebrated energy-conservation framework, which investigates the energy flow through each iteration of an adaptive filter. Aside from deriving earlier results in a unified manner, we obtain new performance results for more general filters without restricting the regression data to a particular distribution. Simulations show good agreement with the theoretical findings.

This paper proposes an associative memory neural network whose limiting state is the nearest point in a polyhedron from a given input. Two implementations of the proposed associative memory network are presented based on Dykstra's algorithm and a fixed point theorem for nonexpansive mappings. By these implementations, the set of all correctable errors by the network is characterized as a dual cone of the polyhedron at each pattern to be memorized, which leads to a simple amplifying technique to improve the error correction capability. It is shown by numerical examples that the proposed associative memory realizes much better error correction performance than the conventional one based on POCS at the expense of the increase of necessary number of iterations in the recalling stage.

In this paper, we propose a unified algebraic design of the generalized Moreau enhancement matrix (GME matrix) for the Linearly involved Generalized-Moreau-Enhanced (LiGME) model. The LiGME model has been established as a framework to construct linearly involved nonconvex regularizers for sparsity (or low-rank) aware estimation, where the design of GME matrix is a key to guarantee the overall convexity of the model. The proposed design is applicable to general linear operators involved in the regularizer of the LiGME model, and does not require any eigendecomposition or iterative computation. We also present an application of the LiGME model with the proposed GME matrix to a group sparsity aware least squares estimation problem. Numerical experiments demonstrate the effectiveness of the proposed GME matrix in the LiGME model.

We propose the multi-domain adaptive learning that enables us to find a point meeting possibly time-varying specifications simultaneously in multiple domains, e.g. space, time, frequency, etc. The novel concept is based on the idea of feasibility splitting -- dealing with feasibility in each individual domain. We show that the adaptive projected subgradient method (Yamada, 2003) realizes the multi-domain adaptive learning by employing (i) a projected gradient operator with respect to a ‘fixed’ proximity function reflecting the time-invariant specifications and (ii) a subgradient projection with respect to ‘time-varying’ objective functions reflecting the time-varying specifications. The resulting algorithm is suitable for real-time implementation, because it requires no more than metric projections onto closed convex sets each of which accommodates the specification in each domain. A convergence analysis and numerical examples are presented.

In this letter, we remark a well-known nonlinear filtering technique realize immediate effect to suppress the influence of the additive measurement noise in the input to a set theoretic linear blind deconvolution scheme. Numerical examples show ε-separating nonlinear pre-filtering techniques work suitably to this noisy blind deconvolution problem.

It is thought that we have generally succeeded in establishing learning algorithms for neural networks, such as the back-propagation algorithm. However two major issues remain to be solved. First, there are possibilities of being trapped at a local minimum in learning. Second, the convergence rate is too slow. Chang and Ghaffar proposed to add a new hidden node, whenever stopping at a local minimum, and restart to train the new net until the error converges to zero. Their method designs newly generated weights so that the new net after introducing a new hidden node has less error than that at the original local minimum. In this paper, we propose a new method that improves their convergence rate. Our proposed method is expected to give a lower system error and a larger error gradient magnitude than their method at a starting point of the new net, which leads to a faster convergence rate. Actually, it is shown through numerical examples that the proposed method gives a much better performance than the conventional Chang and Ghaffar's method.

This paper presents a new adaptive minor subspace extraction algorithm based on an idea of Peng and Yi ('07) for approximating the single minor eigenvector of a covariance matrix. By utilizing the idea inductively in the nested orthogonal complement subspaces, the proposed algorithm succeeds to relax the numerical sensitivity which has been annoying conventional adaptive minor subspace extraction algorithms for example, Oja algorithm ('82) and its stabilized version: O-Oja algorithm ('02). Simulation results demonstrate that the proposed algorithm realizes more stable convergence than O-Oja algorithm.

For the nonlinear acoustic echo cancellation, we present an algorithm to estimate the threshold of the clipping effect and the room impulse response vector by suppressing their time-varying cost function. A common way to suppress the time-varying cost function of a pair of parameters is to alternatingly minimize the function with respect to each parameter while keeping the other fixed, which we refer to as adaptive alternating minimization. However, since the cost function for the threshold is nonconvex, the conventional methods approximate the exact minimizations by gradient descent updates, which causes serious degradation of the estimation accuracy in some occasions. In this paper, by exploring the fact that the cost function for the threshold becomes piecewise quadratic, we propose to exactly minimize the cost function for the threshold in a closed form while suppressing the cost function for the impulse response vector in an online manner, which we call exact-online adaptive alternating minimization. The proposed method is expected to approximate more efficiently the adaptive alternating minimization strategy than the conventional methods. Numerical experiments demonstrate the efficacy of the proposed method.

This paper presents a novel blind multiple access interference (MAI) suppression filter in DS/CDMA systems. The filter is adaptively updated by parallel projections onto a series of convex sets. These sets are defined based on the received signal as well as a priori knowledge about the desired user's signature. In order to achieve fast convergence and good performance at steady state, the adaptive projected subgradient method (Yamada et al., 2003) is applied. The proposed scheme also jointly estimates the desired signal amplitude and the filter coefficients based on an approximation of an EM type algorithm, following the original idea proposed by Park and Doherty, 1997. Simulation results highlight the fast convergence behavior and good performance at steady state of the proposed scheme.

In this paper, we propose a projection based design of near perfect reconstruction QMF banks. An advantage of this method is that additional design specifications are easily implemented by defining new convex sets. To apply convex projection technique, the main difficulty is how to approximate the design specifications by some closed convex sets. In this paper, introducing a notion of Magnitude Product Space where a pair of magnitude responses of analysis filters is expressed as a point, we approximate design requirements of QMF banks by multiple closed convex sets in this space. The proposed method iteratively applies a convex projection technique, Hybrid Steepest Descent Method, to find a point corresponding to the optimal analysis filters at each stage, where the closed convex sets are dynamically improved. Design examples show that the proposed design method leads to significant improvement over conventional design methods.

In this paper, we propose a use of the group sparsity in adaptive learning of second-order Volterra filters for the nonlinear acoustic echo cancellation problem. The group sparsity indicates sparsity across the groups, i.e., a vector is separated into some groups, and most of groups only contain approximately zero-valued entries. First, we provide a theoretical evidence that the second-order Volterra systems tend to have the group sparsity under natural assumptions. Next, we propose an algorithm by applying the adaptive proximal forward-backward splitting method to a carefully designed cost function to exploit the group sparsity effectively. The designed cost function is the sum of the weighted group l1 norm which promotes the group sparsity and a weighted sum of squared distances to data-fidelity sets used in adaptive filtering algorithms. Finally, Numerical examples show that the proposed method outperforms a sparsity-aware algorithm in both the system-mismatch and the echo return loss enhancement.

This paper presents two novel blind set-theoretic adaptive filtering algorithms for suppressing "Multiple Access Interference (MAI)," which is one of the central burdens in DS/CDMA systems. We naturally formulate the problem of MAI suppression as an asymptotic minimization of a sequence of cost functions under some linear constraint defined by the desired user's signature. The proposed algorithms embed the constraint into the direction of update, and thus the adaptive filter moves toward the optimal filter without stepping away from the constraint set. In addition, using parallel processors, the proposed algorithms attain excellent performance with linear computational complexity. Geometric interpretation clarifies an advantage of the proposed methods over existing methods. Simulation results demonstrate that the proposed algorithms achieve (i) much higher speed of convergence with rather better bit error rate performance than other blind methods and (ii) much higher speed of convergence than the non-blind NLMS algorithm (indeed, the speed of convergence of the proposed algorithms is comparable to the non-blind RLS algorithm).

In this paper, we present a novel iterative MPEG super-resolution method based on an embedded constraint version of Adaptive projected subgradient method [Yamada & Ogura 2003]. We propose an efficient operator that approximates convex projection onto a set characterizing framewise quantization, whereas a conventional method can only handle a convex projection defined for each DCT coefficient of a frame. By using the operator, the proposed method generates a sequence that efficiently approaches to a solution of super-resolution problem defined in terms of quantization error of MPEG compression.

This paper presents a closed form solution to a problem of constructing the best lower bound of a convex function under certain conditions. The function is assumed (I) bounded below by -ρ, and (II) differentiable and its derivative is Lipschitz continuous with Lipschitz constant L. To construct the lower bound, it is also assumed that we can use the values ρ and L together with the values of the function and its derivative at one specified point. By using the proposed lower bound, we derive a computationally efficient deep monotone approximation operator to the level set of the function. This operator realizes better approximation than subgradient projection which has been utilized, as a monotone approximation operator to level sets of differentiable convex functions as well as nonsmooth convex functions. Therefore, by using the proposed operator, we can improve many signal processing algorithms essentially based on the subgradient projection.

Recently, a great deal of effort has been devoted to the design problem of "constrained least squares M-D FIR filter" because a significant improvement of the squared error is expected by a slight relaxation of the minimax error condition. Unfortunately, no design method has been reported, which has some theoretical guarantee of the convergence to the optimal solution. In this paper, we propose a class of novel design methods of "constrained least squares M-D FIR filter. " The most remarkable feature is that all of the proposed methods have theoretical guarantees of convergences to the unique optimal solution under any consistent set of prescribed maximal error conditions. The proposed methods are based on "convex projection techniques" that computes the metric projection onto the intersection of multiple closed convex sets in real Hilbert space. Moreover, some of the proposed methods can still be applied even for the problem with any inconsistent set of maximal error conditions. These lead to the unique optimal solution over the set of all filters that attain the least sum of squared distances to all constraint sets.

The Canonical Polyadic Decomposition (CPD) is the tensor analog of the Singular Value Decomposition (SVD) for a matrix and has many data science applications including signal processing and machine learning. For the CPD, the Alternating Least Squares (ALS) algorithm has been used extensively. Although the ALS algorithm is simple, it is sensitive to a noise of a data tensor in the applications. In this paper, we propose a novel strategy to realize the noise suppression for the CPD. The proposed strategy is decomposed into two steps: (Step 1) denoising the given tensor and (Step 2) solving the exact CPD of the denoised tensor. Step 1 can be realized by solving a structured low-rank approximation with the Douglas-Rachford splitting algorithm and then Step 2 can be realized by solving the simultaneous diagonalization of a matrix tuple constructed by the denoised tensor with the DODO method. Numerical experiments show that the proposed algorithm works well even in typical cases where the ALS algorithm suffers from the so-called bottleneck/swamp effect.

Approximate Simultaneous Diagonalization (ASD) is a problem to find a common similarity transformation which approximately diagonalizes a given square-matrix tuple. Many data science problems have been reduced into ASD through ingenious modelling. For ASD, the so-called Jacobi-like methods have been extensively used. However, the methods have no guarantee to suppress the magnitude of off-diagonal entries of the transformed tuple even if the given tuple has an exact common diagonalizer, i.e., the given tuple is simultaneously diagonalizable. In this paper, to establish an alternative powerful strategy for ASD, we present a novel two-step strategy, called Approximate-Then-Diagonalize-Simultaneously (ATDS) algorithm. The ATDS algorithm decomposes ASD into (Step 1) finding a simultaneously diagonalizable tuple near the given one; and (Step 2) finding a common similarity transformation which diagonalizes exactly the tuple obtained in Step 1. The proposed approach to Step 1 is realized by solving a Structured Low-Rank Approximation (SLRA) with Cadzow's algorithm. In Step 2, by exploiting the idea in the constructive proof regarding the conditions for the exact simultaneous diagonalizability, we obtain an exact common diagonalizer of the obtained tuple in Step 1 as a solution for the original ASD. Unlike the Jacobi-like methods, the ATDS algorithm has a guarantee to find an exact common diagonalizer if the given tuple happens to be simultaneously diagonalizable. Numerical experiments show that the ATDS algorithm achieves better performance than the Jacobi-like methods.

In this paper, we introduce the following m-layered hard constrained convex feasibility problem HCF(m): Find a point u m, where 0:=H (a real Hilbert space), i: = arg min gi(i-1) and gi(u):=wi,jd 2(u,Ci,j) are defined for (i) nonempty closed convex sets Ci,jH and (ii) weights wi,j > 0 satisfying wi,j=1 (i {1,,m}, j {1,,Mi}. This problem is regarded as a natural extension of the standard convex feasibility problem: find a point u Ci, where Ci H (i {1,, M}) are closed convex sets. Unlike the standard problem, HCF(m) can handle the inconsistent case; i.e., i,j Ci,j = , which unfortunately arises in many signal processing, estimation and design problems. As an application of the hybrid steepest descent method for the asymptotically shrinking nonexpansive mapping, we present an algorithm, based on the use of the metric projections onto Ci,j, which generates a sequence (un) satisfying limn d(un,3) = 0 (for M1 = 1) when at least one of C1,1 or C2,j's is bounded and H is finite dimensional. An application of the proposed algorithm to the pulse shaping problem is given to demonstrate the great flexibility of the method.

In this paper, we propose a method of linear time-varying filtering of discrete time signals. The objective of this method is to derive a component, of an input signal, whose alias-free generalized discrete time-frequency distribution [Jeong & Williams 1992] concentrates on a specific region of a time-frequency plane. The method is essentially realized by computing an orthogonal projection of an input onto a subspace that is spanned by orthonormal signals, whose distributions concentrate on the region. We show that such orthonormal signals can be derived as eigenvectors of a matrix whose components are explicitly expressed by using the kernel of the distribution and the regions. This result shows that we can design such a filter prior to processing of the input if the specific region is given as a priori. This result is a generalization of [Hlawatsch & Kozek 1994], that is originally derived for the continuous Wigner distributions, to the discrete distributions.