The Canonical Polyadic Decomposition (CPD) is the tensor analog of the Singular Value Decomposition (SVD) for a matrix and has many data science applications including signal processing and machine learning. For the CPD, the Alternating Least Squares (ALS) algorithm has been used extensively. Although the ALS algorithm is simple, it is sensitive to a noise of a data tensor in the applications. In this paper, we propose a novel strategy to realize the noise suppression for the CPD. The proposed strategy is decomposed into two steps: (Step 1) denoising the given tensor and (Step 2) solving the exact CPD of the denoised tensor. Step 1 can be realized by solving a structured low-rank approximation with the Douglas-Rachford splitting algorithm and then Step 2 can be realized by solving the simultaneous diagonalization of a matrix tuple constructed by the denoised tensor with the DODO method. Numerical experiments show that the proposed algorithm works well even in typical cases where the ALS algorithm suffers from the so-called bottleneck/swamp effect.

Approximate Simultaneous Diagonalization (ASD) is a problem to find a common similarity transformation which approximately diagonalizes a given square-matrix tuple. Many data science problems have been reduced into ASD through ingenious modelling. For ASD, the so-called Jacobi-like methods have been extensively used. However, the methods have no guarantee to suppress the magnitude of off-diagonal entries of the transformed tuple even if the given tuple has an exact common diagonalizer, i.e., the given tuple is simultaneously diagonalizable. In this paper, to establish an alternative powerful strategy for ASD, we present a novel two-step strategy, called Approximate-Then-Diagonalize-Simultaneously (ATDS) algorithm. The ATDS algorithm decomposes ASD into (Step 1) finding a simultaneously diagonalizable tuple near the given one; and (Step 2) finding a common similarity transformation which diagonalizes exactly the tuple obtained in Step 1. The proposed approach to Step 1 is realized by solving a Structured Low-Rank Approximation (SLRA) with Cadzow's algorithm. In Step 2, by exploiting the idea in the constructive proof regarding the conditions for the exact simultaneous diagonalizability, we obtain an exact common diagonalizer of the obtained tuple in Step 1 as a solution for the original ASD. Unlike the Jacobi-like methods, the ATDS algorithm has a guarantee to find an exact common diagonalizer if the given tuple happens to be simultaneously diagonalizable. Numerical experiments show that the ATDS algorithm achieves better performance than the Jacobi-like methods.

In this paper, we introduce the following m-layered hard constrained convex feasibility problem HCF(m): Find a point u m, where 0:=H (a real Hilbert space), i: = arg min gi(i-1) and gi(u):=wi,jd 2(u,Ci,j) are defined for (i) nonempty closed convex sets Ci,jH and (ii) weights wi,j > 0 satisfying wi,j=1 (i {1,,m}, j {1,,Mi}. This problem is regarded as a natural extension of the standard convex feasibility problem: find a point u Ci, where Ci H (i {1,, M}) are closed convex sets. Unlike the standard problem, HCF(m) can handle the inconsistent case; i.e., i,j Ci,j = , which unfortunately arises in many signal processing, estimation and design problems. As an application of the hybrid steepest descent method for the asymptotically shrinking nonexpansive mapping, we present an algorithm, based on the use of the metric projections onto Ci,j, which generates a sequence (un) satisfying limn d(un,3) = 0 (for M1 = 1) when at least one of C1,1 or C2,j's is bounded and H is finite dimensional. An application of the proposed algorithm to the pulse shaping problem is given to demonstrate the great flexibility of the method.

In this paper, we propose a method of linear time-varying filtering of discrete time signals. The objective of this method is to derive a component, of an input signal, whose alias-free generalized discrete time-frequency distribution [Jeong & Williams 1992] concentrates on a specific region of a time-frequency plane. The method is essentially realized by computing an orthogonal projection of an input onto a subspace that is spanned by orthonormal signals, whose distributions concentrate on the region. We show that such orthonormal signals can be derived as eigenvectors of a matrix whose components are explicitly expressed by using the kernel of the distribution and the regions. This result shows that we can design such a filter prior to processing of the input if the specific region is given as a priori. This result is a generalization of [Hlawatsch & Kozek 1994], that is originally derived for the continuous Wigner distributions, to the discrete distributions.

In this paper, we propose two adaptive filtering schemes for Stereophonic Acoustic Echo Cancellation (SAEC), which are based on the adaptive projected subgradient method (Yamada et al., 2003). To overcome the so-called non-uniqueness problem, the schemes utilize a certain preprocessing technique which generates two different states of input signals. The first one simultaneously uses, for fast convergence, data from two states of inputs, meanwhile the other selects, for stability, data based on a simple min-max criteria. In addition to the above difference, the proposed schemes commonly enjoy (i) robustness against noise by introducing the stochastic property sets, and (ii) only linear computational complexity, since it is free from solving systems of linear equations. Numerical examples demonstrate that the proposed schemes achieve, even in noisy situations, compared with the conventional technique, (i) much faster and more stable convergence in the learning process as well as (ii) lower level mis-identification of echo paths and higher level Echo Return Loss Enhancement (ERLE) around the steady state.

In this paper, we propose a novel higher order time-frequency distribution (GDH) for a discrete time signal. This distribution is defined over the original discrete time-frequency grids through a delicate discretization of an equivalent expression of a higher order distribution, for a continuous time signal, in [4]. We also present a constructive design method, for the kernel of the GDH, by which the distribution satisfies (i) the alias free condition as well as (ii) the marginal conditions. Numerical examples show that the proposed distributions reasonably suppress the artifacts which are observed severely in the Wigner distribution and its simple higher order generalization.

A robust adaptive filtering algorithm was established recently (I. Yamada, K. Slavakis, K. Yamada 2002) based on the interactive use of statistical noise information and the ideas developed originally for efficient algorithmic solutions to the convex feasibility problems. The algorithm is computationally efficient and robust to noise because it requires only an iterative parallel projection onto a series of closed half spaces highly expected to contain the unknown system to be identified and is free from the computational load of solving a system of linear equations. In this letter, we show the potential applicability of the adaptive algorithm to the identification problem for the second order Volterra systems. The numerical examples demonstrate that a straightforward application of the algorithm to the problem soundly realizes fast and stable convergence for highly colored excited speech like input signals in possibly noisy environments.

Solving image recovery problems requires the use of some efficient regularizations based on a priori information with respect to the unknown original image. Naturally, we can assume that an image is modeled as the sum of smooth, edge, and texture components. To obtain a high quality recovered image, appropriate regularizations for each individual component are required. In this paper, we propose a novel image recovery technique which performs decomposition and recovery simultaneously. We formulate image recovery as a nonsmooth convex optimization problem and design an iterative scheme based on the alternating direction method of multipliers (ADMM) for approximating its global minimizer efficiently. Experimental results reveal that the proposed image recovery technique outperforms a state-of-the-art method.

In this paper, we propose a method that recovers a smooth high-resolution image from several blurred and roughly quantized low-resolution images. For compensation of the quantization effect we introduce measurements of smoothness, Huber function that is originally used for suppression of block noises in a JPEG compressed image [Schultz & Stevenson '94] and a smoothed version of total variation. With a simple operator that approximates the convex projection onto constraint set defined for each quantized image [Hasegawa et al. '05], we propose a method that minimizes these cost functions, which are smooth convex functions, over the intersection of all constraint sets, i.e. the set of all images satisfying all quantization constraints simultaneously, by using hybrid steepest descent method [Yamada & Ogura '04]. Finally in the numerical example we compare images derived by the proposed method, Projections Onto Convex Sets (POCS) based conventinal method, and generalized proposed method minimizing energy of output of Laplacian.

In this paper, we propose a simple method to find the optimal rational function, with a fixed denominator, which minimizes an integral of polynomially weighted squared error to given analytic function. Firstly, we present a generalization of the Walsh's theorem. By using the knowledge on the zeros of the fixed denominator, this theorem characterizes the optimal rational function with a system of linear equations on the coefficients of its numerator polynomial. Moreover when the analytic function is specially given as a polynomial, we show that the optimal numerator can be derived without using any numerical integration or any root finding technique. Numerical examples demonstrate the practical applicability of the proposed method.

We consider a unified approach to the tracking analysis of adaptive filters with error and matrix data nonlinearities. Using energy-conservation arguments, we not only derive earlier results in a unified manner, but we also obtain new performance results for more general adaptive algorithms without requiring the restriction of the regression data to a particular distribution. Numerical simulations support the theoretical results.

In this paper, we present an efficient downlink power control scheme, for wireless networks, based on two key ideas: (i) global-local fixed-point-approximation technique (GLOFPAT) and (ii) bottleneck removal criterion (BRC). The proposed scheme copes with all scenarios including infeasible case where no power allocation can provide all multiple accessing users with target quality of service (QoS). For feasible case, the GLOFPAT efficiently computes a desired power allocation which corresponds to the allocation achieved by conventional algorithms. For infeasible case, the GLOFPAT offers valuable information to detect bottleneck users, to be removed based on the BRC, which deteriorate overall QoS. The GLOFPAT is a mathematically-sound distributed algorithm approximating desired power allocation as a unique fixed-point of an isotone mapping. The unique fixed-point of the global mapping is iteratively computed by fixed-point-approximations of multiple distributed local mappings, which can be computed in parallel by base stations respectively. For proper detection of bottleneck users, complete analysis of the GLOFPAT is presented with aid of the Tarski's fixed-point theorem. Extensive simulations demonstrate that the proposed scheme converges faster than the conventional algorithm and successfully increases the number of happy users receiving target QoS.

Recently, Kundur and Hatzinakos showed that a linear restoration filter designed by using the almost obvious a priori knowledge on the original image, such as (i) nonnegativity of the true image and (ii) the smallest rectangle encompassing the original object, can realize a remarkable performance for a blind image deconvolution problem. In this paper, we propose a new set-theoretic blind image deconvolution scheme based on a recently developed convex projection technique called Hybrid Steepest Descent Method (HSDM), where some partial information can be utilized set-theoretically by parallel projections onto convex sets while the others are incorporated in a cost function to be minimized by a steepest descent method. Numerical comparisons with the standard set-theoretic scheme based on POCS illustrate the effectiveness of the proposed scheme.

This paper presents a unified treatment of the tracking analysis of adaptive filters with data normalization and error nonlinearities. The approach we develop is based on the celebrated energy-conservation framework, which investigates the energy flow through each iteration of an adaptive filter. Aside from deriving earlier results in a unified manner, we obtain new performance results for more general filters without restricting the regression data to a particular distribution. Simulations show good agreement with the theoretical findings.

This paper proposes an associative memory neural network whose limiting state is the nearest point in a polyhedron from a given input. Two implementations of the proposed associative memory network are presented based on Dykstra's algorithm and a fixed point theorem for nonexpansive mappings. By these implementations, the set of all correctable errors by the network is characterized as a dual cone of the polyhedron at each pattern to be memorized, which leads to a simple amplifying technique to improve the error correction capability. It is shown by numerical examples that the proposed associative memory realizes much better error correction performance than the conventional one based on POCS at the expense of the increase of necessary number of iterations in the recalling stage.

In this paper, we propose a unified algebraic design of the generalized Moreau enhancement matrix (GME matrix) for the Linearly involved Generalized-Moreau-Enhanced (LiGME) model. The LiGME model has been established as a framework to construct linearly involved nonconvex regularizers for sparsity (or low-rank) aware estimation, where the design of GME matrix is a key to guarantee the overall convexity of the model. The proposed design is applicable to general linear operators involved in the regularizer of the LiGME model, and does not require any eigendecomposition or iterative computation. We also present an application of the LiGME model with the proposed GME matrix to a group sparsity aware least squares estimation problem. Numerical experiments demonstrate the effectiveness of the proposed GME matrix in the LiGME model.

We propose the multi-domain adaptive learning that enables us to find a point meeting possibly time-varying specifications simultaneously in multiple domains, e.g. space, time, frequency, etc. The novel concept is based on the idea of feasibility splitting -- dealing with feasibility in each individual domain. We show that the adaptive projected subgradient method (Yamada, 2003) realizes the multi-domain adaptive learning by employing (i) a projected gradient operator with respect to a ‘fixed’ proximity function reflecting the time-invariant specifications and (ii) a subgradient projection with respect to ‘time-varying’ objective functions reflecting the time-varying specifications. The resulting algorithm is suitable for real-time implementation, because it requires no more than metric projections onto closed convex sets each of which accommodates the specification in each domain. A convergence analysis and numerical examples are presented.

In this letter, we remark a well-known nonlinear filtering technique realize immediate effect to suppress the influence of the additive measurement noise in the input to a set theoretic linear blind deconvolution scheme. Numerical examples show ε-separating nonlinear pre-filtering techniques work suitably to this noisy blind deconvolution problem.

It is thought that we have generally succeeded in establishing learning algorithms for neural networks, such as the back-propagation algorithm. However two major issues remain to be solved. First, there are possibilities of being trapped at a local minimum in learning. Second, the convergence rate is too slow. Chang and Ghaffar proposed to add a new hidden node, whenever stopping at a local minimum, and restart to train the new net until the error converges to zero. Their method designs newly generated weights so that the new net after introducing a new hidden node has less error than that at the original local minimum. In this paper, we propose a new method that improves their convergence rate. Our proposed method is expected to give a lower system error and a larger error gradient magnitude than their method at a starting point of the new net, which leads to a faster convergence rate. Actually, it is shown through numerical examples that the proposed method gives a much better performance than the conventional Chang and Ghaffar's method.