In this study, we introduce shift-invariant sparse image representations using tree-structured dictionaries. Sparse coding is a generative signal model that approximates signals by the linear combinations of atoms in a dictionary. Since a sparsity penalty is introduced during signal approximation and dictionary learning, the dictionary represents the primal structures of the signals. Under the shift-invariance constraint, the dictionary comprises translated structuring elements (SEs). The computational cost and number of atoms in the dictionary increase along with the increasing number of SEs. In this paper, we propose an algorithm for shift-invariant sparse image representation, in which SEs are learnt with a tree-structured approach. By using a tree-structured dictionary, we can reduce the computational cost of the image decomposition to the logarithmic order of the number of SEs. We also present the results of our experiments on the SE learning and the use of our algorithm in image recovery applications.

In this paper, we present a new dependency parsing method for languages which have very small annotated corpus and for which methods of segmentation and morphological analysis producing a unique (automatically disambiguated) result are very unreliable. Our method works on a morphosyntactic lattice factorizing all possible segmentation and part-of-speech tagging results. The quality of the input to syntactic analysis is hence much better than that of an unreliable unique sequence of lemmatized and tagged words. We propose an adaptation of Eisner's algorithm for finding the k-best dependency trees in a morphosyntactic lattice structure encoding multiple results of morphosyntactic analysis. Moreover, we present how to use Dependency Insertion Grammar in order to adjust the scores and filter out invalid trees, the use of language model to rescore the parse trees and the k-best extension of our parsing model. The highest parsing accuracy reported in this paper is 74.32% which represents a 6.31% improvement compared to the model taking the input from the unreliable morphosyntactic analysis tools.

A channel code is constructed using sparse matrices for stationary memoryless channels that do not necessarily have a symmetric property like a binary symmetric channel. It is also shown that the constructed code has the following remarkable properties. 1. Joint source-channel coding: Combining channel code with lossy source code, which is also constructed by sparse matrices, a simpler joint source-channel code can be constructed than that constructed by the ordinary block code. 2. Universal coding: The constructed channel code has a universal property under a specified condition.

Given a sparse linear system, A x = b, we can solve the equivalent system P A PT y = P b, x = PT y, where P is a permutation matrix. It has been known that, for example, when P is the RCMK (Reverse Cuthill-Mckee) ordering permutation, the convergence rate of the Krylov subspace method combined with the ILU-type preconditioner is often enhanced, especially if the matrix A is highly nonsymmetric. In this paper we offer a reordering heuristic for accelerating the solution of large sparse linear systems by the Krylov subspace methods with the ILUT preconditioner. It is the LRB (Line Red/Black) ordering based on the well-known 2-point Red-Black ordering. We show that for some model-like PDE (partial differential equation)s the LRB ordered FDM (Finite Difference Method)/FEM (Finite Element Method) discretization matrices require much less fill-ins in the ILUT factorizations than those of the Natural ordering and the RCMK ordering and hence, produces a more accurate preconditioner, if a high level of fill-in is used. It implies that the LRB ordering could outperform the other two orderings combined with the ILUT preconditioned Krylov subspace method if the level of fill-in is high enough. We compare the performance of our heuristic with that of the RCMK (Reverse Cuthill-McKee) ordering. Our test matrices are obtained from various standard discretizations of two-dimensional and three-dimensional model-like PDEs on structured grids by the FDM or the FEM. We claim that for the resulting matrices the performance of our reordering strategy for the Krylov subspace method combined with the ILUT preconditioner is superior to that of RCMK ordering, when the proper number of fill-in was used for the ILUT. Also, while the RCMK ordering is known to have little advantage over the Natural ordering in the case of symmetric matrices, the LRB ordering still can improve the convergence rate, even if the matrices are symmetric.

In this paper we compare various parallel preconditioners such as Point-SSOR (Symmetric Successive OverRelaxation), ILU(0) (Incomplete LU) in the Wavefront ordering, ILU(0) in the Multi-color ordering, Multi-Color Block SOR (Successive OverRelaxation), SPAI (SParse Approximate Inverse) and pARMS (Parallel Algebraic Recursive Multilevel Solver) for solving large sparse linear systems arising from two-dimensional PDE (Partial Differential Equation)s on structured grids. Point-SSOR is well-known, and ILU(0) is one of the most popular preconditioner, but it is inherently serial. ILU(0) in the Wavefront ordering maximizes the parallelism in the natural order, but the lengths of the wavefronts are often nonuniform. ILU(0) in the Multi-color ordering is a simple way of achieving a parallelism of the order N, where N is the order of the matrix, but its convergence rate often deteriorates as compared to that of natural ordering. We have chosen the Multi-Color Block SOR preconditioner combined with direct sparse matrix solver, since for the Laplacian matrix the SOR method is known to have a nondeteriorating rate of convergence when used with the Multi-Color ordering. By using block version we expect to minimize the interprocessor communications. SPAI computes the sparse approximate inverse directly by least squares method. Finally, ARMS is a preconditioner recursively exploiting the concept of independent sets and pARMS is the parallel version of ARMS. Experiments were conducted for the Finite Difference and Finite Element discretizations of five two-dimensional PDEs with large meshsizes up to a million on an IBM p595 machine with distributed memory. Our matrices are real positive, i.e., their real parts of the eigenvalues are positive. We have used GMRES(m) as our outer iterative method, so that the convergence of GMRES(m) for our test matrices are mathematically guaranteed. Interprocessor communications were done using MPI (Message Passing Interface) primitives. The results show that in general ILU(0) in the Multi-Color ordering and ILU(0) in the Wavefront ordering outperform the other methods but for symmetric and nearly symmetric 5-point matrices Multi-Color Block SOR gives the best performance, except for a few cases with a small number of processors.

This study proposes a method to decompose a signal into a set of periodic signals. The proposed decomposition method imposes a penalty on the resultant periodic subsignals in order to improve the sparsity of decomposition and avoid the overestimation of periods. This penalty is defined as the weighted sum of the l2 norms of the resultant periodic subsignals. This decomposition is approximated by an unconstrained minimization problem. In order to solve this problem, a relaxation algorithm is applied. In the experiments, decomposition results are presented to demonstrate the simultaneous detection of periods and waveforms hidden in signal mixtures.

We address pilot-aided channel estimation for a flat-fading spatially-correlated MIMO system, which employing Uniform Linear Arrays (ULA) on dual side and working in sparse scattering (multipath) environment. In case of sparse scattering, channel matrix and spatial correlation of flat-fading MIMO systems are parameterized by structure of multipaths, which is represented as Direction of Arrivals (DOAs), Direction of Departures (DODs) and complex path fading of each path. Based on this and block-fading property of channel, we design a channel estimation method via estimating multipath parameters using ESPRIT-like DOA-Matrix method which exploits shift-invariance property of ULA. The proposed method is able to obtain improved Mean-Square-Error performance than Least-Square method without prior information of spatial correlation.

In this paper, a sparse multipath channel estimation algorithm is proposed for multiple-input multiple-output (MIMO) single-carrier systems with frequency-domain decision feedback equalizer (FD-DFE). This algorithm exploits the orthogonality of an optimal MIMO preamble based on repeated, phase-rotated, Chu (RPC) sequences, leading to a dramatic reduction in computation. Furthermore, the proposed algorithm employs an improved non-iterative procedure utilizing the Generalized AIC criterion (GAIC), resulting in further computational saving and performance improvement. The proposed scheme is simulated for 802.16d SCa-PHY and SUI-5 sparse channel model under a 22 spatial multiplexing scenario, with the MIMO FD-DFE as the receiver. The result shows that the channel estimation accuracy is significantly improved, and the performance loss compared to the known channel case is only 0.7 dB at the BER of 10-3.

The most time consuming operation to verify a signature with the Elliptic Curve Digital Signature Algorithm is a multi-scalar multiplication with two scalars. Efficient methods for its computation are the Shamir method and the Interleave method, whereas the performance of those methods can be improved by using general base-2 representations of the scalars. In exchange for the speed-up, those representations require the precomputation of several points that must be stored. In the case of two precomputed points, the Interleave method and the Shamir method provide the same, optimal efficiency. In the case of more precomputed points, only the Interleave method can be sped-up in an optimal way and is currently more efficient than the Shamir method. This paper proposes a new general base-2 representation of the scalars that can be used to speed up the Shamir method. It requires the precomputation of ten points and is more efficient than any other representation that also requires ten precomputed points. Therefore, the proposed method is the first to improve the Shamir method such that it is faster than the Interleave method.

An algorithm for encoding low-density parity check (LDPC) codes is investigated. The algorithm computes parity check symbols by solving a set of sparse equations, and the triangular factorization is employed to solve the equations efficiently. It is shown analytically and experimentally that the proposed algorithm is more efficient than the Richardson's encoding algorithm if the code has a small gap.

Syntax and prosody are closely related to each other. This paper is concerned with the problem of exploiting pause information for recovering dependency structures of read Japanese sentences. Our parser can handle both symbolic information such as dependency rule and numerical information such as the probability of dependency distance of a phrase in a unified way as linguistic information. In our past work, post-phrase pause that immediately succeeds a phrase in question was employed as prosodic information. In this paper, we employed two kinds of pauses in addition to the post-phrase pause: post-post-phrase pause that immediately succeeds the phrase that follows a phrase in question, and pre-phrase pause that immediately precedes a phrase in question. By combining the three kinds of pause information linearly with the optimal combination weights that were determined experimentally, the parsing accuracy was improved compared to the case where only the post-phrase pause was used as in our previous work. Linear combination of pause and fundamental frequency information yielded further improvement of parsing accuracy.

In this paper, we propose a statistical model for detecting article errors, which Japanese learners of English often make in English writing. It is based on the three head words--the verb head, the preposition, and the noun head. To overcome the data sparseness problem, we apply the backed-off estimate to it. Experiments show that its performance (F-measure=0.70) is better than that of other methods. Apart from the performance, it has two advantages: (i) Rules for detecting article errors are automatically generated as conditional probabilities once a corpus is given; (ii) Its recall and precision rates are adjustable.

This paper focuses on the underdetermined blind source separation (BSS) of three speech signals mixed in a real environment from measurements provided by two sensors. To date, solutions to the underdetermined BSS problem have mainly been based on the assumption that the speech signals are sufficiently sparse. They involve designing binary masks that extract signals at time-frequency points where only one signal was assumed to exist. The major issue encountered in previous work relates to the occurrence of distortion, which affects a separated signal with loud musical noise. To overcome this problem, we propose combining sparseness with the use of an estimated mixing matrix. First, we use a geometrical approach to detect when only one source is active and to perform a preliminary separation with a time-frequency mask. This information is then used to estimate the mixing matrix, which allows us to improve our separation. Experimental results show that this combination of time-frequency mask and mixing matrix estimation provides separated signals of better quality (less distortion, less musical noise) than those extracted without using the estimated mixing matrix in reverberant conditions where the reverberant time (TR) was 130 ms and 200 ms. Furthermore, informal listening tests clearly show that musical noise is deeply lowered by the proposed method comparatively to the classical approaches.

We propose a robust edge detection method based on independent component analysis (ICA). It is known that most of the basis functions extracted from natural images by ICA are sparse and similar to localized and oriented receptive fields, and in the proposed edge detection method, a target image is first transformed by ICA basis functions and then the edges are detected or reconstructed with sparse components only. Furthermore, by applying a shrinkage algorithm to filter out the components of noise in the ICA domain, we can readily obtain the sparse components of the original image, resulting in a kind of robust edge detection even for a noisy image with a very low SN ratio. The efficiency of the proposed method is demonstrated by experiments with some natural images.

To overcome inefficiencies on tree-based and mesh-based scheme, overlay multicast scheme for MANET has been recently proposed to provide higher packet delivery ratio than the former as well as more efficient resource usage than the latter. However, previous all overlay multicast schemes are not designed with any considerations for dense/sparse environments resulted from unlimited movement of mobile nodes. For this reason, all packets should be transmitted in the form of unicast packet so they cannot fully make use of node's broadcast capability even though some group members are densely distributed within single-hop on the same shared media. Due to above reason, this causes extra forwarding overhead, low resource utilization as well as high packet collision. In this paper, we propose a novel hybrid overlay multicast scheme, EAOM (Environment-Aware Overlay Multicast), which uses neighboring group members' information to deliver packet with low cost. In EAOM, a group member has two modes depending on the number of neighboring group members. Under dense environment, host group model in wired network is applied. While in sparse environment, packets are delivered to each receiver along overlay DDT (Data Delivery Tree) as same as previous overlay multicast schemes. Hence, EAOM can not only remove mentioned obstacles in dense environment, but also cope with network mobility very well in sparse environment. Using simulation results, we demonstrate that EAOM has good packet delivery ratio, low control overhead as well as short end-to-end delay.

In this note, we present some results about parses of codes. First we present a sufficient condition of a bifix code to have the bounded indicator. Next we consider a proper parse, introduced notion. We prove that for a strongly infix code, the number of proper parses is at most three under some condition. We also prove that if a code X has a unique proper parse for each word under the same condition, then X is a strongly infix code.

We propose new language models to represent phrasal structures by patterns extracted from parse trees. First, modified word trigram models are proposed. They are extracted from sentences analyzed by the preprocessing of the parser with knowledge. Since sentences are analyzed to create sub-trees of a few words, these trigram models can represent relations among a few neighbor words more strongly than conventional word trigram models. Second, word pattern models are used on these modified word trigram models. The word patterns are extracted from parse trees and can represent phrasal structures and much longer word-dependency than trigram models. Experimental results show that modified trigram models are more effective than traditional trigram models and that pattern models attain slight improvements over modified trigram models. Furthermore, additional experiments show that pattern models are more effective for long sentences.

In this paper, an associative memory model with a forgetting process proposed by Mezard et al. is investigated as a means of storing sparsely encoded patterns by the SCSNA proposed by Shiino and Fukai. Similar to the case of storing non-sparse (non-biased) patterns as analyzed by Mezard et al., this sparsely encoded associative memory model is also free from a catastrophic deterioration of the memory caused by memory pattern overloading. We theoretically obtain a relationship between the storage capacity and the forgetting rate, and find that there is an optimal forgetting rate leading to the maximum storage capacity. We call this the optimal storage capacity rate. As the memory pattern firing rate decreases, the optimal storage capacity increases and the optimal forgetting rate decreases. Furthermore, we shown that the capacity rate (i.e. the ratio of the storage capacity for the conventional correlation learning rule to the optimal storage capacity) is almost constant with respect to the memory pattern firing rate.

Terrestrial radio links with sparsely distributed multipath delays can be represented by a tapped-delay line with a few significant tap coefficients. This letter presents criteria and performance of identification methods that determine channel taps with significant power. In particular, a tap identification method derived from the maximum-likelihood criterion and its closed form error probabilities are presented. Performance improvement over a previously reported scheme is quantified using the derived error probabilities.

Associative memories composed of sparsely interconnected neural networks (SINNs) are suitable for analog hardware implementation. However, the sparsely interconnected structure also gives rise to a decrease in the capability of SINNs for associative memories. Although this problem can be solved by increasing the number of interconnections, the hardware cost goes up rapidly. Therefore, we propose associative memories consisting of multilayer perceptrons (MLPs) with 3-valued weights and SINNs. It is expected that such MLPs can be realized at a lower cost than increasing interconnections in SINNs and can give each neuron in SINNs the global information of an input pattern to improve the storage capacity. Finally, it is confirmed by simulations that our proposed associative memories have good performance.