This work presents a DNN accelerator architecture specifically designed for performing efficient inference on compressed and sparse DNN models. Leveraging the data sparsity, a runtime processing scheme is proposed to deal with the encoded weights and activations directly in the compressed domain without decompressing. Furthermore, a new data flow is proposed to facilitate the reusage of input activations across the fully-connected (FC) layers. The proposed design is implemented and verified using the Xilinx Virtex-7 FPGA. Experimental results show it achieves 1.99×, 1.95× faster and 20.38×, 3.04× more energy efficient than CPU and mGPU platforms, respectively, running AlexNet.

This paper presents a method to optimize 2-D sparse array configurations along with a technique to interpolate holes to accurately estimate the direction of arrival (DOA). Conventional 2-D sparse arrays are often defined using a closed-form representation and have the property that they can create hole-free difference co-arrays that can estimate DOAs of incident signals that outnumber the physical elements. However, this property restricts the array configuration to a limited structure and results in a significant mutual coupling effect between consecutive sensors. In this paper, we introduce an optimization-based method for designing 2-D sparse arrays that enhances flexibility of array configuration as well as DOA estimation accuracy. We also propose a method to interpolate holes in 2-D co-arrays by nuclear norm minimization (NNM) that permits holes and to extend array aperture to further enhance DOA estimation accuracy. The performance of the proposed optimum arrays is evaluated through numerical examples.

This paper proposes a high-quality computed tomography (CT) image reconstruction method from low-dose X-ray projection data. A state-of-the-art method, proposed by Xu et al., exploits dictionary learning for image patches. This method generates an overcomplete dictionary from patches of standard-dose CT images and reconstructs low-dose CT images by minimizing the sum of a data fidelity and a regularization term based on sparse representations with the dictionary. However, this method does not take characteristics of each patch, such as textures or edges, into account. In this paper, we propose to classify all patches into several classes and utilize an individual dictionary with an individual regularization parameter for each class. Furthermore, for fast computation, we introduce the orthogonality to column vectors of each dictionary. Since similar patches are collected in the same cluster, accuracy degradation by the orthogonality hardly occurs. Our simulations show that the proposed method outperforms the state-of-the-art in terms of both accuracy and speed.

Sparse modeling is one of the most active research areas in engineering and science. The technique provides solutions from far fewer samples exploiting sparsity, that is, the majority of the data are zero. This paper reviews sparse modeling in radio techniques. The first half of this paper introduces direction-of-arrival (DOA) estimation from signals received by multiple antennas. The estimation is carried out using compressed sensing, an effective tool for the sparse modeling, which produces solutions to an underdetermined linear system with a sparse regularization term. The DOA estimation performance is compared among three compressed sensing algorithms. The second half reviews channel state information (CSI) acquisitions in multiple-input multiple-output (MIMO) systems. In time-varying environments, CSI estimated with pilot symbols may be outdated at the actual transmission time. We describe CSI prediction based on sparse DOA estimation, and show excellent precoding performance when using the CSI prediction. The other topic in the second half is sparse Bayesian learning (SBL)-based channel estimation. A base station (BS) has many antennas in a massive MIMO system. A major obstacle for using the massive MIMO system in frequency-division duplex mode is an overhead for downlink CSI acquisition because we need to send many pilot symbols from the BS and to get the feedback from user equipment. An SBL-based channel estimation method can mitigate this issue. In this paper, we describe the outline of the method, and show that the technique can reduce the downlink pilot symbols.

Sparse orthogonal matrices are proposed to improve the convergence property of expectation propagation (EP) for sparse signal recovery from compressed linear measurements subject to known dense and ill-conditioned multiplicative noise. As a typical problem, this letter addresses generalized spatial modulation (GSM) in over-loaded and spatially correlated multiple-input multiple-output (MIMO) systems. The proposed sparse orthogonal matrices are used in precoding and constructed efficiently via a generalization of the fast Walsh-Hadamard transform. Numerical simulations show that the proposed sparse orthogonal precoding improves the convergence property of EP in over-loaded GSM MIMO systems with known spatially correlated channel matrices.

Diffusion least-mean-square (LMS) is a method to estimate and track an unknown parameter at multiple nodes in a network. When the unknown vector has sparsity, the sparse promoting version of diffusion LMS, which utilizes a sparse regularization term in the cost function, is known to show better convergence performance than that of the original diffusion LMS. This paper proposes a novel choice of the coefficients involved in the updates of sparse diffusion LMS using the idea of message propagation. Moreover, we optimize the proposed coefficients with respect to mean-square-deviation at the steady-state. Simulation results demonstrate that the proposed method outperforms conventional methods in terms of the convergence performance.

Expectation propagation (EP) decoding is proposed for sparse superposition coding in orthogonal frequency division multiplexing (OFDM) systems. When a randomized discrete Fourier transform (DFT) dictionary matrix is used, the EP decoding has the same complexity as approximate message-passing (AMP) decoding, which is a low-complexity and powerful decoding algorithm for the additive white Gaussian noise (AWGN) channel. Numerical simulations show that the EP decoding achieves comparable performance to AMP decoding for the AWGN channel. For OFDM systems, on the other hand, the EP decoding is much superior to the AMP decoding while the AMP decoding has an error-floor in high signal-to-noise ratio regime.

In this paper, we propose L0 norm optimization in a scrambled sparse representation domain and its application to an Encryption-then-Compression (EtC) system. We design a random unitary transform that conserves L0 norm isometry. The resulting encryption method provides a practical orthogonal matching pursuit (OMP) algorithm that allows computation in the encrypted domain. We prove that the proposed method theoretically has exactly the same estimation performance as the nonencrypted variant of the OMP algorithm. In addition, we demonstrate the security strength of the proposed secure sparse representation when applied to the EtC system. Even if the dictionary information is leaked, the proposed scheme protects the privacy information of observed signals.

In the realization of convolutional neural networks (CNNs) in resource-constrained embedded hardware, the memory footprint of weights is one of the primary problems. Pruning techniques are often used to reduce the number of weights. However, the distribution of nonzero weights is highly skewed, which makes it more difficult to utilize the underlying parallelism. To address this problem, we present SENTEI*, filter-wise pruning with distillation, to realize hardware-aware network architecture with comparable accuracy. The filter-wise pruning eliminates weights such that each filter has the same number of nonzero weights, and retraining with distillation retains the accuracy. Further, we develop a zero-weight skipping inter-layer pipelined accelerator on an FPGA. The equalization enables inter-filter parallelism, where a processing block for a layer executes filters concurrently with straightforward architecture. Our evaluation of semantic-segmentation tasks indicates that the resulting mIoU only decreased by 0.4 points. Additionally, the speedup and power efficiency of our FPGA implementation were 33.2× and 87.9× higher than those of the mobile GPU. Therefore, our technique realizes hardware-aware network with comparable accuracy.

This paper proposes a deterministic pilot pattern placement optimization scheme based on the quantum genetic algorithm (QGA) which aims to improve the performance of sparse channel estimation in orthogonal frequency division multiplexing (OFDM) systems. By minimizing the mutual incoherence property (MIP) of the sensing matrix, the pilot pattern placement optimization is modeled as the solution of a combinatorial optimization problem. QGA is used to solve the optimization problem and generate optimized pilot pattern that can effectively avoid local optima traps. The simulation results demonstrate that the proposed method can generate a sensing matrix with a smaller MIP than a random search or the genetic algorithm (GA), and the optimized pilot pattern performs well for sparse channel estimation in OFDM systems.

In this paper, we propose a secure computation of sparse coding and its application to Encryption-then-Compression (EtC) systems. The proposed scheme introduces secure sparse coding that allows computation of an Orthogonal Matching Pursuit (OMP) algorithm in an encrypted domain. We prove theoretically that the proposed method estimates exactly the same sparse representations that the OMP algorithm for non-encrypted computation does. This means that there is no degradation of the sparse representation performance. Furthermore, the proposed method can control the sparsity without decoding the encrypted signals. Next, we propose an EtC system based on the secure sparse coding. The proposed secure EtC system can protect the private information of the original image contents while performing image compression. It provides the same rate-distortion performance as that of sparse coding without encryption, as demonstrated on both synthetic data and natural images.

Web service users are overwhelmed by the amount of information presented to them and have difficulties in finding the information that they need. Therefore, a recommendation system that predicts users' taste is an essential factor for the success of businesses. However, recommendation systems require users' personal information and can thus lead to serious privacy violations. To solve this problem, many research has been conducted about protecting personal information in recommendation systems and implementing differential privacy, a privacy protection technique that inserts noise into the original data. However, previous studies did not examine the following factors in applying differential privacy to recommendation systems. First, they did not consider the sparsity of user rating information. The total number of items is much more than the number of user-rated items. Therefore, a rating matrix created for users and items will be very sparse. This characteristic renders the identification of user patterns in rating matrixes difficult. Therefore, the sparsity issue should be considered in the application of differential privacy to recommendation systems. Second, previous studies focused on protecting user rating information but did not aim to protect the lists of user-rated items. Recommendation systems should protect these item lists because they also disclose user preferences. In this study, we propose a differentially private recommendation scheme that bases on a grouping method to solve the sparsity issue and to protect user-rated item lists and user rating information. The proposed technique shows better performance and privacy protection on actual movie rating data in comparison with an existing technique.

Software defect prediction (SDP) plays a vital role in allocating testing resources reasonably and ensuring software quality. When there are not enough labeled historical modules, considerable semi-supervised SDP methods have been proposed, and these methods utilize limited labeled modules and abundant unlabeled modules simultaneously. Nevertheless, most of them make use of traditional features rather than the powerful deep feature representations. Besides, the cost of the misclassification of the defective modules is higher than that of defect-free ones, and the number of the defective modules for training is small. Taking the above issues into account, we propose a cost-sensitive and sparse ladder network (CSLN) for SDP. We firstly introduce the semi-supervised ladder network to extract the deep feature representations. Besides, we introduce the cost-sensitive learning to set different misclassification costs for defective-prone and defect-free-prone instances to alleviate the class imbalance problem. A sparse constraint is added on the hidden nodes in ladder network when the number of hidden nodes is large, which enables the model to find robust structures of the data. Extensive experiments on the AEEEM dataset show that the CSLN outperforms several state-of-the-art semi-supervised SDP methods.

In this work, we present one novel rust detection method based upon one-class classification and L2 sparse representation (SR) with decision fusion. Firstly, a new color contrast descriptor is proposed for extracting the rust features of steel structure images. Considering that the patterns of rust features are more simplified than those of non-rust ones, one-class support vector machine (SVM) classifier and L2 SR classifier are designed with these rust image features, respectively. After that, a multiplicative fusion rule is advocated for combining the one-class SVM and L2 SR modules, thereby achieving more accurate rust detecting results. In the experiments, we conduct numerous experiments, and when compared with other developed rust detectors, the presented method can offer better rust detecting performances.

In this paper, we propose secure dictionary learning based on a random unitary transform for sparse representation. Currently, edge cloud computing is spreading to many application fields including services that use sparse coding. This situation raises many new privacy concerns. Edge cloud computing poses several serious issues for end users, such as unauthorized use and leak of data, and privacy failures. The proposed scheme provides practical MOD and K-SVD dictionary learning algorithms that allow computation on encrypted signals. We prove, theoretically, that the proposal has exactly the same dictionary learning estimation performance as the non-encrypted variant of MOD and K-SVD algorithms. We apply it to secure image modeling based on an image patch model. Finally, we demonstrate its performance on synthetic data and a secure image modeling application for natural images.

Linear Prediction (LP) analysis is commonly used in speech processing. LP is based on Auto-Regressive (AR) model and it estimates the AR model parameter from signals with l2-norm optimization. Recently, sparse estimation is paid attention since it can extract significant features from big data. The sparse estimation is realized by l1 or l0-norm optimization or regularization. Sparse LP analysis methods based on l1-norm optimization have been proposed. Since excitation of speech is not white Gaussian, a sparse LP estimation can estimate more accurate parameter than the conventional l2-norm based LP. These are time-invariant and real-valued analysis. We have been studied Time-Varying Complex AR (TV-CAR) analysis for an analytic signal and have evaluated the performance on speech processing. The TV-CAR methods are l2-norm methods. In this paper, we propose the sparse TV-CAR analysis based on adaptive LASSO (Least absolute shrinkage and selection operator) that is l1-norm regularization and evaluate the performance on F0 estimation of speech using IRAPT (Instantaneous RAPT). The experimental results show that the sparse TV-CAR methods perform better for a high level of additive Pink noise.

Sparse arrays can usually achieve larger array apertures than uniform linear arrays (ULA) with the same number of physical antennas. However, the conventional direction-of-arrival (DOA) estimation algorithms for sparse arrays usually require the spatial smoothing operation to recover the matrix rank which inevitably involves heavy computational complexity and leads to a reduction in the degrees-of-freedom (DOFs). In this paper, a low-complex DOA estimation algorithm by exploiting the discrete Fourier transform (DFT) is proposed. Firstly, the spatial spectrum of the virtual array constructed from the sparse array is established by exploiting the DFT operation. The initial DOA estimation can obtain directly by searching the peaks in the DFT spectrum. However, since the number of array antennas is finite, there exists spectrum power leakage which will cause the performance degradation. To further improve the angle resolution, an iterative process is developed to suppress the spectrum power leakage. Thus, the proposed algorithm does not require the spatial smoothing operation and the computational complexity is reduced effectively. In addition, due to the extention of DOF with the application of the sparse arrays, the proposed algorithm can resolve the underdetermined DOA estimation problems. The superiority of the proposed algorithm is demonstrated by simulation results.

Direction of arrival (DOA) estimation as a fundamental issue in array signal processing has been extensively studied for many applications in military and civilian fields. Many DOA estimation algorithms have been developed for different application scenarios such as low signal-to-noise ratio (SNR), limited snapshots, etc. However, there are still some practical problems that make DOA estimation very difficult. One of them is the correlation between sources. In this paper, we develop a sparsity-based method to estimate the DOA of coherent signals with sparse linear array (SLA). We adopt the off-grid signal model and solve the DOA estimation problem in the sparse Bayesian learning (SBL) framework. By considering the SLA as a ‘missing sensor’ ULA, our proposed method treats the output of the SLA as a partial output of the corresponding virtual uniform linear array (ULA) to make full use of the expanded aperture character of the SLA. Then we employ the expectation-maximization (EM) method to update the hyper-parameters and the output of the virtual ULA in an iterative manner. Numerical results demonstrate that the proposed method has a better performance in correlated signal scenarios than the reference methods in comparison, confirming the advantage of exploiting the extended aperture feature of the SLA.

To efficiently collect sensor readings in cluster-based wireless sensor networks, we propose a structural compressed network coding (SCNC) scheme that jointly considers structural compressed sensing (SCS) and network coding (NC). The proposed scheme exploits the structural compressibility of sensor readings for data compression and reconstruction. Random linear network coding (RLNC) is used to re-project the measurements and thus enhance network reliability. Furthermore, we calculate the energy consumption of intra- and inter-cluster transmission and analyze the effect of the cluster size on the total transmission energy consumption. To that end, we introduce an iterative reweighed sparsity recovery algorithm to address the all-or-nothing effect of RLNC and decrease the recovery error. Experiments show that the SCNC scheme can decrease the number of measurements required for decoding and improve the network's robustness, particularly when the loss rate is high. Moreover, the proposed recovery algorithm has better reconstruction performance than several other state-of-the-art recovery algorithms.

We research device-free (DF) multi-target tracking scheme in this paper. The existing localization and tracking algorithms are always pay attention to the single target and need to collect a large amount of localization information. In this paper, we exploit the sparse property of multiple target locations to achieve target trace accurately with much less sampling both in the wireless links and the time slots. The proposed approach mainly includes the target localization part and target trace recovery part. In target localization part, by exploiting the inherent sparsity of the target number, Compressive Sensing (CS) is utilized to reduce the wireless links distributed. In the target trace recovery part, we exploit the compressive property of target trace, as well as designing the measurement matrix and the sparse matrix, to reduce the samplings in time domain. Additionally, Kronecker Compressive Sensing (KCS) theory is used to simultaneously recover the multiple traces both of the X label and the Y Label. Finally, simulations show that the proposed approach holds an effective recovery performance.