2D and 3D semantic segmentation play important roles in robotic scene understanding. However, current 3D semantic segmentation heavily relies on 3D point clouds, which are susceptible to factors such as point cloud noise, sparsity, estimation and reconstruction errors, and data imbalance. In this paper, a novel approach is proposed to enhance 3D semantic segmentation by incorporating 2D semantic segmentation from RGB-D sequences. Firstly, the RGB-D pairs are consistently segmented into 2D semantic maps using the tracking pipeline of Simultaneous Localization and Mapping (SLAM). This process effectively propagates object labels from full scans to corresponding labels in partial views with high probability. Subsequently, a novel Semantic Projection (SP) block is introduced, which integrates features extracted from localized 2D fragments across different camera viewpoints into their corresponding 3D semantic features. Lastly, the 3D semantic segmentation network utilizes a combination of 2D-3D fusion features to facilitate a merged semantic segmentation process for both 2D and 3D. Extensive experiments conducted on public datasets demonstrate the effective performance of the proposed 2D-assisted 3D semantic segmentation method.

Synthetic aperture radar (SAR) is a device for observing the ground surface and is one of the important technologies in the field of microwave remote sensing. In SAR observation, a platform equipped with a small-aperture antenna flies in a straight line and continuously radiates pulse waves to the ground during the flight. After that, by synthesizing the series of observation data obtained during the flight, one realize high-resolution ground surface observation. In SAR observation, there are two spatial resolutions defined in the range and azimuth directions and they are limited by the bandwidth of the SAR system. The purpose of this study is to improve the resolution of SAR by sparse reconstruction. In particular, we aim to improve the resolution of SAR without changing the frequency parameters. In this paper, we propose to improve the resolution of SAR using the deconvolution iterative shrinkage-thresholding algorithm (ISTA) and verify the proposed method by carrying out an experimental analysis using an actual SAR dataset. Experimental results show that the proposed method can improve the resolution of SAR with low computational complexity.

We assess how well the recently created MRI reconstruction technique, Multi-Resolution Convolutional Neural Network (MRCNN), performs in the core medical vision field (classification). The primary goal of MRCNN is to identify the best k-space undersampling patterns to accelerate the MRI. In this study, we use the Figshare brain tumor dataset for MRI classification with 3064 T1-weighted contrast-enhanced MRI (CE-MRI) over three categories: meningioma, glioma, and pituitary tumors. We apply MRCNN to the dataset, which is a method to reconstruct high-quality images from under-sampled k-space signals. Next, we employ the pre-trained VGG16 model, which is a Deep Neural Network (DNN) based image classifier to the MRCNN restored MRIs to classify the brain tumors. Our experiments showed that in the case of MRCNN restored data, the proposed brain tumor classifier achieved 92.79% classification accuracy for a 10% sampling rate, which is slightly higher than that of SRCNN, MoDL, and Zero-filling methods have 91.89%, 91.89%, and 90.98% respectively. Note that our classifier was trained using the dataset consisting of the images with full sampling and their labels, which can be regarded as a model of the usual human diagnostician. Hence our results would suggest MRCNN is useful for human diagnosis. In conclusion, MRCNN significantly enhances the accuracy of the brain tumor classification system based on the tumor location using under-sampled k-space signals.

Focal stacks (FS) have attracted attention as an alternative representation of light field (LF). However, the problem of reconstructing LF from its FS is considered ill-posed. Although many regularization methods have been discussed, no method has been proposed to solve this problem perfectly. This paper showed that the LF can be perfectly reconstructed from the FS through a filter bank in theory for Lambertian scenes without occlusion if the camera aperture for acquiring the FS is a Cauchy function. The numerical simulation demonstrated that the filter bank allows perfect reconstruction of the LF.

Micro-expression recognition (MER) draws intensive research interest as micro-expressions (MEs) can infer genuine emotions. Prior information can guide the model to learn discriminative ME features effectively. However, most works focus on researching the general models with a stronger representation ability to adaptively aggregate ME movement information in a holistic way, which may ignore the prior information and properties of MEs. To solve this issue, driven by the prior information that the category of ME can be inferred by the relationship between the actions of facial different components, this work designs a novel model that can conform to this prior information and learn ME movement features in an interpretable way. Specifically, this paper proposes a Decomposition and Reconstruction-based Graph Representation Learning (DeRe-GRL) model to efectively learn high-level ME features. DeRe-GRL includes two modules: Action Decomposition Module (ADM) and Relation Reconstruction Module (RRM), where ADM learns action features of facial key components and RRM explores the relationship between these action features. Based on facial key components, ADM divides the geometric movement features extracted by the graph model-based backbone into several sub-features, and learns the map matrix to map these sub-features into multiple action features; then, RRM learns weights to weight all action features to build the relationship between action features. The experimental results demonstrate the effectiveness of the proposed modules, and the proposed method achieves competitive performance.

In order to improve the frequency band utilization and avoid mutual interference between signals, the BD3 satellite signals adopt Binary Offset Carrier (BOC) modulation. On one hand, BOC modulation has a narrow main peak width and strong anti-interference ability; on the other hand, the phenomenon of false acquisition locking caused by the multi-peak characteristic of BOC modulation itself needs to be resolved. In this context, this paper proposes a new BOC(n,n) unambiguous acquisition algorithm based on segmentation reconstruction. The algorithm is based on splitting the local BOC signal into four parts in each subcarrier period. The branch signal and the received signal are correlated with the received signal to generate four branch correlation signals. After a series of combined reconstructions, the final signal detection function completely eliminates secondary peaks. A simulation shows that the algorithm can completely eliminate the sub-peak interference for the BOC signals modulated by subcarriers with different phase. The characteristics of narrow correlation peak are retained. Experiments show that the proposed algorithm has superior performance in detection probability and peak-to-average ratio.

3D reconstruction methods using neural networks are popular and have been studied extensively. However, the resulting models typically lack detail, reducing the quality of the 3D reconstruction. This is because the network is not designed to capture the fine details of the object. Therefore, in this paper, we propose two networks designed to capture both the coarse and fine details of the object to improve the reconstruction of the detailed parts of the object. To accomplish this, we design two networks. The first network uses a multi-scale architecture with skip connections to associate and merge features from other levels. For the second network, we design a multi-branch deep generative network that separately learns the local features, generic features, and the intermediate features through three different tailored components. In both network architectures, the principle entails allowing the network to learn features at different levels that can reconstruct the fine parts and the overall shape of the reconstructed 3D model. We show that both of our methods outperformed state-of-the-art approaches.

To solve the problem of metamerism in the color reproduction process, various spectral reflectance reconstruction methods combined with neural network have been proposed in recent years. However, these methods are generally sensitive to initial values and can easily converge to local optimal solutions, especially on small data sets. In this paper, we propose a spectral reflectance reconstruction algorithm based on the Back Propagation Neural Network (BPNN) and an improved Sparrow Search Algorithm (SSA). In this algorithm, to solve the problem that BPNN is sensitive to initial values, we propose to use SSA to initialize BPNN, and we use the sine chaotic mapping to further improve the stability of the algorithm. In the experiment, we tested the proposed algorithm on the X-Rite ColorChecker Classic Mini Chart which contains 24 colors, the results show that the proposed algorithm has significantly better performance compared to other algorithms and moreover it can meet the needs of spectral reflectance reconstruction on small data sets. Code is avaible at https://github.com/LuraZhang/spectral-reflectance-reconsctuction.

Millimeter wave synthetic aperture interferometric radiometers (SAIR) are very powerful instruments, which can effectively realize high-precision imaging detection. However due to the existence of interference factor and complex near-field error, the imaging effect of near-field SAIR is usually not ideal. To achieve better imaging results, a new fully connected imaging network (FCIN) is proposed for near-field SAIR. In FCIN, the fully connected network is first used to reconstruct the image domain directly from the visibility function, and then the residual dense network is used for image denoising and enhancement. The simulation results show that the proposed FCIN method has high imaging accuracy and shorten imaging time.

Neural network-based three-dimensional (3D) reconstruction methods have produced promising results. However, they do not pay particular attention to reconstructing detailed parts of objects. This occurs because the network is not designed to capture the fine details of objects. In this paper, we propose a network designed to capture both the coarse and fine details of objects to improve the reconstruction of the fine parts of objects.

A typical approach to reconstructing a 3D environment model is scanning the environment with a depth sensor and fitting the accumulated point cloud to 3D models. In this kind of scenario, a general 3D environment reconstruction application assumes temporally continuous scanning. However in some practical uses, this assumption is unacceptable. Thus, a point cloud matching method for stitching several non-continuous 3D scans is required. Point cloud matching often includes errors in the feature point detection because a point cloud is basically a sparse sampling of the real environment, and it may include quantization errors that cannot be ignored. Moreover, depth sensors tend to have errors due to the reflective properties of the observed surface. We therefore make the assumption that feature point pairs between two point clouds will include errors. In this work, we propose a feature description method robust to the feature point registration error described above. To achieve this goal, we designed a deep learning based feature description model that consists of a local feature description around the feature points and a global feature description of the entire point cloud. To obtain a feature description robust to feature point registration error, we input feature point pairs with errors and train the models with metric learning. Experimental results show that our feature description model can correctly estimate whether the feature point pair is close enough to be considered a match or not even when the feature point registration errors are large, and our model can estimate with higher accuracy in comparison to methods such as FPFH or 3DMatch. In addition, we conducted experiments for combinations of input point clouds, including local or global point clouds, both types of point cloud, and encoders.

We propose a HDR (high dynamic range) reconstruction method in an image sensor with a pixel-parallel ADC (analog-to-digital converter) for non-destructively reading out the intermediate exposure image. We report the circuit design for such an image sensor and the evaluation of the basic HDR reconstruction method.

A neural network that outputs reconstructed images based on projection data containing scattered X-rays is presented, and the proposed scheme exhibits better accuracy than conventional computed tomography (CT), in which the scatter information is removed. In medical X-ray CT, it is a common practice to remove scattered X-rays using a collimator placed in front of the detector. In this study, the scattered X-rays were assumed to have useful information, and a method was devised to utilize this information effectively using a neural network. Therefore, we generated 70,000 projection data by Monte Carlo simulations using a cube comprising 216 (6 × 6 × 6) smaller cubes having random density parameters as the target object. For each projection simulation, the densities of the smaller cubes were reset to different values, and detectors were deployed around the target object to capture the scattered X-rays from all directions. Then, a neural network was trained using these projection data to output the densities of the smaller cubes. We confirmed through numerical evaluations that the neural-network approach that utilized scattered X-rays reconstructed images with higher accuracy than did the conventional method, in which the scattered X-rays were removed. The results of this study suggest that utilizing the scattered X-ray information can help significantly reduce patient dosing during imaging.

This paper proposes a high-quality computed tomography (CT) image reconstruction method from low-dose X-ray projection data. A state-of-the-art method, proposed by Xu et al., exploits dictionary learning for image patches. This method generates an overcomplete dictionary from patches of standard-dose CT images and reconstructs low-dose CT images by minimizing the sum of a data fidelity and a regularization term based on sparse representations with the dictionary. However, this method does not take characteristics of each patch, such as textures or edges, into account. In this paper, we propose to classify all patches into several classes and utilize an individual dictionary with an individual regularization parameter for each class. Furthermore, for fast computation, we introduce the orthogonality to column vectors of each dictionary. Since similar patches are collected in the same cluster, accuracy degradation by the orthogonality hardly occurs. Our simulations show that the proposed method outperforms the state-of-the-art in terms of both accuracy and speed.

A robust adaptive beamforming algorithm is proposed based on the precise interference-plus-noise covariance matrix reconstruction and steering vector estimation of the desired signal, even existing large gain-phase errors. Firstly, the model of array mismatches is proposed with the first-order Taylor series expansion. Then, an iterative method is designed to jointly estimate calibration coefficients and steering vectors of the desired signal and interferences. Next, the powers of interferences and noise are estimated by solving a quadratic optimization question with the derived closed-form solution. At last, the actual interference-plus-noise covariance matrix can be reconstructed as a weighted sum of the steering vectors and the corresponding powers. Simulation results demonstrate the effectiveness and advancement of the proposed method.

Synthetic aperture interferometric radiometer (SAIR) is a powerful sensors for high-resolution imaging. However, because of the observation errors and small number of visibility sampling points, the accuracy of reconstructed images is usually low. To overcome this deficiency, a novel super-resolution imaging (SrI) method based on super-resolution reconstruction idea is proposed in this paper. In SrI method, sparse visibility functions are first measured at different observation locations. Then the sparse visibility functions are utilized to simultaneously construct the fusion visibility function and the fusion imaging model. Finally, the high-resolution image is reconstructed by solving the sparse optimization of fusion imaging model. The simulation results demonstrate that the proposed SrI method has higher reconstruction accuracy and can improve the imaging quality of SAIR effectively.

This paper proposes a sampling strategy for bandlimited graph signals over perturbed graph, in which we assume the edge between any pair of the nodes may be deleted randomly. Considering the mismatch between the true graph and the presumed graph, we derive the mean square error (MSE) of the reconstructed bandlimited graph signals. To minimize the MSE, we propose a greedy-based algorithm to obtain the optimal sampling set. Furthermore, we use Neumann series to avoid the pseudo-inverse computing. An efficient algorithm with low-complexity is thus proposed. Finally, numerical results show the superiority of our proposed algorithms over the other existing algorithms.

Internal user threats such as information leakage or system destruction can cause significant damage to the organization, however it is very difficult to prevent or detect this attack in advance. In this paper, we propose an anomaly-based insider threat detection method with local features and global statistics over the assumption that a user shows different patterns from regular behaviors during harmful actions. We experimentally show that our detection mechanism can achieve superior performance compared to the state of the art approaches for CMU CERT dataset.

A light field, which is equivalent to a dense set of multi-view images, has various applications such as depth estimation and 3D display. One of the essential problems in light field applications is light field interpolation, i.e., view interpolation. The interpolation accuracy is enhanced by exploiting an inherent property of a light field. One example is that an epipolar plane image (EPI), which is a 2D subset of the 4D light field, consists of many lines, and these lines have almost the same slope in a local region. This structure induces a sparse representation in the frequency domain, where most of the energy resides on a line passing through the origin. On the basis of this observation, we propose a group sparsity prior suitable for light fields to exploit their line structure fully for interpolation. Specifically, we designed the directional groups in the discrete Fourier transform (DFT) domain so that the groups can represent the concentration of the energy, and we thereby formulated an LF interpolation problem as an overlapping group lasso. We also introduce several techniques to improve the interpolation accuracy such as applying a window function, determining group weights, expanding processing blocks, and merging blocks. Our experimental results show that the proposed method can achieve better or comparable quality as compared to state-of-the-art LF interpolation methods such as convolutional neural network (CNN)-based methods.

A pupil function of aperture in image capturing systems is theoretically derived such that one can perfectly reconstruct all-in-focus image through linear filtering of the focal stack. The perfect reconstruction filters are also designed based on the derived pupil function. The designed filters are space-invariant; hence the presented method does not require region segmentation. Simulation results using synthetic scenes shows effectiveness of the derived pupil function and the filters.