Infrared and visible image fusion can combine the thermal radiation information and the textures to provide a high-quality fused image. In this letter, we propose a hybrid variational fusion model to achieve this end. Specifically, an ℓ0 term is adopted to preserve the highlighted targets with salient gradient variation in the infrared image, an ℓ1 term is used to suppress the noise in the fused image and an ℓ2 term is employed to keep the textures of the visible image. Experimental results demonstrate the superiority of the proposed variational model and our results have more sharpen textures with less noise.

Various haze removal methods based on the atmospheric scattering model have been presented in recent years. Most methods have targeted strong haze images where light is scattered equally in all color channels. This paper presents a haze removal method using near-infrared (NIR) images for relatively weak haze images. In order to recover the lost edges, the presented method first extracts edges from an appropriately weighted NIR image and fuses it with the color image. By introducing a wavelength-dependent scattering model, our method then estimates the transmission map for each color channel and recovers the color more naturally from the edge-recovered image. Finally, the edge-recovered and the color-recovered images are blended. In this blending process, the regions with high lightness, such as sky and clouds, where unnatural color shifts are likely to occur, are effectively estimated, and the optimal weighting map is obtained. Our qualitative and quantitative evaluations using 59 pairs of color and NIR images demonstrated that our method can recover edges and colors more naturally in weak haze images than conventional methods.

The thermal imaging pedestrian segmentation system has excellent performance in different illumination conditions, but it has some drawbacks(e.g., weak pedestrian texture information, blurred object boundaries). Meanwhile, high-performance large models have higher latency on edge devices with limited computing performance. To solve the above problems, in this paper, we propose a real-time thermal infrared pedestrian segmentation method. The feature extraction layers of our method consist of two paths. Firstly, we utilize the lossless spatial downsampling to obtain boundary texture details on the spatial path. On the context path, we use atrous convolutions to improve the receptive field and obtain more contextual semantic information. Then, the parameter-free attention mechanism is introduced at the end of the two paths for effective feature selection, respectively. The Feature Fusion Module (FFM) is added to fuse the semantic information of the two paths after selection. Finally, we accelerate method inference through multi-threading techniques on the edge computing device. Besides, we create a high-quality infrared pedestrian segmentation dataset to facilitate research. The comparative experiments on the self-built dataset and two public datasets with other methods show that our method also has certain effectiveness. Our code is available at https://github.com/mcjcs001/LEIPNet.

In this letter, we propose a semantic guided infrared and visible image fusion method, which can train a network to fuse different semantic objects with different fusion weights according to their own characteristics. First, we design the appropriate fusion weights for each semantic object instead of the whole image. Second, we employ the semantic segmentation technology to obtain the semantic region of each object, and generate special weight maps for the infrared and visible image via pre-designed fusion weights. Third, we feed the weight maps into the loss function to guide the image fusion process. The trained fusion network can generate fused images with better visual effect and more comprehensive scene representation. Moreover, we can enhance the modal features of various semantic objects, benefiting subsequent tasks and applications. Experiment results demonstrate that our method outperforms the state-of-the-art in terms of both visual effect and quantitative metrics.

A robust pedestrian detection approach in thermal infrared imageries for an all-day surveillance is proposed. Firstly, the candidate regions which are likely to contain pedestrians are extracted based on a saliency detection method. Then a deep convolutional network with a multi-task loss is constructed to recognize the pedestrians. The experimental results show the superiority of the proposed approach in pedestrian detection.

Novel thermopiles based on modulation doped AlGaAs/InGaAs and AlGaN/GaN heterostructures are proposed and developed for the first time, for uncooled infrared FPA (Focal Plane Array) image sensor application. The high responsivity with the high speed response time are designed to 4,900 V/W with 110 µs for AlGaAs/InGaAs, and to 460 V/W with 9 µs for AlGaN/GaN thermopiles, respectively. Based on integrated HEMT-MEMS technology, the AlGaAs/InGaAs 3232 matrix FPAs are fabricated to demonstrate its enhanced performances by black body measurement. The technology presented here demonstrates the potential of this approach for low-cost uncooled infrared FPA image sensor application.

Novel thermopiles based on modulation doped AlGaAs/InGaAs, AlGaN/GaN, and ZnMgO/ZnO heterostructures are proposed and designed for the first time, for uncooled infrared image sensor application. These devices are expected to offer high performances due to both the superior Seebeck coefficient and the excellently high mobility of 2DEG and 2DHG due to high purity channel layers at the heterojunction interface. The AlGaAs/InGaAs thermopile has the figure-of-merit Z of as large as 1.110-2/K (ZT = 3.3 over unity at T = 300 K), and can be realized with a high responsivity R of 15,200 V/W and a high detectivity D* of 1.8109 cmHz1/2/W with uncooled low-cost potentiality. The AlGaN/GaN and the ZnMgO/ZnO thermopiles have the advantages of high sheet carrier concentration due to their large polarization charge effects (spontaneous and piezo polarization charges) as well as of a high Seebeck coefficient due to their strong phonon-drag effect. The high speed response time τ of 0.9 ms with AlGaN/GaN, and also the lower cost with ZnMgO/ZnO thermopiles can be realized. The modulation-doped heterostructure thermopiles presented here are expected to be used for uncooled infrared image sensor applications, and for monolithic integrations with other photon detectors such as InGaAs, GaN, and ZnO PiN photodiodes, as well as HEMT functional integrated circuit devices.

A novel image enhancement algorithm that can efficiently detect a small target of panoramic infrared (IR) imagery is proposed. Image enhancement is the first step for detecting and recognizing a small target in the IR imagery. The essence of the proposed algorithm is to utilize the independent histogram equalization (HE) separately over two sub-images obtained by decomposing the given image through the statistical hypothesis testing (SHT). Experimental results show that the proposed algorithm has better discrimination and lower false alarm rate than the conventional algorithms.