In this paper, we present a novel photometric compensation network named CASEformer, which is built upon the Swin module. For the first time, we combine coordinate attention and channel attention mechanisms to extract rich features from input images. Employing a multi-level encoder-decoder architecture with skip connections, we establish multiscale interactions between projection surfaces and projection images, achieving precise inference and compensation. Furthermore, through an attention fusion module, which simultaneously leverages both coordinate and channel information, we enhance the global context of feature maps while preserving enhanced texture coordinate details. The experimental results demonstrate the superior compensation effectiveness of our approach compared to the current state-of-the-art methods. Additionally, we propose a method for multi-surface projection compensation, further enriching our contributions.

To simultaneously enhance data rate and physical layer security (PLS) for low-luminance smartphone screen to camera uplink communication, space division multiplexing using high-luminance cell-size reduction arrangement is numerically analyzed and experimentally verified. The uplink consists of a low-luminance smartphone screen and an indoor telephoto camera at a long distance of 3.5 meters. The high-luminance cell-size reduction arrangement avoids the influence of spatial inter-symbol interference (ISI) and ambient light to obtain a stable low-luminance screen. To reduce the screen luminance without decreasing the screen pixel value, the arrangement reduces only the high-luminance cell area while keeping the cell spacing. In this study, two technical issues related to high-luminance cell-size reduction arrangement are solved. First, a numerical analysis and experimental results show that the high-luminance cell-size reduction arrangement is more effective in reducing the spatial ISI at low luminance than the conventional low-luminance cell arrangement. Second, in view point of PLS enhancement at wide angles, symbol error rate should be low in front of the screen and high at wide angles. A numerical analysis and experimental results show that the high-luminance cell-size reduction arrangement is more suitable for enhancing PLS at wide angles than the conventional low-luminance cell arrangement.

Privacy violations via spy cameras are becoming increasingly serious. With the recent advent of various smart home IoT devices, such as smart TVs and robot vacuum cleaners, spycam attacks that steal users' information are being carried out in more unpredictable ways. In this paper, we introduce a new spycam attack on a mobile WebVR environment. It is performed by a web attacker who maliciously accesses the back-facing cameras of victims' mobile devices while they are browsing the attacker's WebVR site. This has the power to allow the attacker to capture victims' surroundings even at the desired field of view through sophisticated content placement in VR scenes, resulting in serious privacy breaches for mobile VR users. In this letter, we introduce a new threat facing mobile VR and show that it practically works with major browsers in a stealthy manner.

A new approach for multi-target tracking in an occlusion environment is presented. In pedestrian tracking using a video camera, pedestrains must be tracked accurately and continuously in the images. However, in a crowded environment, the conventional tracking algorithm has a problem in that tracks do not continue when pedestrians are hidden behind the foreground object. In this study, we propose a robust tracking method for occlusion that introduces a degeneration hypothesis that relaxes the track hypothesis which has one measurement to one track constraint. The proposed method relaxes the hypothesis that one measurement and multiple trajectories are associated based on the endpoints of the bounding box when the predicted trajectory is approaching, therefore the continuation of the tracking is improved using the measurement in the foreground. A numerical evaluation using MOT (Multiple Object Tracking) image data sets is performed to demonstrate the effectiveness of the proposed algorithm.

We propose an augmented-reality-based method for arranging furniture using natural markers extracted from the edges of the walls of rooms. The proposed method extracts natural markers and estimates the camera parameters from single images of rooms using deep neural networks. Experimental results show that in all the measurements, the superimposition error of the proposed method was lower than that of general marker-based methods that use practical-sized markers.

Observed results of arc discharges generated between the brush and commutator are reported. The motion of the arc discharges was observed by a high-speed camera. The brush and commutator were installed to an experimental device that simulated the rotational motion of a real DC motor. The aim of this paper is to investigate the occurring position, dimensions, and moving characteristics of the arc discharges by means of high-speed imaging. Time evolutions of the arc voltage and current were measured, simultaneously. The arc discharges were generated when an inductive circuit was interrupted. Circuit current before interruption was 4A. The metal graphite or graphite brush and a copper commutator were used. Following results were obtained. The arc discharge was dragged on the brush surface and the arc discharge was sticking to the side surface of the commutator. The positions of the arc spots were on the end of the commutator and the center of the brush in rotational direction. The dimensions of the arc discharge were about 0.2 mm in length and about 0.3 mm in width. The averaged arc voltage during arc duration became higher and the light emission from the arc discharge became brighter, as the copper content of the cathode decreased.

Automobile companies have been trying to replace side mirrors of cars with small cameras for reducing air resistance. It enables us to apply some image processing to improve the quality of the image. Contrast Limited Adaptive Histogram Equalization (CLAHE) is one of such techniques to improve the quality of the image for the side mirror camera, which requires a large computation performance. Here, an implementation method of CLAHE on a low-end FPGA board by high-level synthesis is proposed. CLAHE has two main processing parts: cumulative distribution function (CDF) generation, and bilinear interpolation. During the CDF generation, the effect of increasing loop initiation interval can be greatly reduced by placing multiple Processing Elements (PEs). and during the interpolation, latency and BRAM usage were reduced by revising how to hold CDF and calculation method. Finally, by connecting each module with streaming interfaces, using data flow pragmas, overlapping processing, and hiding data transfer, our HLS implementation achieved a comparable result to that of HDL. We parameterized the components of the algorithm so that the number of tiles and the size of the image can be easily changed. The source code for this research can be downloaded from https://github.com/kokihonda/fpga_clahe.

Dependences of arc duration D and contact gap at arc extinction d on contact opening speed v are studied for break arcs generated in a 48VDC resistive circuit at constant contact opening speeds. The opening speed v is varied over a wide range from 0.05 to 0.5m/s. Circuit current while electrical contacts are closed I0 is varied to 10A, 20A, 50A, 100A, 200A, and 300A. The following results were obtained. For each current I0, the arc duration D decreased with increasing contact opening speed v. However, the D at I0=300A was shorter than that at I0=200A. On the other hand, the contact gap at arc extinction d tended to increase with increasing the I0. However, the d at I0=300A was shorter than that at I0=200A. The d was almost constant with increasing the v for each current I0 when the I0 was lower than 200A. However, the d became shorter when the v was slower at I0=200A and 300A. At the v=0.05m/s, for example, the d at I0=300A was shorter than that at I0=100A. To explain the cause of the results of the d, in addition, arc length just before extinction L were analyzed. The L tended to increase with increasing current I0. The L was almost constant with increasing the v when the I0 was lower than 200A. However, when I0=200A and 300A, the L tended to become longer when the v was slower. The characteristics of the d will be discussed using the analyzed results of the L and motion of break arcs. At higher currents at I0=200A and 300A, the shorter d at the slowest v was caused by wide motion of the arc spots on contact surfaces and larger deformation of break arcs.

In this paper, we present an efficient and robust method for estimating Homography matrix for soccer field registration between a captured camera image and a soccer field model. The presented method first detects reliable field lines from the camera image through clustering. Constructing a novel directional feature of the intersection points of the lines in both the camera image and the model, the presented method then finds matching pairs of these points between the image and the model. Finally, Homography matrix estimations and validations are performed using the obtained matching pairs, which can reduce the required number of Homography matrix calculations. Our presented method uses possible intersection points outside image for the point matching. This effectively improves robustness and accuracy of Homography estimation as demonstrated in experimental results.

This paper presents an efficient method for solving PnP, PnPf, and PnPfr problems, which are the problems of determining camera parameters from 2D-3D point correspondences. The proposed method is derived based on a simple usage of linear algebra, similarly to the classical DLT methods. Therefore, the new method is easier to understand, easier to implement, and several times faster than the state-of-the-art methods using Gröbner basis. Contrary to the existing Gröbner basis methods, the proposed method consists of three algorithms depending on the number of the points and the 3D point configuration. Experimental results show that the proposed method is as accurate as the state-of-the-art methods even in near-planar scenes while achieving up to three times faster.

Coded-aperture imaging has been utilized for compressive light field acquisition; several images are captured using different aperture patterns, and from those images, an entire light field is computationally reconstructed. This method has been extended to dynamic light fields (moving scenes). However, this method assumed that the patterns were gray-valued and of arbitrary shapes. Implementation of such patterns required a special device such as a liquid crystal on silicon (LCoS) display, which made the imaging system costly and prone to noise. To address this problem, we propose the use of a binary aperture pattern rotating along time, which can be implemented with a rotating plate with a hole. We demonstrate that although using such a pattern limits the design space, our method can still achieve a high reconstruction quality comparable to the original method.

During the detection of maritime targets, the jitter of the shipborne camera usually causes the video instability and the false or missed detection of targets. Aimed at tackling this problem, a novel algorithm for maritime target detection based on the electronic image stabilization technology is proposed in this study. The algorithm mainly includes three models, namely the points line model (PLM), the points classification model (PCM), and the image classification model (ICM). The feature points (FPs) are firstly classified by the PLM, and stable videos as well as target contours are obtained by the PCM. Then the smallest bounding rectangles of the target contours generated as the candidate bounding boxes (bboxes) are sent to the ICM for classification. In the experiments, the ICM, which is constructed based on the convolutional neural network (CNN), is trained and its effectiveness is verified. Our experimental results demonstrate that the proposed algorithm outperformed the benchmark models in all the common metrics including the mean square error (MSE), peak signal to noise ratio (PSNR), structural similarity index (SSIM), and mean average precision (mAP) by at least -47.87%, 8.66%, 6.94%, and 5.75%, respectively. The proposed algorithm is superior to the state-of-the-art techniques in both the image stabilization and target ship detection, which provides reliable technical support for the visual development of unmanned ships.

In this study, we present a novel method for removing smoke from videos based on a single image sequence. Smoke is a significant artifact in images or videos because it can reduce the visibility in disaster scenes. Our proposed method for removing smoke involves two main processes: (1) the development of a smoke imaging model and (2) smoke removal using spatio-temporal pixel compensation. First, we model the optical phenomena in natural scenes including smoke, which is called a smoke imaging model. Our smoke imaging model is developed by extending conventional haze imaging models. We then remove the smoke from a video in a frame-by-frame manner based on the smoke imaging model. Next, we refine the appearance of the smoke-free video by spatio-temporal pixel compensation, where we align the smoke-free frames using the corresponding pixels. To obtain the corresponding pixels, we use SIFT and color features with distance constraints. Finally, in order to obtain a clear video, we refine the pixel values based on the spatio-temporal weightings of the corresponding pixels in the smoke-free frames. We used simulated and actual smoke videos in our validation experiments. The experimental results demonstrated that our method can obtain effective smoke removal results from dynamic scenes. We also quantitatively assessed our method based on a temporal coherence measure.

Input devices based on direct touch have replaced traditional ones and become the mainstream interactive technology for handheld devices. Although direct touch interaction proves to be easy to use, its problems, e.g. the occlusion problem and the fat finger problem, lower user experience. Camera-based mobile interaction is one of the solutions to overcome the problems. There are two typical interaction styles to generate camera-based pointing interaction for handheld devices: move the device or move an object before the camera. In the first interaction style, there are two approaches to move a cursor's position across the handheld display: move it towards the same direction or the opposite direction which the device moves to. In this paper, the results of a comparison research, which compared the pointing performances of three camera-based pointing techniques, are presented. All pointing techniques utilized input from the rear-facing camera. The results indicate that the interaction style of moving a finger before the camera outperforms the other one in efficiency, accuracy, and throughput. The results also indicate that within the interaction style of moving the device, the cursor positioning style of moving the cursor to the opposite direction is slightly better than the other one in efficiency and throughput. Based on the findings, we suggest giving priority to the interaction style of moving a finger when deploying camera-based pointing techniques on handheld devices. Given that the interaction style of moving the device supports one-handed manipulation, it also worth deploying when one-handed interaction is needed. According to the results, the cursor positioning style of moving the cursor towards the opposite direction which the device moves to may be a better choice.

We propose a video authentication scheme to verify whether a given video file is recorded by a camera device or touched by a video editing tool. The proposed scheme prepares software characteristics of camera devices and video editing tools in advance, and compares them with the metadata of the given video file. Through practical implementation, we show that the proposed scheme has benefits of fast analysis time, high accuracy and full automation.

Since wild animals are causing more accidents and damages, it is important to safely detect them as early as possible. In this paper, we propose two battery-powered wild animal detection nodes based on deep learning that can automatically detect wild animals; the detection information is notified to the people concerned immediately. To use the proposed nodes outdoors where power is not available, we devise power saving techniques for the proposed nodes. For example, deep learning is used to save power by avoiding operations when wild animals are not detected. We evaluate the operation time and the power consumption of the proposed nodes. Then, we evaluate the energy consumption of the proposed nodes. Also, we evaluate the detection range of the proposed nodes, the accuracy of deep learning, and the success rate of communication through field tests to demonstrate that the proposed nodes can be used to detect wild animals outdoors.

Silver electrical contacts were separated at constant opening speed in a 200V-500VDC/10A resistive circuit. Break arcs were extinguished by magnetic blowing-out with transverse magnetic field of a permanent magnet. The permanent magnet was appropriately located to simplify the lengthened shape of the break arcs. Magnetic flux density of the transverse magnetic field was varied from 20 to 140mT. Images of the break arcs were observed from the horizontal and vertical directions using two high speed cameras simultaneously. Arc length just before extinction was analyzed from the observed images. It was shown that shapes of the break arcs were simple enough to trace the most part of paths of the break arcs for all experimental conditions owing to simplification of the shapes of the break arcs by appropriate arrangement of the magnet. The arc length increased with increasing supply voltage and decreased with increasing magnetic flux density. These results will be discussed in the view points of arc lengthening time and arc lengthening velocity.

Multiple human tracking is widely used in various fields such as marketing and surveillance. The typical approach associates human detection results between consecutive frames using the features and bounding boxes (position+size) of detected humans. Some methods use an omnidirectional camera to cover a wider area, but ID switch often occurs in association with detections due to following two factors: i) The feature is adversely affected because the bounding box includes many background regions when a human is captured from an oblique angle. ii) The position and size change dramatically between consecutive frames because the distance metric is non-uniform in an omnidirectional image. In this paper, we propose a novel method that accurately tracks humans with an association metric for omnidirectional images. The proposed method has two key points: i) For feature extraction, we introduce local rectification, which reduces the effect of background regions in the bounding box. ii) For distance calculation, we describe the positions in a world coordinate system where the distance metric is uniform. In the experiments, we confirmed that the Multiple Object Tracking Accuracy (MOTA) improved 3.3 in the LargeRoom dataset and improved 2.3 in the SmallRoom dataset.

We present an accurate and easy-to-use multi-sensor fusion toolbox for autonomous vehicles. It includes a ‘target-less’ multi-LiDAR (Light Detection and Ranging), and Camera-LiDAR calibration, sensor fusion, and a fast and accurate point cloud ground classifier. Our calibration methods do not require complex setup procedures, and once the sensors are calibrated, our framework eases the fusion of multiple point clouds, and cameras. In addition we present an original real-time ground-obstacle classifier, which runs on the CPU, and is designed to be used with any type and number of LiDARs. Evaluation results on the KITTI dataset confirm that our calibration method has comparable accuracy with other state-of-the-art contenders in the benchmark.

A multi-camera setup for a surveillance system enables a larger coverage area, especially when a single camera has limited monitoring capability due to certain obstacles. Therefore, for large-scale coverage, multiple cameras are the best option. In this paper, we present a method for detecting multiple objects using several cameras with large overlapping views as this allows synchronization of object identification from a number of views. The proposed method uses a graph structure that is robust enough to represent any detected moving objects by defining their vertices and edges to determine their relationships. By evaluating these object features, represented as a set of attributes in a graph, we can perform lightweight multiple object detection using several cameras, as well as performing object tracking within each camera's field of view and between two cameras. By evaluating each vertex hierarchically as a subgraph, we can further observe the features of the detected object and perform automatic separation of occluding objects. Experimental results show that the proposed method would improve the accuracy of object tracking by reducing the occurrences of incorrect identification compared to individual camera-based tracking.