A multi-camera setup for a surveillance system enables a larger coverage area, especially when a single camera has limited monitoring capability due to certain obstacles. Therefore, for large-scale coverage, multiple cameras are the best option. In this paper, we present a method for detecting multiple objects using several cameras with large overlapping views as this allows synchronization of object identification from a number of views. The proposed method uses a graph structure that is robust enough to represent any detected moving objects by defining their vertices and edges to determine their relationships. By evaluating these object features, represented as a set of attributes in a graph, we can perform lightweight multiple object detection using several cameras, as well as performing object tracking within each camera's field of view and between two cameras. By evaluating each vertex hierarchically as a subgraph, we can further observe the features of the detected object and perform automatic separation of occluding objects. Experimental results show that the proposed method would improve the accuracy of object tracking by reducing the occurrences of incorrect identification compared to individual camera-based tracking.

In this paper, a nonoverlapping multi-camera and people re-identification algorithm is proposed. It applies inflated major color features for re-identification to reduce computation time. The inflated major color features can dramatically improve efficiency while retaining high accuracy of object re-identification. The proposed method is evaluated over a wide range of experimental databases. The accuracy attains upwards of 40.7% in Rank 1 and 84% in Rank 10 on average, while it obtains three to 15 times faster than algorithms reported in the literature. The proposed algorithm has been implemented on a SOC-FPGA platform to reach 50 FPS with 1280×720 HD resolution and 25 FPS with 1920×1080 FHD resolution for real-time processing. The results show a performance improvement and reduction in computation complexity, which is especially ideal for embedded platform.

Break arcs are generated in a 48VDC resistive circuit. Circuit current I0 when electrical contacts are closed is changed from 50A to 300A. The break arcs are observed by a high-speed camera with appropriate settings of exposure from horizontal direction. Length of the break arcs L is measured from images of the break arcs. Time evolutions of the length L and gap voltage Vg are investigated. The following results are obtained. By appropriate settings of the high-speed camera, the time evolution of the length L is obtained from just after ignition to before arc extinction. Tendency of increase of the length L is similar to that of increase of the voltage Vg for each current I0.

We propose an object detector using a sliding window method for an event-driven camera which outputs a subtracted frame (usually a binary value) when changes are detected in captured images. Since sliding window skips unchanged portions of the output, the number of target object area candidates decreases dramatically, which means that our system operates faster and with lower power consumption than a system using a straightforward sliding window approach. Since the event-driven camera output consists of binary precision frames, an all binarized convolutional neural network (ABCNN) can be available, which means that it allows all convolutional layers to share the same binarized convolutional circuit, thereby reducing the area requirement. We implemented our proposed method on the Xilinx Inc. Zedboard and then evaluated it using the PETS 2009 dataset. The results showed that our system outperformed BCNN system from the viewpoint of detection performance, hardware requirement, and computation time. Also, we showed that FPGA is an ideal method for our system than mobile GPU. From these results, our proposed system is more suitable for the embedded systems based on stationary cameras (such as security cameras).

The arrival of the era of the Internet of Things (IoT) has ensured the ubiquity of human-sensing technologies. Cameras have become inexpensive instruments for human sensing and have been increasingly used for this purpose. Because cameras produce large quantities of information, they are powerful tools for sensing; however, because camera images contain information allowing individuals to be personally identified, their use poses risks of personal privacy violations. In addition, because IoT-ready home appliances are connected to the Internet, camera-captured images of individual users may be unintentionally leaked. In developing our human-detection method [33], [34], we proposed techniques for detecting humans from unclear images in which individuals cannot be identified; however, a drawback of this method was its inability to detect moving humans. Thus, to enable tracking of humans even through the images are blurred to protect privacy, we introduce a particle-filter framework and propose a human-tracking method based on motion detection and heart-rate detection. We also show how the use of integral images [32] can accelerate the execution of our algorithms. In performance tests involving unclear images, the proposed method yields results superior to those obtained with the existing mean-shift method or with a face-detection method based on Haar-like features. We confirm the acceleration afforded by the use of integral images and show that the speed of our method is sufficient to enable real-time operation. Moreover, we demonstrate that the proposed method allows successful tracking even in cases where the posture of the individual changes, such as when the person lies down, a situation that arises in real-world usage environments. We discuss the reasons behind the superior behavior of our method in performance tests compared to those of other methods.

We propose a method for extracting multi-view images from a light field (plenoptic) camera that accurately handles the physical pixel arrangement of this camera. We use a Lytro Illum camera to obtain 4D light field data (a set of multi-viewpoint images) through a micro-lens array. The light field data are multiplexed on a single image sensor, and thus, the data is first demultiplexed into a set of multi-viewpoint (sub-aperture) images. However, the demultiplexing process usually includes interpolation of the original data such as demosaicing for a color filter array and pixel resampling for the hexagonal pixel arrangement of the original sub-aperture images. If this interpolation is performed, some information is added or lost to/from the original data. In contrast, we preserve the original data as faithfully as possible, and use them directly for the super resolution reconstruction, where the super-resolved image and the corresponding depth map are alternatively refined. We experimentally demonstrate the effectiveness of our method in resolution enhancement through comparisons with Light Field Toolbox and Lytro Desktop Application. Moreover, we also mention another type of light field cameras, a Raytrix camera, and describe how it can be handled to extract high-quality multi-view images.

Since camera networks for surveillance are becoming extremely dense, finding the most informative and desirable views from different cameras are of increasing importance. In this paper, we propose a camera selection method to achieve the goal of providing the clearest visibility possible and selecting the cameras which exactly capture targets for the far-field surveillance. We design a benefit function that takes into account image visibility and the degree of target matching between different cameras. Here, visibility is defined using the entropy of intensity histogram distribution, and the target correspondence is based on activity features rather than photometric features. The proposed solution is tested in both artificial and real environments. A performance evaluation shows that our target correspondence method well suits far-field surveillance, and our proposed selection method is more effective at identifying the cameras that exactly capture the surveillance target than existing methods.

In this paper, we propose a novel method to extrinsically calibrate a camera to a 3D reference object that is not directly visible from the camera. We use a human cornea as a spherical mirror and calibrate the extrinsic parameters from the reflections of the reference points. The main contribution of this paper is to present a cornea-reflection-based calibration algorithm with a simple configuration: five reference points on a single plane and one mirror pose. In this paper, we derive a linear equation and obtain a closed-form solution of extrinsic calibration by introducing two ideas. The first is to model the cornea as a virtual sphere, which enables us to estimate the center of the cornea sphere from its projection. The second is to use basis vectors to represent the position of the reference points, which enables us to deal with 3D information of reference points compactly. We demonstrate the performance of the proposed method with qualitative and quantitative evaluations using synthesized and real data.

Electrical contacts are separated at constant speed and break arcs are generated in nitrogen or air in a 200V-450VDC/10A resistive circuit. The break arcs are extinguished by magnetic blow-out. Arc duration for the silver and copper contact pairs is investigated for each supply voltage. Following results are shown. The arc duration for Cu contacts in nitrogen is the shortest. For Cu contacts, the arc dwell time in air was considerably longer than that of nitrogen. For Ag contacts, the arc duration in nitrogen was almost the same as that in air.

An improved radiometric calibration algorithm by extending the Mitsunaga and Nayar least-square minimization based algorithm with two major ideas is presented. First, a noise & outlier removal procedure based on the analysis of brightness transfer function is included for improving the algorithm's capability on handling noise and outlier in least-square estimation. Second, an alternative minimization formulation based on weighted least square is proposed to improve the weakness of least square minimization when dealing with biased distribution observations. The performance of the proposed algorithm with regards to two baseline algorithms is demonstrated, i.e. the classical least square based algorithm proposed by Mitsunaga and Nayar and the state-of-the-art rank minimization based algorithm proposed by Lee et al. From the results, the proposed algorithm outperforms both baseline algorithms on both the synthetic dataset and the dataset of real-world images.

In this paper, we propose a method to generate a three-dimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

Depth estimation for a lense-array type light field camera is a challenging problem because of the sensor noise and the radiometric distortion which is a global brightness change among sub-aperture images caused by a vignetting effect of the micro-lenses. We propose a depth map estimation method which has robustness against sensor noise and radiometric distortion. Our method first binarizes sub-aperture images by applying the census transform. Next, the binarized images are matched by computing the majority operations between corresponding bits and summing up the Hamming distance. An initial depth obtained by matching has ambiguity caused by extremely short baselines among sub-aperture images. After an initial depth estimation process, we refine the result with following refinement steps. Our refinement steps first approximate the initial depth as a set of depth planes. Next, we optimize the result of plane fitting with an edge-preserving smoothness term. Experiments show that our method outperforms the conventional methods.

A joint deployment of base stations (BSs) and RGB-depth (RGB-D) cameras for camera-assisted millimeter-wave (mmWave) access networks is discussed in this paper. For the deployment of a wide variety of devices in heterogeneous networks, it is crucial to consider the synergistic effects among the different types of nodes. A synergy between mmWave networks and cameras reduces the power consumption of mmWave BSs through sleep control. A purpose of this work is to optimize the number of nodes of each type, to maximize the average achievable rate within the constraint of a predefined total power budget. A stochastic deployment problem is formulated as a submodular optimization problem, by assuming that the deployment of BSs and cameras forms two independent Poisson point processes. An approximate algorithm is presented to solve the deployment problem, and it is proved that a (1-e-1)/2-approximate solution can be obtained for submodular optimization, using a modified greedy algorithm. The numerical results reveal the deployment conditions under which the average achievable rate of the camera-assisted mmWave system is higher than that of a conventional system that does not employ RGB-D cameras.

This paper presents a hierarchical-masked image filtering method for privacy-protection. Cameras are widely used for various applications, e.g., crime surveillance, environment monitoring, and marketing. However, invasion of privacy has become a serious social problem, especially regarding the use of surveillance cameras. Many surveillance cameras point at many people; thus, a large amount of our private information of our daily activities are under surveillance. However, several surveillance cameras currently on the market and related research often have a complicated or institutional masking privacy-protection functionality. To overcome this problem, a Hierarchical-Masked image Filtering (HMF) method is proposed, which has unmaskable (mask reversal) capability and is applicable to current surveillance camera systems for privacy-information protection and can satisfy privacy-protection related requirements. This method has five main features: unmasking of the original image from only the masked image and a cipher key, hierarchical-mask level control using parameters for the length of a pseudorandom number, robustness against malicious attackers, fast processing on an embedded processor, and applicability of mask operation to current surveillance camera systems. Previous studies have difficulty in providing these features. To evaluate HMF on actual equipment, an HMF-based prototype system is developed that mainly consists of a USB web camera, ultra-compact single board computer, and notebook PC. Through experiments, it is confirmed that the proposed method achieves mask level control and is robust against attacks. The increase in processing time of the HMF-based prototype system compared with a conventional non-masking system is only about 1.4%. This paper also reports on the comparison of the proposed method with conventional privacy protection methods and favorable responses of people toward the HMF-based prototype system both domestically and abroad. Therefore, the proposed HMF method can be applied to embedded systems such as those equipped with surveillance cameras for protecting privacy.

Arc runners are fixed on silver electrical contacts. Break arcs are generated between the contacts in a 450VDC circuit. Break arcs are magnetically blown-out and air is blown to the break arcs. The air flow was not used to our previous reports with runners. Circuit current when contacts are closed is 10A. Flow rate of air Q is changed from 1 to 10L/min. Supply voltage E is changed from 200V to 450V. The following results are shown. Arc duration D tends to decrease with increasing flow rate Q. The number of reignitions N increases with increasing supply voltage E for each flow rate Q. The number of reignitions is the least when the flow rate Q is 2L/min.

Break arcs are rotated with a radial magnetic field formed by a permanent magnet embedded in a fixed contact. The break arcs are generated in a 48VDC resistive circuit. The circuit current is 10A when the contacts are closed. The polarity of the fixed contact in which the magnet is embedded is changed. The rotational radius and the difference of position between the cathode and anode spots are investigated. The following results are obtained. The cathode spot is moved more easily than the anode spot by the radial magnetic field. The rotational radius of the break arcs is affected by the Lorentz force that is caused by the circumferential component of the arc current and the axial component of the magnetic field. The circumferential component of the arc current is caused by the difference of the positions of the rotating cathode and anode spots.

Image sensor communication (ISC), a type of visible light communication, is an emerging wireless communication technology that uses LEDs to transmit a signal and uses an image sensor in a camera to receive the signal. This paper discusses the present status of and future trends in ISC by describing the essential characteristics and features of ISC. Moreover, we overview the products and expected future applications of ISC.

In this paper, we propose an algorithm that enhances the number of pixels for high-speed imaging. High-speed cameras have a principle problem that the number of pixels reduces when the number of frames per second (fps) increases. To enhance the number of pixels, we suppose an optical structure that block-randomly selects some percent of pixels in an image. Then, we need to reconstruct the entire image. For this, a state-of-the-art method takes three-dimensional reconstruction strategy, which requires a heavy computational cost in terms of time. To reduce the cost, the proposed method reconstructs the entire image frame-by-frame using a new cost function exploiting two types of sparsity. One is within each frame and the other is induced from the similarity between adjacent frames. The latter further means not only in the image domain, but also in a sparsifying transformed domain. Since the cost function we define is convex, we can find the optimal solution using a convex optimization technique with small computational cost. We conducted simulations using grayscale image sequences. The results show that the proposed method produces a sequence, mostly the same quality as the state-of-the-art method, with dramatically less computational time.

Environment perception is an important task for intelligent vehicles applications. Typically, multiple sensors with different characteristics are employed to perceive the environment. To robustly perceive the environment, the information from the different sensors are often integrated or fused. In this article, we propose to perform the sensor fusion and registration of the LIDAR and stereo camera using the particle swarm optimization algorithm, without the aid of any external calibration objects. The proposed algorithm automatically calibrates the sensors and registers the LIDAR range image with the stereo depth image. The registered LIDAR range image functions as the disparity map for the stereo disparity estimation and results in an effective sensor fusion mechanism. Additionally, we perform the image denoising using the modified non-local means filter on the input image during the stereo disparity estimation to improve the robustness, especially at night time. To evaluate our proposed algorithm, the calibration and registration algorithm is compared with baseline algorithms on multiple datasets acquired with varying illuminations. Compared to the baseline algorithms, we show that our proposed algorithm demonstrates better accuracy. We also demonstrate that integrating the LIDAR range image within the stereo's disparity estimation results in an improved disparity map with significant reduction in the computational complexity.

Silver electrical contacts are separated at constant speed and break arcs are generated between them in a 200V-450VDC and 10A resistive circuit. The motion of the break arcs is restricted by some surrounding alumina plates. Transverse magnetic field of a permanent magnet is applied to the break arcs. Changing the supply voltage and the height of a wall located at the upper side of the break arcs, the arc lengthening time and motion of the break arcs are investigated. As a result, the higher supply voltage causes an increase of the arc lengthening time. The arc lengthening time increases significantly when the break arcs expand into the whole of the surrounding walls.