Input devices based on direct touch have replaced traditional ones and become the mainstream interactive technology for handheld devices. Although direct touch interaction proves to be easy to use, its problems, e.g. the occlusion problem and the fat finger problem, lower user experience. Camera-based mobile interaction is one of the solutions to overcome the problems. There are two typical interaction styles to generate camera-based pointing interaction for handheld devices: move the device or move an object before the camera. In the first interaction style, there are two approaches to move a cursor's position across the handheld display: move it towards the same direction or the opposite direction which the device moves to. In this paper, the results of a comparison research, which compared the pointing performances of three camera-based pointing techniques, are presented. All pointing techniques utilized input from the rear-facing camera. The results indicate that the interaction style of moving a finger before the camera outperforms the other one in efficiency, accuracy, and throughput. The results also indicate that within the interaction style of moving the device, the cursor positioning style of moving the cursor to the opposite direction is slightly better than the other one in efficiency and throughput. Based on the findings, we suggest giving priority to the interaction style of moving a finger when deploying camera-based pointing techniques on handheld devices. Given that the interaction style of moving the device supports one-handed manipulation, it also worth deploying when one-handed interaction is needed. According to the results, the cursor positioning style of moving the cursor towards the opposite direction which the device moves to may be a better choice.

There are some technologies like QR codes to obtain digital information from printed matters. Digital watermarking is one of such techniques. Compared with other techniques, digital watermarking is suitable for adding information to images without spoiling their design. For such purposes, digital watermarking methods for printed matters using detection markers or image registration techniques for detecting watermarked areas are proposed. However, the detection markers themselves can damage the appearance such that the advantages of digital watermarking, which do not lose design, are not fully utilized. On the other hand, methods using image registration techniques are not able to work for non-registered images. In this paper, we propose a novel digital watermarking method using deep learning for the detection of watermarked areas instead of using detection markers or image registration. The proposed method introduces a semantic segmentation based on deep learning model for detecting watermarked areas from printed matters. We prepare two datasets for training the deep learning model. One is constituted of geometrically transformed non-watermarked and watermarked images. The number of images in this dataset is relatively large because the images can be generated based on image processing. This dataset is used for pre-training. The other is obtained from actually taken photographs including non-watermarked or watermarked printed matters. The number of this dataset is relatively small because taking the photographs requires a lot of effort and time. However, the existence of pre-training allows a fewer training images. This dataset is used for fine-tuning to improve robustness for print-cam attacks. In the experiments, we investigated the performance of our method by implementing it on smartphones. The experimental results show that our method can carry 96 bits of information with watermarked printed matters.

We propose a video authentication scheme to verify whether a given video file is recorded by a camera device or touched by a video editing tool. The proposed scheme prepares software characteristics of camera devices and video editing tools in advance, and compares them with the metadata of the given video file. Through practical implementation, we show that the proposed scheme has benefits of fast analysis time, high accuracy and full automation.

Since wild animals are causing more accidents and damages, it is important to safely detect them as early as possible. In this paper, we propose two battery-powered wild animal detection nodes based on deep learning that can automatically detect wild animals; the detection information is notified to the people concerned immediately. To use the proposed nodes outdoors where power is not available, we devise power saving techniques for the proposed nodes. For example, deep learning is used to save power by avoiding operations when wild animals are not detected. We evaluate the operation time and the power consumption of the proposed nodes. Then, we evaluate the energy consumption of the proposed nodes. Also, we evaluate the detection range of the proposed nodes, the accuracy of deep learning, and the success rate of communication through field tests to demonstrate that the proposed nodes can be used to detect wild animals outdoors.

Silver electrical contacts were separated at constant opening speed in a 200V-500VDC/10A resistive circuit. Break arcs were extinguished by magnetic blowing-out with transverse magnetic field of a permanent magnet. The permanent magnet was appropriately located to simplify the lengthened shape of the break arcs. Magnetic flux density of the transverse magnetic field was varied from 20 to 140mT. Images of the break arcs were observed from the horizontal and vertical directions using two high speed cameras simultaneously. Arc length just before extinction was analyzed from the observed images. It was shown that shapes of the break arcs were simple enough to trace the most part of paths of the break arcs for all experimental conditions owing to simplification of the shapes of the break arcs by appropriate arrangement of the magnet. The arc length increased with increasing supply voltage and decreased with increasing magnetic flux density. These results will be discussed in the view points of arc lengthening time and arc lengthening velocity.

Multiple human tracking is widely used in various fields such as marketing and surveillance. The typical approach associates human detection results between consecutive frames using the features and bounding boxes (position+size) of detected humans. Some methods use an omnidirectional camera to cover a wider area, but ID switch often occurs in association with detections due to following two factors: i) The feature is adversely affected because the bounding box includes many background regions when a human is captured from an oblique angle. ii) The position and size change dramatically between consecutive frames because the distance metric is non-uniform in an omnidirectional image. In this paper, we propose a novel method that accurately tracks humans with an association metric for omnidirectional images. The proposed method has two key points: i) For feature extraction, we introduce local rectification, which reduces the effect of background regions in the bounding box. ii) For distance calculation, we describe the positions in a world coordinate system where the distance metric is uniform. In the experiments, we confirmed that the Multiple Object Tracking Accuracy (MOTA) improved 3.3 in the LargeRoom dataset and improved 2.3 in the SmallRoom dataset.

Logic encryption is an IC protection technique which inserts extra logic and key inputs to hide a circuit's functionality. An encrypted circuit needs to be activated with a secret key for being functional. SAT attack and Removal attack are two most advanced decryption methods that have shown their effectiveness to break most of the existing logic encryption methods within a few hours. In this paper, we propose SMARTLock, a SAT attack and reMoval Attack-Resistant Tree-based logic Locking method, for resisting them simultaneously. To encrypt a circuit, the method finds large AND and OR functions in it and encrypts them by inserting duplicate tree functions. There are two types of structurally identical tree encryptions that aim to resist SAT attack and Removal attack, respectively. The experimental results show that the proposed method is effective for encrypting a set of benchmarks from ISCAS'85, MCNC, and IWLS. 16 out of 40 benchmarks encrypted by the proposed method with the area overhead of no more than 5% are uncrackable by SAT attack within 5 hours. Additionally, compared to the state-of-the-art logic encryption methods, the proposed method provides better security for most benchmarks.

We present an accurate and easy-to-use multi-sensor fusion toolbox for autonomous vehicles. It includes a ‘target-less’ multi-LiDAR (Light Detection and Ranging), and Camera-LiDAR calibration, sensor fusion, and a fast and accurate point cloud ground classifier. Our calibration methods do not require complex setup procedures, and once the sensors are calibrated, our framework eases the fusion of multiple point clouds, and cameras. In addition we present an original real-time ground-obstacle classifier, which runs on the CPU, and is designed to be used with any type and number of LiDARs. Evaluation results on the KITTI dataset confirm that our calibration method has comparable accuracy with other state-of-the-art contenders in the benchmark.

A multi-camera setup for a surveillance system enables a larger coverage area, especially when a single camera has limited monitoring capability due to certain obstacles. Therefore, for large-scale coverage, multiple cameras are the best option. In this paper, we present a method for detecting multiple objects using several cameras with large overlapping views as this allows synchronization of object identification from a number of views. The proposed method uses a graph structure that is robust enough to represent any detected moving objects by defining their vertices and edges to determine their relationships. By evaluating these object features, represented as a set of attributes in a graph, we can perform lightweight multiple object detection using several cameras, as well as performing object tracking within each camera's field of view and between two cameras. By evaluating each vertex hierarchically as a subgraph, we can further observe the features of the detected object and perform automatic separation of occluding objects. Experimental results show that the proposed method would improve the accuracy of object tracking by reducing the occurrences of incorrect identification compared to individual camera-based tracking.

In this paper, a nonoverlapping multi-camera and people re-identification algorithm is proposed. It applies inflated major color features for re-identification to reduce computation time. The inflated major color features can dramatically improve efficiency while retaining high accuracy of object re-identification. The proposed method is evaluated over a wide range of experimental databases. The accuracy attains upwards of 40.7% in Rank 1 and 84% in Rank 10 on average, while it obtains three to 15 times faster than algorithms reported in the literature. The proposed algorithm has been implemented on a SOC-FPGA platform to reach 50 FPS with 1280×720 HD resolution and 25 FPS with 1920×1080 FHD resolution for real-time processing. The results show a performance improvement and reduction in computation complexity, which is especially ideal for embedded platform.

Break arcs are generated in a 48VDC resistive circuit. Circuit current I0 when electrical contacts are closed is changed from 50A to 300A. The break arcs are observed by a high-speed camera with appropriate settings of exposure from horizontal direction. Length of the break arcs L is measured from images of the break arcs. Time evolutions of the length L and gap voltage Vg are investigated. The following results are obtained. By appropriate settings of the high-speed camera, the time evolution of the length L is obtained from just after ignition to before arc extinction. Tendency of increase of the length L is similar to that of increase of the voltage Vg for each current I0.

We propose an object detector using a sliding window method for an event-driven camera which outputs a subtracted frame (usually a binary value) when changes are detected in captured images. Since sliding window skips unchanged portions of the output, the number of target object area candidates decreases dramatically, which means that our system operates faster and with lower power consumption than a system using a straightforward sliding window approach. Since the event-driven camera output consists of binary precision frames, an all binarized convolutional neural network (ABCNN) can be available, which means that it allows all convolutional layers to share the same binarized convolutional circuit, thereby reducing the area requirement. We implemented our proposed method on the Xilinx Inc. Zedboard and then evaluated it using the PETS 2009 dataset. The results showed that our system outperformed BCNN system from the viewpoint of detection performance, hardware requirement, and computation time. Also, we showed that FPGA is an ideal method for our system than mobile GPU. From these results, our proposed system is more suitable for the embedded systems based on stationary cameras (such as security cameras).

The arrival of the era of the Internet of Things (IoT) has ensured the ubiquity of human-sensing technologies. Cameras have become inexpensive instruments for human sensing and have been increasingly used for this purpose. Because cameras produce large quantities of information, they are powerful tools for sensing; however, because camera images contain information allowing individuals to be personally identified, their use poses risks of personal privacy violations. In addition, because IoT-ready home appliances are connected to the Internet, camera-captured images of individual users may be unintentionally leaked. In developing our human-detection method [33], [34], we proposed techniques for detecting humans from unclear images in which individuals cannot be identified; however, a drawback of this method was its inability to detect moving humans. Thus, to enable tracking of humans even through the images are blurred to protect privacy, we introduce a particle-filter framework and propose a human-tracking method based on motion detection and heart-rate detection. We also show how the use of integral images [32] can accelerate the execution of our algorithms. In performance tests involving unclear images, the proposed method yields results superior to those obtained with the existing mean-shift method or with a face-detection method based on Haar-like features. We confirm the acceleration afforded by the use of integral images and show that the speed of our method is sufficient to enable real-time operation. Moreover, we demonstrate that the proposed method allows successful tracking even in cases where the posture of the individual changes, such as when the person lies down, a situation that arises in real-world usage environments. We discuss the reasons behind the superior behavior of our method in performance tests compared to those of other methods.

We propose a method for extracting multi-view images from a light field (plenoptic) camera that accurately handles the physical pixel arrangement of this camera. We use a Lytro Illum camera to obtain 4D light field data (a set of multi-viewpoint images) through a micro-lens array. The light field data are multiplexed on a single image sensor, and thus, the data is first demultiplexed into a set of multi-viewpoint (sub-aperture) images. However, the demultiplexing process usually includes interpolation of the original data such as demosaicing for a color filter array and pixel resampling for the hexagonal pixel arrangement of the original sub-aperture images. If this interpolation is performed, some information is added or lost to/from the original data. In contrast, we preserve the original data as faithfully as possible, and use them directly for the super resolution reconstruction, where the super-resolved image and the corresponding depth map are alternatively refined. We experimentally demonstrate the effectiveness of our method in resolution enhancement through comparisons with Light Field Toolbox and Lytro Desktop Application. Moreover, we also mention another type of light field cameras, a Raytrix camera, and describe how it can be handled to extract high-quality multi-view images.

Since camera networks for surveillance are becoming extremely dense, finding the most informative and desirable views from different cameras are of increasing importance. In this paper, we propose a camera selection method to achieve the goal of providing the clearest visibility possible and selecting the cameras which exactly capture targets for the far-field surveillance. We design a benefit function that takes into account image visibility and the degree of target matching between different cameras. Here, visibility is defined using the entropy of intensity histogram distribution, and the target correspondence is based on activity features rather than photometric features. The proposed solution is tested in both artificial and real environments. A performance evaluation shows that our target correspondence method well suits far-field surveillance, and our proposed selection method is more effective at identifying the cameras that exactly capture the surveillance target than existing methods.

In this paper, we propose a novel method to extrinsically calibrate a camera to a 3D reference object that is not directly visible from the camera. We use a human cornea as a spherical mirror and calibrate the extrinsic parameters from the reflections of the reference points. The main contribution of this paper is to present a cornea-reflection-based calibration algorithm with a simple configuration: five reference points on a single plane and one mirror pose. In this paper, we derive a linear equation and obtain a closed-form solution of extrinsic calibration by introducing two ideas. The first is to model the cornea as a virtual sphere, which enables us to estimate the center of the cornea sphere from its projection. The second is to use basis vectors to represent the position of the reference points, which enables us to deal with 3D information of reference points compactly. We demonstrate the performance of the proposed method with qualitative and quantitative evaluations using synthesized and real data.

Electrical contacts are separated at constant speed and break arcs are generated in nitrogen or air in a 200V-450VDC/10A resistive circuit. The break arcs are extinguished by magnetic blow-out. Arc duration for the silver and copper contact pairs is investigated for each supply voltage. Following results are shown. The arc duration for Cu contacts in nitrogen is the shortest. For Cu contacts, the arc dwell time in air was considerably longer than that of nitrogen. For Ag contacts, the arc duration in nitrogen was almost the same as that in air.

An improved radiometric calibration algorithm by extending the Mitsunaga and Nayar least-square minimization based algorithm with two major ideas is presented. First, a noise & outlier removal procedure based on the analysis of brightness transfer function is included for improving the algorithm's capability on handling noise and outlier in least-square estimation. Second, an alternative minimization formulation based on weighted least square is proposed to improve the weakness of least square minimization when dealing with biased distribution observations. The performance of the proposed algorithm with regards to two baseline algorithms is demonstrated, i.e. the classical least square based algorithm proposed by Mitsunaga and Nayar and the state-of-the-art rank minimization based algorithm proposed by Lee et al. From the results, the proposed algorithm outperforms both baseline algorithms on both the synthetic dataset and the dataset of real-world images.

In this paper, we propose a method to generate a three-dimensional (3D) thermal map and RGB + thermal (RGB-T) images of a scene from thermal-infrared and RGB images. The scene images are acquired by moving both a RGB camera and an thermal-infrared camera mounted on a stereo rig. Before capturing the scene with those cameras, we estimate their respective intrinsic parameters and their relative pose. Then, we reconstruct the 3D structures of the scene by using Direct Sparse Odometry (DSO) using the RGB images. In order to superimpose thermal information onto each point generated from DSO, we propose a method for estimating the scale of the point cloud corresponding to the extrinsic parameters between both cameras by matching depth images recovered from the RGB camera and the thermal-infrared camera based on mutual information. We also generate RGB-T images using the 3D structure of the scene and Delaunay triangulation. We do not rely on depth cameras and, therefore, our technique is not limited to scenes within the measurement range of the depth cameras. To demonstrate this technique, we generate 3D thermal maps and RGB-T images for both indoor and outdoor scenes.

Depth estimation for a lense-array type light field camera is a challenging problem because of the sensor noise and the radiometric distortion which is a global brightness change among sub-aperture images caused by a vignetting effect of the micro-lenses. We propose a depth map estimation method which has robustness against sensor noise and radiometric distortion. Our method first binarizes sub-aperture images by applying the census transform. Next, the binarized images are matched by computing the majority operations between corresponding bits and summing up the Hamming distance. An initial depth obtained by matching has ambiguity caused by extremely short baselines among sub-aperture images. After an initial depth estimation process, we refine the result with following refinement steps. Our refinement steps first approximate the initial depth as a set of depth planes. Next, we optimize the result of plane fitting with an edge-preserving smoothness term. Experiments show that our method outperforms the conventional methods.