A simple depth-key-based image composition is proposed, which uses two still images with depth information, background and foreground object. The proposed method can place the object at various locations in the background considering the depth in the 3D world coordinate system. The main feature is that a simple algorithm is provided, which enables us to achieve the depthward movement within the camera plane, without being aware of the 3D world coordinate system. Two algorithms are proposed (P-OMDD and O-OMDD), which are based on the pin-hole camera model. As an advantage, camera calibration is not required before applying the algorithm in these methods. Since a single image is used for the object representation, each of the proposed methods has its limitations in terms of fidelity of the composite image. P-OMDD faithfully reproduces the angle at which the object is seen, but the pixels of the hidden surface are missing. On the contrary, O-OMDD can avoid the hidden surface problem, but the angle of the object is fixed, wherever it moves. It is verified through several experiments that, when using O-OMDD, subjectively natural composite images can be obtained under any object movement, in terms of size and position in the camera plane. Future tasks include improving the change in illumination due to positional changes and the partial loss of objects due to noise in depth images.

This paper presents an approach to analyze and model tasks of machines being operated. The executions of the tasks were captured through egocentric vision. Each task was decomposed into a sequence of physical hand-machine interactions, which are described with touch-based hotspots and interaction patterns. Modeling the tasks was achieved by integrating the experiences of multiple experts and using a hidden Markov model (HMM). Here, we present the results of more than 70 recorded egocentric experiences of the operation of a sewing machine. Our methods show good potential for the detection of hand-machine interactions and modeling of machine operation tasks.

We present a system to improve the robustness of real-time 3D surface reconstruction by utilizing non-inertial localization sensor. Benefiting from such sensor, our easy-to-build system can effectively avoid tracking drift and lost comparing with conventional dense tracking and mapping systems. To best fusing the sensor, we first adopt a hand-eye calibration and performance analysis for our setup and then propose a novel optimization framework based on adaptive criterion function to improve the robustness as well as accuracy. We apply our system to several challenging reconstruction tasks, which show significant improvement in scanning robustness and reconstruction quality.

A joint deployment of base stations (BSs) and RGB-depth (RGB-D) cameras for camera-assisted millimeter-wave (mmWave) access networks is discussed in this paper. For the deployment of a wide variety of devices in heterogeneous networks, it is crucial to consider the synergistic effects among the different types of nodes. A synergy between mmWave networks and cameras reduces the power consumption of mmWave BSs through sleep control. A purpose of this work is to optimize the number of nodes of each type, to maximize the average achievable rate within the constraint of a predefined total power budget. A stochastic deployment problem is formulated as a submodular optimization problem, by assuming that the deployment of BSs and cameras forms two independent Poisson point processes. An approximate algorithm is presented to solve the deployment problem, and it is proved that a (1-e-1)/2-approximate solution can be obtained for submodular optimization, using a modified greedy algorithm. The numerical results reveal the deployment conditions under which the average achievable rate of the camera-assisted mmWave system is higher than that of a conventional system that does not employ RGB-D cameras.

In this study, we address the problem of online RGB-D tracking which confronted with various challenges caused by deformation, occlusion, background clutter, and abrupt motion. Various trackers have different strengths and weaknesses, and thus a single tracker can merely perform well in specific scenarios. We propose a 3D tracker-level fusion algorithm (TLF3D) which enhances the strengths of different trackers and suppresses their weaknesses to achieve robust tracking performance in various scenarios. The fusion result is generated from outputs of base trackers by optimizing an energy function considering both the 3D cube attraction and 3D trajectory smoothness. In addition, three complementary base RGB-D trackers with intrinsically different tracking components are proposed for the fusion algorithm. We perform extensive experiments on a large-scale RGB-D benchmark dataset. The evaluation results demonstrate the effectiveness of the proposed fusion algorithm and the superior performance of the proposed TLF3D tracker against state-of-the-art RGB-D trackers.

To substantially alleviate the human blockage problem in mmWave communications, this paper proposes a proactive handover system based on human blockage prediction using RGB and depth (RGB-D) cameras. The proposed scheme uses RGB-D camera images to estimate the mobility of pedestrians and to predict when blockage will occur. On the basis of this information, the proposed system transfers a mobile station (STA) communicating with one wireless BS (base station) to another BS before human blockage occurs and thus avoids blockage-induced throughput degradation. This paper presents performance modeling of both proactive handover scheme and reactive handover scheme which is based on the received power level. A numerical evaluation reveals conditions under which the proactive handover scheme achieves higher spectral efficiency compared to reactive scheme. In addition, using IEEE 802.11ad-based wireless local area network (WLAN) devices, a testbed for implementing the proposed system is built. The innovative experimental results demonstrate that the proactive handover system can considerably reduce the duration of human blockage-induced degradation of throughput performance relative to the reactive scheme.