This paper presents a hierarchical-masked image filtering method for privacy-protection. Cameras are widely used for various applications, e.g., crime surveillance, environment monitoring, and marketing. However, invasion of privacy has become a serious social problem, especially regarding the use of surveillance cameras. Many surveillance cameras point at many people; thus, a large amount of our private information of our daily activities are under surveillance. However, several surveillance cameras currently on the market and related research often have a complicated or institutional masking privacy-protection functionality. To overcome this problem, a Hierarchical-Masked image Filtering (HMF) method is proposed, which has unmaskable (mask reversal) capability and is applicable to current surveillance camera systems for privacy-information protection and can satisfy privacy-protection related requirements. This method has five main features: unmasking of the original image from only the masked image and a cipher key, hierarchical-mask level control using parameters for the length of a pseudorandom number, robustness against malicious attackers, fast processing on an embedded processor, and applicability of mask operation to current surveillance camera systems. Previous studies have difficulty in providing these features. To evaluate HMF on actual equipment, an HMF-based prototype system is developed that mainly consists of a USB web camera, ultra-compact single board computer, and notebook PC. Through experiments, it is confirmed that the proposed method achieves mask level control and is robust against attacks. The increase in processing time of the HMF-based prototype system compared with a conventional non-masking system is only about 1.4%. This paper also reports on the comparison of the proposed method with conventional privacy protection methods and favorable responses of people toward the HMF-based prototype system both domestically and abroad. Therefore, the proposed HMF method can be applied to embedded systems such as those equipped with surveillance cameras for protecting privacy.

Similarity search is an important and fundamental problem, and thus widely used in various fields of computer science including multimedia, computer vision, database, information retrieval, etc. Recently, since loitering behavior often leads to abnormal situations, such as pickpocketing and terrorist attacks, its analysis attracts increasing attention from research communities. In this paper, we present AntiLoiter, a loitering discovery system adopting efficient similarity search on surveillance videos. As we know, most of existing systems for loitering analysis, mainly focus on how to detect or identify loiterers by behavior tracking techniques. However, the difficulties of tracking-based methods are known as that their analysis results are heavily influenced by occlusions, overlaps, and shadows. Moreover, tracking-based methods need to track the human appearance continuously. Therefore, existing methods are not readily applied to real-world surveillance cameras due to the appearance discontinuity of criminal loiterers. To solve this problem, we abandon the tracking method, instead, propose AntiLoiter to efficiently discover loiterers based on their frequent appearance patterns in longtime multiple surveillance videos. In AntiLoiter, we propose a novel data structure Luigi that indexes data using only similarity value returned by a corresponding function (e.g., face matching). Luigi is adopted to perform efficient similarity search to realize loitering discovery. We conducted extensive experiments on both synthetic and real surveillance videos to evaluate the efficiency and efficacy of our approach. The experimental results show that our system can find out loitering candidates correctly and outperforms existing method by 100 times in terms of runtime.

Considering that existing clutter cancellation methods process information either in the time domain or in the spatial domain, this paper proposes a new clutter cancellation method that utilizes joint multi-domain information for passive radar. Assuming that there is a receiving array at the surveillance channel, firstly we propose a multi-domain information clutter cancellation model by constructing a time domain weighted matrix and a spatial weighted vector. Secondly the weighted matrix and vector can be updated adaptively utilizing the constant modulus constraint. Finally the weighted matrix is derived from the principle of optimal filtering and the recursion formula of weighted vector is obtained utilizing the Gauss-Newton method. Making use of the information in both time and spatial domain, the proposed method attenuates the noise and residual clutter whose directions are different from that of the target echo. Simulation results prove that the proposed method has higher clutter attenuation (CA) compared with the traditional methods in the low signal to noise ratio condition, and it also improves the detection performance of weak targets.

Time synchronization is of paramount importance in wireless sensor networks (WSNs) due to the inherent distributed characteristics of WSNs. Border surveillance WSNs, especially, require a highly secure and accurate time synchronization scheme to detect and track intruders. In this paper, we propose a Secure and Efficient Time synchronization scheme for Border surveillance WSNs (SETB) which meets the requirements of border surveillance WSNs while minimizing the resource usage. To accomplish this goal, we first define the performance and security requirements for time synchronization in border surveillance WSNs in detail. Then, we build our time synchronization scheme optimized for these requirements. By utilizing both heterogeneous WSNs and one-way key chains, SETB satisfies the requirements with much less overhead than existing schemes. Additionally, we introduce on-demand time synchronization, which implies that time synchronization is conducted only when an intruder enters the WSN, in order to reduce energy consumption. Finally, we propose a method of deploying time-source nodes to keep the synchronization error within the requirement. Our analysis shows that SETB not only satisfies the performance and security requirements, but also is highly efficient in terms of communication and computation overhead, thus minimizing energy consumption.

The problem of identifying moving objects in a video recording produced by a range sensor camera is due to the limited information available for classifying different objects. On the other hand, the infrared signal from a range sensor camera is more robust for extreme luminance intensity when the monitored area has light conditions that are too bright or too dark. This paper proposes a method of detection and tracking moving objects in image sequences captured by stationary range sensor cameras. Here, the depth information is utilized to correctly identify each of detected objects. Firstly, camera calibration and background subtraction are performed to separate the background from the moving objects. Next, a 2D projection mapping is performed to obtain the location and contour of the objects in the 2D plane. Based on this information, graph matching is performed based on features extracted from the 2D data, namely object position, size and the behavior of the objects. By observing the changes in the number of objects and the objects' position relative to each other, similarity matching is performed to track the objects in the temporal domain. Experimental results show that by using similarity matching, object identification can be correctly achieved even during occlusion.

To eliminate casting shadows of moving objects, which cause difficulties in vision applications, a novel method is proposed based on Visual background extractor by altering its updating mechanism using relevant spatiotemporal information. An adaptive threshold and a spatial adjustment are also employed. Experiments on typical surveillance scenes validate this scheme.

For digital forensic investigations, the proposed scheme verifies the integrity of video contents in legacy surveillance camera systems with no built-in integrity protection. The scheme exploits video frames remaining in slack space of storage media, instead of timestamp information vulnerable to tampering. The scheme is applied to integrity verification of video contents formatted with AVI or MP4 files in automobile blackboxes.

In consideration of the image noise captured by photoelectric cameras at nighttime, a robust motion detection algorithm based on sparse representation is proposed in this study. A universal dictionary for arbitrary scenes is presented. Realistic and synthetic experiments demonstrate the robustness of the proposed approach.

This paper proposes a novel method for determining a three-dimensional (3D) bounding box to estimate pose (position and orientation) and size of a 3D object corresponding to a segmented object region in an image acquired by a single calibrated camera. The method is designed to work upon an object on the ground and to determine a bounding box aligned to the direction of the object, thereby reducing the number of degrees of freedom in localizing the bounding box to 5 from 9. Observations associated with the structural properties of back-projected object regions on the ground are suggested, which are useful for determining the object points expected to be on the ground. A suitable base is then estimated from the expected on-ground object points by applying to them an assumption of bilateral symmetry. A bounding box with this base is finally constructed by determining its height, such that back-projection of the constructed box onto the ground minimally encloses back-projection of the given object region. Through experiments with some 3D-modelled objects and real objects, we found that a bounding box aligned to the dominant direction estimated from edges with common direction looks natural, and the accuracy of the pose and size is enough for localizing actual on-ground objects in an industrial working space. The proposed method is expected to be used effectively in the fields of smart surveillance and autonomous navigation.

Re-identifying the same person in different images is a distinct challenge for visual surveillance systems. Building an accurate correspondence between highly variable images requires a suitable dissimilarity measure. To date, most existing measures have used adapted distance based on a learned metric. Unfortunately, real-world human image data, which tends to show large intra-class variations and small inter-class differences, continues to prevent these measures from achieving satisfactory re-identification performance. Recognizing neighboring distribution can provide additional useful information to help tackle the deviation of the to-be-measured samples, we propose a novel dissimilarity measure from the neighborhood-wise relative information perspective, which can deliver the effectiveness of those well-distributed samples to the badly-distributed samples to make intra-class dissimilarities smaller than inter-class dissimilarities, in a learned discriminative space. The effectiveness of this method is demonstrated by explanation and experimentation.

Recently, video surveillance systems have been widely introduced in various places, and protecting the privacy of objects in the scene has been as important as ensuring security. Masking each moving object with a background subtraction method is an effective technique to protect its privacy. However, the background subtraction method is heavily affected by sunshine change, and a redundant masking by over-extraction is inevitable. Such superfluous masking disturbs the quality of video surveillance. In this paper, we propose a moving object masking method combining background subtraction and machine learning based on Real AdaBoost. This method can reduce the superfluous masking while maintaining the reliability of privacy protection. In the experiments, we demonstrate that the proposed method achieves about 78-94% accuracy for classifying superfluous masking regions and moving objects.

A covariance-based algorithm is proposed to find a barrage jammer suppression filter for surveillance radar with an adaptive array. The conventional adaptive beamformer (ABF) or adaptive sidelobe canceller (ASLC) with auxiliary antennas can be used successfully in sidelobe jammer rejection. When a jammer shares the same bearing with the target of interest, however, those methods inherently cancel the target in their attempt to null the jammer. By exploiting the jammer multipath scattered returns incident from other angles, the proposed algorithm uses only the auto-covariance matrix of the sample data produced by stacking range cell returns in a pulse repetition interval (PRI). It does not require estimation of direction of arrival (DOA) or time difference of arrival (TDOA) of multipath propagation, thus making it applicable to electronic countermeasure (ECM) environments with high power barrage jammers and it provides the victim radar with the ability to null both the sidelobe (sidebeam) and mainlobe (mainbeam) jammers simultaneously. Numeric simulations are provided to evaluate the performance of this filter in the presence of an intensive barrage jammer with jammer-to-signal ratio (JSR) greater than 30dB, and the achieved signal-to-jammer-plus-noise ratio (SJNR) improvement factor (IF) exceeds 46dB.

Multiple-shot person re-identification, which is valuable for application in visual surveillance, tackles the problem of building the correspondence between images of the same person from different cameras. It is challenging because of the large within-class variations due to the changeable body appearance and environment and the small between-class differences arising from the possibly similar body shape and clothes style. A novel method named “Bi-level Relative Information Analysis” is proposed in this paper for the issue by treating it as a set-based ranking problem. It creatively designs a relative dissimilarity using set-level neighborhood information, called “Set-level Common-Near-Neighbor Modeling”, complementary to the sample-level relative feature “Third-Party Collaborative Representation” which has recently been proven to be quite effective for multiple-shot person re-identification. Experiments implemented on several public benchmark datasets show significant improvements over state-of-the-art methods.

This paper presents a technique that analyzes pedestrians' attributes such as gender and bag-possession status from surveillance video. One of the technically challenging issues is that we use only top-view camera images to protect privacy. The shape features over the frames are extracted by bag-of-features (BoF) using histogram of oriented gradients (HoG) vectors. In order to enhance the classification accuracy, a two-staged classification framework is presented. Multiple classifiers are trained by changing the parameters in the first stage. The outputs from the first stage is further trained and classified in the second stage classifier. The experiments using 60-minute video captured at Haneda Airport, Japan, show that the accuracies for the gender classification and the bag-possession classification were 95.8% and 97.2%, respectively, which is a significant improvement from our previous work.

Emerging video surveillance technologies are based on foreground detection to achieve event detection automatically. Integration foreground detection with a modern multi-camera surveillance system can significantly increase the surveillance efficiency. The foreground detection often leads to high computational load and increases the cost of surveillance system when a mass deployment of end cameras is needed. This paper proposes a DSP-based foreground detection algorithm. Our algorithm incorporates a temporal data correlation predictor (TDCP) which can exhibit the correlation of data and reduce computation based on this correlation. With the DSP-oriented foreground detection, an adaptive frame rate control is developed as a low cost solution for multi-camera surveillance system. The adaptive frame rate control automatically detects the computational load of foreground detection on multiple video sources and adaptively tunes the TDCP to meet the real-time specification. Therefore, no additional hardware cost is required when the number of deployed cameras is increased. Our method has been validated on a demonstration platform. Performance can achieve real-time CIF frame processing for a 16-camera surveillance system by single-DSP chip. Quantitative evaluation demonstrates that our solution provides satisfied detection rate, while significantly reducing the hardware cost.

A novel approach for detecting anomaly in visual surveillance system is proposed in this paper. It is composed of three parts: (a) a dense motion field and motion statistics method, (b) motion directional PCA for feature dimensionality reduction, (c) an improved one-class SVM for one-class classification. Experiments demonstrate the effectiveness of the proposed algorithm in detecting abnormal events in surveillance video, while keeping a low false alarm rate. Our scheme works well in complicated situations that common tracking or detection modules cannot handle.

We propose a statistical method for counting pedestrians. Previous pedestrian counting methods are not applicable to highly crowded areas because they rely on the detection and tracking of individuals. The performance of detection-and-tracking methods are easily degraded for highly crowded scene in terms of both accuracy and computation time. The proposed method employs feature-based regression in the spatiotemporal domain to count pedestrians. The proposed method is accurate and requires less computation time, even for large crowds, because it does not include the detection and tracking of objects. Our test results from four hours of video sequence obtained from a highly crowded shopping mall, reveal that the proposed method is able to measure human traffic with an accuracy of 97.2% and requires only 14 ms per frame.

In this letter, we propose a novel motion detection method in order to accurately perform the detection of moving objects in the automatic video surveillance system. Based on the proposed Background Generation Mechanism, the presence of either moving object or background information is firstly checked in order to supply the selective updating of the high-quality adaptive background model, which facilitates the further motion detection using the Laplacian distribution model. The overall results of the detection accuracy will be demonstrated that our proposed method attains a substantially higher degree of efficacy, outperforming the state-of-the-art method by average Similarity accuracy rates of up to 56.64%, 27.78%, 50.04%, 43.33%, and 44.09%, respectively.

This paper presents an automatic system for detecting and re-identifying people moving in different sites with non-overlapping views. We first propose an automatic process for silhouette extraction based on the combination of an adaptive background subtraction algorithm and a motion detection module. Such a combination takes advantage of both approaches and is able to tackle the problem of particular environments. The silhouette extraction results are then clustered based on their spatial belonging and colorimetric characteristics in order to preserve only the key regions that effectively represent the appearance of a person. The next important step consists in characterizing the extracted silhouettes by the appearance-based signatures. Our proposed descriptor, which includes both color and spatial feature of objects, leads to satisfying results compared to other descriptors in the literature. Since the passage of a person needs to be characterized by multiple frames, a large quantity of data has to be processed. Thus, a graph-based algorithm is used to realize the comparison of passages of people in front of cameras and to make the final decision of re-identification. The global system is tested on two real and difficult data sets recorded in very different environments. The experimental results show that our proposed system leads to very satisfactory results.

The Hough transform is known to be an effective technique for target detection and track initiation in search radars. However, most papers have focused on the simplistic applications of this technique which consider a 2-D data space for the Hough transform. In this paper, a new method based on xthe Hough transform is introduced for detecting targets in a 3-D data space. The data space is constructed from returned surveillance radar signal using the range and bearing information of several successive scans. This information is mapped into a 3-D x-y-t Cartesian data space. Targets are modeled with four parameters in this data space. The proposed 3-D Hough detector is then used to detect the existent targets in the 3-D surveillance space by mapping the returned signal of the radar from the data space to the parameter space. This detector, which is constructed of two detection stages, integrates the returned data of each target non-coherently along its 3-D trajectory in one parameter space cell related to this target. Hence, the detection performance will improve. The effectiveness of the new 3-D Hough detector is demonstrated through deriving the detection statistics analytically and comparing the results with those of several comprehensive simulations. The performance improvement of this detector is shown by comparing its detection range with the conventional detector. The proposed detector is also evaluated with real radar data and its efficiency is confirmed.