Salient Region Extraction provides an alternative methodology to image description in many applications such as adaptive content delivery and image retrieval. In this paper, we propose a robust approach to extracting the salient region based on bottom-up visual attention. The main contributions are twofold: 1) Instead of the feature parallel integration, the proposed saliencies are derived by serial processing between texture and color features. Hence, the proposed approach intrinsically provides an alternative methodology to model attention with low implementation complexity. 2) A constructive approach is proposed for rendering an image by a non-linear intensity mapping, which can efficiently eliminate high contrast noise regions in the image. And then the salient map can be robustly generated for a variety of nature images. Experiments show that the proposed algorithm is effective and can characterize the human perception well.

Boundary detection is one of the most studied problems in computer vision. It is the foundation of contour grouping, and initially affects the performance of grouping algorithms. In this paper we propose a novel boundary detection algorithm for contour grouping, which is a selective attention guided coarse-to-fine scale pyramid model. Our algorithm evaluates each edge instead of each pixel location, which is different from others and suitable for contour grouping. Selective attention focuses on the whole saliency objects instead of local details, and gives global spatial prior for boundary existence of objects. The evolving process of edges through the coarsest scale to the finest scale reflects the importance and energy of edges. The combination of these two cues produces the most saliency boundaries. We show applications for boundary detection on natural images. We also test our approach on the Berkeley dataset and use it for contour grouping. The results obtained are pretty good.

A multi-agent object attention system is proposed, which is based on biologically inspired attractor selection model. Object attention is facilitated by using a video sequence and a depth map obtained through a compound-eye image sensor TOMBO. Robustness of the multi-agent system over environmental changes is enhanced by utilizing the biological model of adaptive response by attractor selection. To implement the proposed system, an efficient VLSI architecture is employed with reducing enormous computational costs and memory accesses required for depth map processing and multi-agent attractor selection process. According to the FPGA implementation result of the proposed object attention system, which is accomplished by using 7,063 slices, 640512 pixel input images can be processed in real-time with three agents at a rate of 9 fps in 48 MHz operation.

The manifold-ranking algorithm has been successfully adopted in content-based image retrieval (CBIR) in recent years. However, while the global low-level features are widely utilized in current systems, region-based features have received little attention. In this paper, a novel attention-driven transductive framework based on a hierarchical graph representation is proposed for region-based image retrieval (RBIR). This approach can be characterized by two key properties: (1) Since the issue about region significance is the key problem in region-based retrieval, a visual attention model is chosen here to measure the regions' significance. (2) A hierarchical graph representation which combines region-level with image-level similarities is utilized for the manifold-ranking method. A novel propagation energy function is defined which takes both low-level visual features and regional significance into consideration. Experimental results demonstrate that the proposed approach shows the satisfactory retrieval performance compared to the global-based and the block-based manifold-ranking methods.

Modeling visual attention provides an alternative methodology to image description in many applications such as adaptive content delivery and image retrieval. In this paper, we propose a robust approach to the modeling bottom-up visual attention. The main contributions are twofold: 1) We use a principal component analysis (PCA) to transform the RGB color space into three principal components, which intrinsically leads to an opponent representation of colors to ensure good saliency analysis. 2) A practicable framework for modeling visual attention is presented based on a region-level reliability analysis for each feature map. And then the salient map can be robustly generated for a variety of nature images. Experiments show that the proposed algorithm is effective and can characterize the human perception well.

Educational websites contain rich knowledge components on a web page. Detecting student attention on web pages fulfills the recommendation of adequate knowledge components to students based on students' current interests. Previous studies have shown the application of learner attention in intelligent learning systems. This study proposes a methodology to analyze student on-line mouse movement patterns that indicate student attentions. The methodology can be combined with learning systems that implement pedagogical models such as inquiry-based learning and problem-solving learning activities. The feasibility and effectiveness of the proposed methodology have been evaluated by student mouse movements in problem-solving scenarios.

This paper presents Dynamic Attention Map by Ising model for face detection. In general, a face detector can not know where faces there are and how many faces there are in advance. Therefore, the face detector must search the whole regions on the image and requires much computational time. To speed up the search, the information obtained at previous search points should be used effectively. In order to use the likelihood of face obtained at previous search points effectively, Ising model is adopted to face detection. Ising model has the two-state spins; "up" and "down". The state of a spin is updated by depending on the neighboring spins and an external magnetic field. Ising spins are assigned to "face" and "non-face" states of face detection. In addition, the measured likelihood of face is integrated into the energy function of Ising model as the external magnetic field. It is confirmed that face candidates would be reduced effectively by spin flip dynamics. To improve the search performance further, the single level Ising search method is extended to the multilevel Ising search. The interactions between two layers which are characterized by the renormalization group method is used to reduce the face candidates. The effectiveness of the multilevel Ising search method is also confirmed by the comparison with the single level Ising search method.

This paper presents a framework for automatic video region-of-interest determination based on visual attention model. We view this work as a preliminary step towards the solution of high-level semantic video analysis. Facing such a challenging issue, in this work, a set of attempts on using video attention features and knowledge of computational media aesthetics are made. The three types of visual attention features we used are intensity, color, and motion. Referring to aesthetic principles, these features are combined according to camera motion types on the basis of a new proposed video analysis unit, frame-segment. We conduct subjective experiments on several kinds of video data and demonstrate the effectiveness of the proposed framework.

In this paper, the false-peaks problem of the conventional correlation-based video tracking is investigated using a simple mathematical analysis. To reduce the false detection problem, a selective-attention correlation measure is proposed. The problem with the conventional correlation measures is that all pixels in the reference block image are equally treated in the computation of the correlation measures irrespective of target or background pixels. Therefore, the more the reference block image includes background pixels, the higher probability of false-peaks is introduced due to the correlation between the background pixels of the reference block and those of the input search image. The proposed selective-attention correlation measure has different consideration according to target and background pixels in the matching process, which conform with the selective-attention property of human visual system. Various computer simulations validated these analyses and confirmed that the proposed selective-attention measure is effective to reduce considerably the probability of the false-peaks.

This paper presents the design of an automatic surveillance system to monitor the dangerous non-frontal gazes of the car driver. To track the driver's eyes, we propose a novel filter to locate the "between-eye", which is the middle point between the two eyes, to help the fast locating of eyes. We also propose a specially designed criterion function named mean ratio function to accurately locate the positions of eyes. To analyze the gazes of the driver, a multilayer perceptron neural network is trained to examine whether the driver is losing the proper gaze or not. By incorporating the neural network output with some well-designed alarm-issuing rules, the system performs the monitoring task for single dedicated driver and multiple different drivers with a satisfied performance in our experiments.

In the study, we build a system called Adaptive Visual Attentive Tracker (AVAT) for the purpose of developing a non-verbal communication channel between the system and an operator who presents intended movements. In the system, we constructed an HMM (Hidden Markov Models)-based TD (Temporal Difference) learning algorithm to track and zoom in on an operator's behavioral sequence which represents his/her intention. AVAT extracts human intended movements from ordinary walking behavior based on the following two algorithms: the first is to model the movements of human body parts using HMMs algorithm, and the second is to learn the model of the tracker's action using a model-based TD learning algorithm. In the paper, we describe the integrated algorithm of the above two methods: whose linkage is established by assigning the state transition probability in HMM as a reward in TD learning. Experimental results of extracting an operator's hand sign action sequence during her natural walking motion are shown which demonstrates the function of AVAT as it is developed within the framework of perceptual organization. Identification of the sign gesture context through wavelet analysis autonomously provides a reward value for optimizing AVAT's action patterns.