Binary sparse representation based on arbitrary quality metrics and its applications are presented in this paper. The novelties of the proposed method are twofold. First, the proposed method newly derives sparse representation for which representation coefficients are binary values, and this enables selection of arbitrary image quality metrics. This new sparse representation can generate quality metric-independent subspaces with simplification of the calculation procedures. Second, visual saliency is used in the proposed method for pooling the quality values obtained for all of the parts within target images. This approach enables visually pleasant approximation of the target images more successfully. By introducing the above two novel approaches, successful image approximation considering human perception becomes feasible. Since the proposed method can provide lower-dimensional subspaces that are obtained by better image quality metrics, realization of several image reconstruction tasks can be expected. Experimental results showed high performance of the proposed method in terms of two image reconstruction tasks, image inpainting and super-resolution.

The quality of visual comfort and depth perception is a crucial requirement for virtual reality (VR) applications. This paper investigates major causes of visual discomfort and proposes a novel virtual camera controlling method using visual saliency to minimize visual discomfort. We extract the saliency of each scene and properly adjust the convergence plane to preserve realistic 3D effects. We also evaluate the effectiveness of our method on free-form architecture models. The results indicate that the proposed saliency-guided camera control is more comfortable than typical camera control and gives more realistic depth perception.

It is estimated that 80% of the information entering the human brain is obtained through the eyes. Therefore, it is commonly believed that drawing human attention to particular objects is effective in assisting human activities. In this paper, we propose a novel image modification method for guiding user attention to specific regions of interest by using a novel saliency map model based on spatial frequency components. We modify the frequency components on the basis of the obtained saliency map to decrease the visual saliency outside the specified region. By applying our modification method to an image, human attention can be guided to the specified region because the saliency inside the region is higher than that outside the region. Using gaze measurements, we show that the proposed saliency map matches well with the distribution of actual human attention. Moreover, we evaluate the effectiveness of the proposed modification method by using an eye tracking system.

Blur is one of the most common distortion type and greatly impacts image quality. Most existing no-reference (NR) image blur metrics produce scores without a fixed range, so it is hard to judge the extent of blur directly. This letter presents a NR perceptual blur metric using Saliency Guided Gradient Similarity (SGGS), which produces blur scores with a fixed range of (0,1). A blurred image is first reblurred using a Gaussian low-pass filter, producing a heavily blurred image. With this reblurred image as reference, a local blur map is generated by computing the gradient similarity. Finally, visual saliency is employed in the pooling to adapt to the characteristics of the human visual system (HVS). The proposed metric features fixed range, fast computation and better consistency with the HVS. Experiments demonstrate its advantages.

Selective visual attention is an integral mechanism of the human visual system that is often neglected when designing perceptually relevant image and video quality metrics. Disregarding attention mechanisms assumes that all distortions in the visual content impact equally on the overall quality perception, which is typically not the case. Over the past years we have performed several experiments to study the effect of visual attention on quality perception. In addition to gaining a deeper scientific understanding of this matter, we were also able to use this knowledge to further improve various quality prediction models. In this article, I review our work with the aim to increase awareness on the importance of visual attention mechanisms for the effective design of quality prediction models.

In this paper, we propose a novel method for road sign detection and recognition in complex scene real world images. Our algorithm consists of four basic steps. First, we employ a regional contrast based bottom-up visual saliency method to highlight the traffic sign regions, which usually have dominant color contrast against the background. Second, each type of traffic sign has special color distribution, which can be explored by top-down visual saliency to enhance the detection precision and to classify traffic signs into different categories. A bag-of-words (BoW) model and a color name descriptor are employed to compute the special-class distribution. Third, the candidate road sign blobs are extracted from the final saliency map, which are generated by combining the bottom-up and the top-down saliency maps. Last, the color and shape cues are fused in the BoW model to express blobs, and a support vector machine is employed to recognize road signs. Experiments on real world images show a high success rate and a low false hit rate and demonstrate that the proposed framework is applicable to prohibition, warning and obligation signs. Additionally, our method can be applied to achromatic signs without extra processing.

A new image watermarking scheme is presented to achieve high capacity information hiding and geometric invariance simultaneously. Visually salient region is introduced into watermark synchronization. The saliency value of a region is used as the quantitative measure of robustness, based on which the idea of locally most salient region (LMSR) is proposed to generate the disjoint invariant regions. A meaningful binary watermark is then encoded using Chinese Remainder Theorem (CRT) in transform domain. Simulation results and comparisons demonstrate the effectiveness of the proposed scheme.