Generally, two problems of bag-of-features in image retrieval are still considered unsolved: one is that spatial information about descriptors is not employed well, which affects the accuracy of retrieval; the other is that the trade-off between vocabulary size and good precision, which decides the storage and retrieval performance. In this paper, we propose a novel approach called Hilbert scan based bag-of-features (HS-BoF) for image retrieval. Firstly, Hilbert scan based tree representation (HSBT) is studied, which is built based on the local descriptors while spatial relationships are added into the nodes by a novel grouping rule, resulting of a tree structure for each image. Further, we give two ways of codebook production based on HSBT: multi-layer codebook and multi-size codebook. Owing to the properties of Hilbert scanning and the merits of our grouping method, sub-regions of the tree are not only flexible to the distribution of local patches but also have hierarchical relations. Extensive experiments on caltech-256, 13-scene and 1 million ImageNet images show that HS-BoF obtains higher accuracy with less memory usage.

We are interested in retrieving video shots or videos containing particular people from a video dataset. Owing to the large variations in pose, illumination conditions, occlusions, hairstyles and facial expressions, face tracks have recently been researched in the fields of face recognition, face retrieval and name labeling from videos. However, when the number of face tracks is very large, conventional methods, which match all or some pairs of faces in face tracks, will not be effective. Therefore, in this paper, an efficient method for finding a given person from a video dataset is presented. In our study, in according to performing research on face tracks in a single video, we also consider how to organize all the faces in videos in a dataset and how to improve the search quality in the query process. Different videos may include the same person; thus, the management of individuals in different videos will be useful for their retrieval. The proposed method includes the following three points. (i) Face tracks of the same person appearing for a period in each video are first connected on the basis of scene information with a time constriction, then all the people in one video are organized by a proposed hierarchical clustering method. (ii) After obtaining the organizational structure of all the people in one video, the people are organized into an upper layer by affinity propagation. (iii) Finally, in the process of querying, a remeasuring method based on the index structure of videos is performed to improve the retrieval accuracy. We also build a video dataset that contains six types of videos: films, TV shows, educational videos, interviews, press conferences and domestic activities. The formation of face tracks in the six types of videos is first researched, then experiments are performed on this video dataset containing more than 1 million faces and 218,786 face tracks. The results show that the proposed approach has high search quality and a short search time.

The topic of retrieving videos containing a desired person from a dataset just using the content of faces without any help of textual information has many interesting applications like video surveillance, social network, video mining, etc. However, traditional face matching against a huge number of detected faces leads to an unacceptable response time and may also reduce the accuracy due to the large variations in facial expressions, poses, lighting, etc. Therefore, in this paper we propose a novel method to generate discriminative “signatures” for efficiently retrieving the videos containing the same person with a query. In this research, the signature is defined as a compact, discriminative and reduced dimensionality representation, which is generated from a set of high-dimensional feature vectors of an individual. The desired videos are retrieved based on the similarities between the signature of the query and those of individuals in the database. In particular, we make the following contributions. Firstly, we give an algorithm of two directional linear discriminant analysis with maximum correntropy criterion (2DLDA-MCC) as an extension to our recently proposed maximum correntropy criterion based linear discriminant analysis (LDA-MCC). Both algorithms are robust to outliers and noise. Secondly, we present an approach for transferring a set of exemplars to a fixed-length signature using LDA-MCC and 2DLDA-MCC, resulting in two kinds of signatures that are called 1D signature and 2D signature. Finally, a novel video retrieval scheme is given based on the signatures, which has low storage requirement and can achieve a fast search. Evaluations on a large dataset of videos show reliable measurement of similarities by using the proposed signatures to represent the identities generated from videos. Experimental results also demonstrate that the proposed video retrieval scheme has the potential to substantially reduce the response time and slightly increase the mean average precision of retrieval.