Recently, two approaches investigated indexing and retrieving videos. One approach utilized the visual features of individual objects, and the other approach exploited the spatio-temporal relationships between multiple objects. In this paper, we integrate both approaches into a new video model, called the Visual-Spatio-Temporal (VST) model to represent videos. The visual features are modeled in a topological approach and integrated with the spatio-temporal relationships. As a result, we defined rich sets of VST relationships which support and simplify the formulation of more semantical queries. An intuitive query interface which allows users to describe VST features of video objects by sketch and feature specification is presented. The conducted experiments prove the effectiveness of modeling and querying videos by the visual features of individual objects and the VST relationships between multiple objects.

In this paper, we present a novel approach for efficient search of high-dimensional databases, such as video shots. The idea is to map feature vectors from the high-dimensional feature space into a point in a low-dimensional distance space. Then, a spatial access method, such as an R-tree, is used to cluster these points based on their distances in the low-dimensional space. Our mapping method, called topological mapping, guarantees no false dismissals in the result of a query. However, the result of a query might contain some false alarms. Hence, two refinement steps are performed to remove these false alarms. Comparative experiments on a database of video shots show the superior efficiency of the topological mapping method over other known methods.