The bag-of-words model (BOW) has been extensively adopted by recent human action recognition methods. The pooling operation, which aggregates local descriptor encodings into a single representation, is a key determiner of the performance of the BOW-based methods. However, the spatio-temporal relationship among interest points has rarely been considered in the pooling step, which results in the imprecise representation of human actions. In this paper, we propose a novel pooling strategy named contextual max pooling (CMP) to overcome this limitation. We add a constraint term into the objective function under the framework of max pooling, which forces the weights of interest points to be consistent with their probabilities. In this way, CMP explicitly considers the spatio-temporal contextual relationships among interest points and inherits the positive properties of max pooling. Our method is verified on three challenging datasets (KTH, UCF Sports and UCF Films datasets), and the results demonstrate that our method achieves better results than the state-of-the-art methods in human action recognition.

From lessons learned in medical data mining projects we show that integration of advanced computation techniques and human inspection is indispensable in medical data mining. We proposed an integrated approach that merges data mining and text mining methods plus visualization support for expert evaluation. We also appropriately developed temporal abstraction and text mining methods to exploit the collected data. Furthermore, our visual discovery system D2MS allowed to actively and effectively working with physicians. Significant findings in hepatitis study were obtained by the integrated approach.

Recently, two approaches investigated indexing and retrieving videos. One approach utilized the visual features of individual objects, and the other approach exploited the spatio-temporal relationships between multiple objects. In this paper, we integrate both approaches into a new video model, called the Visual-Spatio-Temporal (VST) model to represent videos. The visual features are modeled in a topological approach and integrated with the spatio-temporal relationships. As a result, we defined rich sets of VST relationships which support and simplify the formulation of more semantical queries. An intuitive query interface which allows users to describe VST features of video objects by sketch and feature specification is presented. The conducted experiments prove the effectiveness of modeling and querying videos by the visual features of individual objects and the VST relationships between multiple objects.

This paper presents an object-oriented model to handle the temporal relationship for all of the multimedia objects at the presentation platform. Synchronization of the composite media objects is achieved by ensuring that all objects presented in the upcoming "manageable" period must be ready for execution. To this end, the nature of overlays is first investigated for various types of objects. Critical overlaps which are crucial in synchronization are also defined. The objective of synchronization is to ensure that the media objects can be initiated precisely at the critical point of the corresponding critical overlap. The concept of manageable presentation interval is introduced and the irreducible media group is defined. The resource scheduling of each presentation group for media object pre-fetch time versus buffer occupancy is also examined. Accordingly, a new model called group cascade object composition Petri-net (GCOCPN) is proposed and an algorithm to implement this temporal synchronization scheme is presented.

Although there are many multimedia presentation systems on the market, they have some shortcomings and most of them only can work on one single computer, and few of them can work on Web. Thus, in the thesis we develop a network multimedia presentation system to let users easily design the multimedia presentation without restriction on technology or presentation time and place. Our system includes 3 main components: User Interface that includes temporal specification editor, spatial specification editor and multimedia object interface, Presentation Interface and Knowledge Base. There is a dynamic homepage generator in our system and we propose a displaying algorithm based on the Allen's theory, that there exist 13 temporal relationships between two intervals, for synchronizing the media objects.

In this paper, we show that by suitably selecting a notation to construct synchronization requirement specifications (SRS) for multimedia presentation we can express the timing characteristics at an abstract level, verify the specification, and obtain a hardware implementation through a sequence of transformations of the specification. First, we introduce the notion of a well-formed SRS and its hardware model. Second, we model an SRS as a timed Petri net and interpret the transitions of the net as hardware signals. To obtain logic functions from the SRS, we simplify the net and obtain a signal transition graph satisfying the unique state coding property. Finally, we show how to obtain a logic-level design of synchronizers.

This paper discusses an object-oriented approach to temporal multimedia data modeling in OMEGA; a multimedia database management under development at the University of Library and Information Science. An object-orientated approach is necessary to integrate various types of heterogeneous multimedia data, but it has become clear that current object-oriented data models are not sufficient to represent multimedia data, particularly when they are temporal. For instance, the current object-oriented data models cannot describe objects whose attribute values change time-dependently. Also, they cannot represent temporal relationships among temporal multimedia objects. We characterize temporal objects as instances of a subclass of class TimeInterval with the temporal attributes and the temporal relationships. This temporal multimedia data model is designed upward compatible with the ODMG-93 standard object model. To organize a temporal multimedia database, a five temporal axes model for representing temporal multimedia objects is also introduced. The five temporal axes--an absolute, an internal, a quasi-, a physical, and a presentation time axis--are necessary to describe time-dependent properties of multimedia objects in modeling, implementing and use. A concrete example of this organization method is also illustrated.