Broadband multimedia information environments are part of the next big advance in communications and computer technology. The use of multimedia infrastructures in offices is becoming very important. This paper deals with a service concept and human interfaces based on a paper metaphor. The proposed service offers the advantages of paper and eliminates the disadvantages. The power of multimedia's expressiveness, user interaction, and hypermedia technology are key points of our solution. We propose a system configuration for implementing the service/human interface.

This paper describes a task-oriented speech dialogue system based on spontaneous speech understanding and response generation (TOSBURG). The system has been developed for a fast food ordering task using speaker-independent keyword-based spontaneous speech understanding. Its purpose being to understand the user's intention from spontaneous speech, the system consists of a noise-robust keyword-spotter, a semantic keyword lattice parser, a user-initiated dialogue manager and a multimodal response generator. After noise immunity keyword-spotting is performed, the spotted keyword candidates are analyzed by a keyword lattice parser to extract the semantic content of the input speech. Then, referring to the dialogue history and context, the dialogue manager interprets the semantic content of the input speech. In cases where the interpretation is ambiguous or uncertain, the dialogue manager invites the user to confirm verbally the system's understanding of the speech input. The system's response to the user throughout the dialogue is multimodal; that is, several modes of communication (synthesized speech, text, animated facial expressions and ordered food items) are used to convey the system's state to the user. The object here is to emulate the multimodal interaction that occurs between humans, and so achieve more natural and efficient human-computer interaction. The real-time dialogue system has been constructed using two general purpose workstations and four DSP accelerators (520MFLOPS). Experimental results have shown the effectiveness of the newly developed speech dialogue system.

This paper presents unique specification environments for LOTOS, which is one of FDTs (Formal Description Techniques) developed in ISO. We first discuss the large gap in terms of syntax and semantics between informal specifications at the early stage of specification design and formal specifications based on FDT such as LOTOS. This large gap has been bridged by human intelligent works thus far. In order to bridge the large gap, we have designed user-friendly specification environments for FDTs. The outlines of SEGL (Specification Environment for G-LOTOS), CBP (Concept-Based Programming environment) and MBP (Model-Based Programming environment) are described. The effectiveness of software development under such an environment is demonstrated using application examples from OSI and non-OSI protocols.

The new notion of "multiuser interface", an interface for groups working together in a shared workspace, originated from the expansion of CSCW research and the spread of the groupware concept. This paper introduces a new multiuser interface design approach based on the translucent video overlay technique. This approach was realized in the multimedia desktop conference system Team WorkStation. Team WorkStation demonstrates that this translucent video overlay technique can achieve two different goals: (1) fused overlay for realizing the open shared workspace, and (2) selective overlay for effectively using limited screen space. This paper first describes the concept of open shared workspace and its implementation based on the fused overlay technique. The shared work window of Team-WorkStation is created by overlaying translucent individual workspace images. Each video layer is originally physically separated. However, because of the spatial relationships among marks on each layer, the set of overlaid layers provides users with sufficient semantics to fuse them into one image. The usefulness of this cognitive fusion was demonstrated through actual usage in design sessions. Second, the problem of screen space limitation is described. To solve this problem, the idea of ClearFace based on selective overlay is introduced. The ClearFace idea is to lay translucent live face video windows over a shared work window. Through the informal observations of experimental use in design sessions, little difficulty was experienced in switching the focus of attention between the face images and the drawing objects. The theory of selective looking accounts for this flexible perception mechanism. Although users can see drawn objects behind a face without difficulty, we found that users hesitate to draw figures or write text over face images. Because of this behavior, we devised the "movable" face window strategy.

In speech output expected as an ideal man-machine interface, there exists an important issue on emotion production in order to not only improve its naturalness but also achieve more sophisticated speech interaction between man and machine. Speech has two aspects, which are prosodic information and phonetic feature. For the purpose of application to natural and high quality speech synthesis, the role of prosody in speech perception has been studied. In this paper, prosodic components, which contribute to the expression of emotions and their intensity, are clarified by analyzing emotional speech and by conducting listening tests of synthetic speech. The analysis is performed by substituting the components of neutral speech (i.e., one with no particular emotion) with those of emotional speech preserving the temporal correspondence by means of DTW. It has been confirmed that prosodic components, which are composed of pitch structure, temporal structure and amplitude structure, contribute to the expression of emotions more than the spectral structure of speech. The results of listening tests using prosodic substituted speech show that temporal structure is the most important for the expression of anger, while all of three components are much more important for the intensity of anger. Pitch structure also plays a significant role in the expression of joy and sadness and their intensity. These results make it possible to convert neutral utterances into utterances expressing various emotions. The results can also be applied to controlling the emotional characteristics of speech in synthesis by rule.

Proposed here is an internal representation and mapping method for multimedia information in which retrieval is based on the impression documents desired to make. A user interface design for a system using this method is also proposed. The proposed internal representation and mapping method represents each desired document impression as an axis in a semantic space. Documents are represented as points in the space. Queries are represented as subspaces. The proposed user interface design employs a method of visual presentation of the semantic space. Pictorial examples are given to illustrate the range of impressions represented by the axes. The relations between the axes are represented by dispersion diagrams for the documents stored in the document base. With this method, the user can intuitively decide the appropriate subspace for his needs and can specify it directly. For evaluation purposes, a prototype system has been developed. An image retrieval experiment shows that the proposed internal representation and mapping method and the user interface design provide effective tools for information retrieval.

The development of computers capable of handling complex objects requires nonverbal interfaces that can bidirectionally mediate nonverbal communication including the gestures of both people and computers. Nonverbal expressions are poweful media for enriching and facilitating humancomputer interaction when used as interface languages. Four gestural modes are appropriate for human-computer interaction: the sign, indication, illustration and manipulation modes. All these modes can be conveyed by a generalized gesture interface that has specific processors for each mode. The basic component of the generalized gesture interface, a gesture dictionary, is proposed. The dictionary can accept sign and indicating gestures in which postures or body shapes are significant, pass their meaning to a computer and display gestures from the computer. For this purpose it converts body shapes into gestural codes by means of two code systems and, moreover, it performs bidirectional conversions of several gesture representations. This dictionary is applied to the translation of Japanese into sign language; it displays an actor who speaks the given Japanese sentences by gesture of sign words and finger alphabets. The performance of this application confirms the adequacy and usefulness of the gesture dictionary.

In order to realize flexible interaction control between user and information processing system, a special purpose user model is proposed on the basis of the knowledge-based design method of user interface system. The user-specific control knowledge of user-oriented interface environment is represented explicitly in the user model and utilized in the user-oriented interface system. Furthermore, the framework of user-oriented interface environment based on this user model called user-model-driven interface system, is proposed as one of user-adaptive human interface systems, in this paper. According to the proposed framework, a prototype system of the user-model-driven interface system is implemented and the facility of user-specific interaction control based on the user model has been verified with respect to an electronic mail handling task.

This paper discusses new trends and directions in human interface (HI) technologies, and the effects of HI technologies on human life or on social activities. This paper postulates that the HI subsumes man-machine interface, human-computer interaction, human-human interaction, human-organizational interface, human-environmental interface, human-social interface, etc. A new communication model, called Human Interface Communication Model (HICOM), and a new human dialogue model, called Human Interface Dialogue model (HIDIM), are derived by reexamining trends and directions on HI technologies from the viewpoint of functional meanings of interfaces, and from the viewpoint of a socially distributed cognition mechanism.