We propose an information integration architecture for a man-machine interface to construct a new agent-based Computer Supported Cooperative Work (CSCW) system. The system acts as a clerk in cooperative work giving users the advantage of using cooperative work space. The system allows users to do their work in the style of an ordinary meeting because spontaneous expressions of speech and gestures by users are detected by sensors so that they can be integrated with a task model at several levels to create suitable responses in a man-machine interface. As a result, users can dedicate themselves to mutually understand other meeting members with no awareness of direction to the CSCW system. In this paper, we describe the whole system and its information integration architecture for the man-machine interface including, the principle of functions, the current status of the system and future directions.