As a general basis for constructing a cooperative and flexible dialogue system, we are interested in modelling the inference process of an agent who participates in a dialogue. For this purpose, it is natural and powerful to model it in his general cognitive framework for problem solving. This paper presents such a framework. In this framework, we represent agent's mental states in the form called Mental World Structure, which consists of multiple mental worlds. Each mental world is a set of mental propositions and corresponds to one modal context, that is, a specific point of view. Modalities in an agent's mental states are represented by path expressions, which are first class citizens of the system and can be composed each other to make up composite modalities. With Mental World Structure, we can handle modalities more flexibly than ordinary modal logics, situation theory and other representation systems. We incorporate smoothly into the structure three basic inference procedures, that is, deduction, abduction and truth maintenance. Precise definitions of the structure and the inference procedures are given. Furthermore, we explain as examples, several cooperative dialogues in our framework.