This paper presents a quantized gradient descent algorithm for distributed nonconvex optimization in multiagent systems that takes into account the bandwidth limitation of communication channels. Each agent encodes its estimation variable using a zoom-in parameter and sends the quantized intermediate variable to the neighboring agents. Then, each agent updates the estimation by decoding the received information. In this paper, we show that all agents achieve consensus and their estimated variables converge to a critical point in the optimization problem. A numerical example of a nonconvex logistic regression shows that there is a trade-off between the convergence rate of the estimation and the communication bandwidth.

Model-based reinforcement learning uses the gathered information, during each experience, more efficiently than model-free reinforcement learning. This is especially interesting in multiagent systems, since a large number of experiences are necessary to achieve a good performance. In this paper, model-based reinforcement learning is developed for a group of self-interested agents with sequential action selection based on traditional prioritized sweeping. Every single situation of decision making in this learning process, called extensive Markov game, is modeled as n-person general-sum extensive form game with perfect information. A modified version of backward induction is proposed for action selection, which adjusts the tradeoff between selecting subgame perfect equilibrium points, as the optimal joint actions, and learning new joint actions. The algorithm is proved to be convergent and discussed based on the new results on the convergence of the traditional prioritized sweeping.

In this paper, we present a new three-dimensional (3D) virtual environment (3DVE) system named "QuViE/P", which can enhance quality of service (QoS), that users actually feel, as good as possible when resources of computers and networks are limited. To realize this, we focus on characteristics of user's perceptual quality evaluation on 3D objects. We propose an effective QoS control scheme for QuViE/P by introducing relationships between system's internal quality parameters and user's perceptual quality parameters. This scheme can appropriately maintain the QoS of the 3DVE system and it is expected to improve convenience when using 3DVE system where resources are insufficient. We designed and implemented a prototype of QuViE/P using a multiagent framework. The experiment results show that even when the computer resource is reduced to 20% of the required amount, the proposed scheme can maintain the quality of important objects to a certain level.

We present a design of knowledge circulation framework for quality of service (QoS) control of multimedia communication service (MCS). This framework aims to realizing user oriented and resource aware MCS by enabling effective placement of QoS control knowledge on the network. In this paper, we propose a conceptual design of the framework with knowledge-based multiagent system. In this framework, QoS control knowledge is actively circulated by getting on the agents. We implement a prototype of real-time bidirectional MCS (videoconference system) using this framework, and show initial experiment results using it to evaluate the effectiveness of the framework.

One fundamental issue in multiagent reinforcement learning is how to deal with the limited local knowledge of an agent in order to achieve effective learning. In this paper, we argue that this issue can be more effectively solved if agents are equipped with a consistent global view. We achieve this by requiring agents to follow an interacting protocol. The properties of the protocol are derived and theoretically analyzed. A distributed protocol that satisfies these properties is presented. The experimental evaluations are conducted for a well-known test-case (i.e., pursuit game) in the context of two learning algorithms. The results show that the protocol is effective and the reinforcement learning algorithms using it perform much better.

Software agents are knowledgeable, autonomous, situated and interactive software entities. Agents' interactions are of special importance when a group of agents interact with each other to solve a problem that is beyond the capability and knowledge of each individual. Efficiency, performance and overall quality of the multi-agent applications depend mainly on how the agents interact with each other effectively. In this paper, we suggest an agent model by which we can clearly distinguish different agent's interaction scenarios. The model has five attributes: goal, control, interface, identity and knowledge base. Using the model, we analyze and describe possible scenarios; devise the appropriate reasoning and decision making techniques for each scenario; and build a library of reasoning and decision making modules that can be used readily in the design and implementation of multiagent systems.

Reinforcement Learning (RL) is an efficient learning method for solving problems that learning agents have no knowledge about the environment a priori. Ant Colony System (ACS) provides an indirect communication method among cooperating agents, which is an efficient method for solving combinatorial optimization problems. Based on the cooperating method of the indirect communication in ACS and the update policy of reinforcement values in RL, this paper proposes the Q-ACS multiagent cooperating learning method that can be applied to both Markov Decision Processes (MDPs) and combinatorial optimization problems. The advantage of the Q-ACS method is for the learning agents to share episodes beneficial to the exploitation of the accumulated knowledge and utilize the learned reinforcement values efficiently. Further, taking the visited times into account, this paper proposes the T-ACS multiagent learning method. The merit of the T-ACS method is that the learning agents share better policies beneficial to the exploration during agent's learning processes. Meanwhile, considering the Q-ACS and the T-ACS as homogeneous multiagent learning methods, in the light of indirect media communication among heterogeneous multiagent, this paper presents a heterogeneous multiagent RL method, the D-ACS that composites the learning policy of the Q-ACS and the T-ACS, and takes different updating policies of reinforcement values. The agents in our methods are given a simply cooperating way exchanging information in the form of reinforcement values updated in the common model of all agents. Owning the advantages of exploring the unknown environment actively and exploiting learned knowledge effectively, the proposed methods are able to solve both problems with MDPs and combinatorial optimization problems effectively. The results of experiments on hunter game and traveling salesman problem demonstrate that our methods perform competitively with representative methods on each domain respectively.

We present a design and implementation of a QoS control mechanism in an Adaptive Multimedia Communication System (AMCS) using multiagent-based computing technology. In this paper, we first define functional requirements for AMCS. Subsequently we describe the design and implementation of AMCS with a knowledge-based multiagent framework to fulfill the functional requirements. Moreover we evaluate the adaptability of the prototype systems of AMCS with the operational situations observed in its experiments. From the result of the experiments, we conclude that the multiagent-based design and implementation is reasonable for construction of AMCS.

Recently a high-resolution image that has more than one million pixels is available easily. However, such an image requires much processing time and memory for an image understanding system. In this paper, we propose an integrated image understanding system of multi-resolution analysis and multi-agent-based architecture for high-resolution images. The system we propose in this paper has capability to treat with a high-resolution image effectively without much extra cost. We implemented an experimental system for images of indoor scenes.

In order to make the ATM network fault-tolerant and the network service flexible, a method for the setting up of backup virtual paths (VP's for short) using multiagents is effective with respect to adaptability to change of network resource and user requirements, examples of which are failure of nodes and links and addition of VP's, respectively. In this method, under the assumption that candidates of backup VP's between different pairs of source and destination nodes are given, the optimum backup VP's are obtained by exchanging information among agents autonomously. First, this paper proposes measures for determining backup VP's between different pairs of source and destination nodes. Next, this paper presents simulation results to evaluate the adaptability of the method. The results show that the method efficiently obtains the optimum backup VP's even when the number of backup VP's increases and that different idle time at each destination node enables to shorten the total processing time while keeping complete detection of shared links.

In this paper, we propose a method for querying heterogeneous molecular biology databases. Since molecular biology data are distributed into multiple databases that represent different biological domains, it is highly desirable to integrate data together with the correlations between the domains. However, since the total amount of such databases is very large and the data contained are frequently updated, it is difficult to maintain the integration of the entire contents of the databases. Thus, we propose a method for dynamic integration based on user demand, which is expressed with an OQL-based query language. By restricting search space according to user demand, the cost of integration can be reduced considerably. Multiple databases also exhibit much heterogeneity, such as semantic mismatching between the database schemas. For example, many databases employ their own independent terminology. For this reason, it is usually required that the task for integrating data based on a user demand should be carried out transitively; first search each database for data that satisfy the demand, then repeatedly retrieve other data that match the previously found data across every database. To cope with this issue, we introduce two types of agents; a database agent and a user agent, which reside at each database and at a user, respectively. The integration task is performed by the agents; user agents generate demands for retrieving data based on the previous search results by database agents, and database agents search their databases for data that satisfy the demands received from the user agents. We have developed a prototype system on a network of workstations. The system integrates four databases; GenBank (a DNA nucleotide database), SWISS-PROT, PIR (protein amino-acid sequence databases), and PDB (a protein three-dimensional structure database). Although the sizes of GenBank and PDB are each over one billion bytes, the system achieved good performance in searching such very large heterogeneous databases.