This paper shows an experimental approach to the understanding system of natural language and pictorial patterns. The system is titles as ISOBAR (an Information understanding System Of BAsic weather Report). It can accept both linguistic and pictorial inputs. Also it produces either linguistic or pictorial, or both, output according to the commands which it received. The most remarkable point of the system is that the performance of the system is based on the semantic processing of the input. As ISOBAR's world is limited within the weather report of Japan and Far East Asian areas with associated weather charts, so the semantic background of the system is very narrow, and is quite a specific one. But the methodologies and algorithms in the system will be the first step for the embodiment of the more complicated and more general systems. ISOBAR has two operating modes in principle. One is the accumulation of meteorological information, and the other is its retrieval. According to the experimental outcomes, the performance of the system is almost good except for the picture processing procedures. Finally, the problems for the future research are remarked.