In this paper, we propose an advanced location-based service that we call a direction-based service, which utilizes both the position and direction of a user. The direction-based service enables a user to point to an object of interest for command or investigation. We also describe the design, implementation and evaluations of a direction-based service system named Azim. With this system, the direction of the user can be obtained by a magnetic-based direction sensor. The sensor is also used for azimuth-based position estimation, in which a user's position is estimated by having the user point to and measure azimuths of several markers or objects whose positions are already known. Because this approach does not require any other accurate position sensors or positive beacons, it can be deployed cost-effectively. Also, because the measurements are naturally associated with some degree of error, the position is calculated as a probability distribution. The calculation considers the error of direction measurement and the pre-obtained field information such as obstacles and magnetic field disturbance, which enables robust position measurements even in geomagnetically disturbed environments. For wide-area use, the system also utilizes a wireless LAN to obtain rough position information by identifying base stations. We have implemented a prototype system for the proposed method and some applications for the direction-based services. Furthermore, we have conducted experiments both indoors and outdoors, and exemplified that positioning accuracy by the proposed method is precise enough for a direction-based service.