This paper presents a new line extraction method to capture vectors based on contours and skeletons from line drawing raster images in which the lines are touched by characters or other lines. Conventionally, two line extraction methods have generally been used. One is a thinning method. The other is a medial line extraction method based on parallel pairs of contours. The thinning method tends to distort the extracted lines, especially at intersections and corners. On the other hand, the medial line extraction method has a poor capability as regards capturing correct lines at intersections. Contours are able to maintain edge shapes well, while skeletons preserve topological features; thus, a combination of these features effectively leads to the best fitting line. In the proposed method, the line which best fits the original image is selected from among various candidate lines. The candidates are created from several merged short skeleton fragments located between pairs of short contour fragments. The method is also extended to circular arc fitting. Experimental results show that the proposed line fitting method is effective.