Automatic disassembly of railway fasteners is of great significance for improving the efficiency of replacing rails. The accurate positioning of fastener is the key factor to realize automatic disassembling. However, most of the existing literature mainly focuses on fastener region positioning and the literature on accurate positioning of fasteners is scarce. Therefore, this paper constructed a visual inspection system for accurate positioning of fastener (VISP). At first, VISP acquires railway image by image acquisition subsystem, and then the subimage of fastener can be obtained by coarse-to-fine method. Subsequently, the accurate positioning of fasteners can be completed by three steps, including contrast enhancement, binarization and spike region extraction. The validity and robustness of the VISP were verified by vast experiments. The results show that VISP has competitive performance for accurate positioning of fasteners. The single positioning time is about 260ms, and the average positioning accuracy is above 90%. Thus, it is with theoretical interest and potential industrial application.