In this paper, a speech recognition system toward a phoneme typewriter without a language model is proposed. The system is realized as an integration of spectrogram reading knowledge and Time-Delay Neural Networks (TDNNs). The system mainly consists of two parts: in the consonant recognition part, a sophisticated integration of knowledge and TDNN is proposed. This improves not only recognition performance and segmentation accuracy, but also reduces insertion errors drastically. In the vowel recognition part, a TDNN is used for detection and rough segmentation using its time shift tolerance advantage. The knowledge part is mainly used for verification of categories and boundaries. A phoneme recognition experiment on 2,620 Japanese words, uttered by one male speaker showed a 91.4% (11,612/12,710) recognition rate, a 3.6% deletion error rate, a 5.0% substitution error rate and a 20.7% insertion error rate, for all Japanese phonemes. This good result was obtained without any language model.