To improve metal oxide semiconductor field effect transistors (MOSFET) performance, flat interface between gate insulator and silicon (Si) should be realized. In this paper, the influence of Si surface roughness on electrical characteristics of MOSFET with hafnium oxynitride (HfON) gate insulator formed by electron cyclotron resonance (ECR) plasma sputtering was investigated for the first time. The surface roughness of Si substrate was reduced by Ar/4.9%H2 annealing utilizing conventional rapid thermal annealing (RTA) system. The obtained root-mean-square (RMS) roughness was 0.07nm (without annealed: 0.18nm). The HfON was formed by 2nm-thick HfN deposition followed by the Ar/O2 plasma oxidation. The electrical properties of HfON gate insulator were improved by reducing Si surface roughness. It was found that the current drivability of fabricated nMOSFETs was remarkably increased by reducing Si surface roughness. Furthermore, the reduction of Si surface roughness also leads to decrease of the 1/f noise.