This paper presents a novel filter to keep from over-smoothing the edges and corners and rectify the outliers in the flow field after each incremental computation step, which plays a key role during the process of estimating flow field. This filter works according to the spatial-temporal derivatives distance of the input image and velocity field distance, whose principle is more reasonable in filtering mechanism for optical flow than other existing nonlinear filters. Moreover, we regard the spatial-temporal derivatives as new powerful descriptions of different motion layers or regions and give a detailed explanation. Experimental results show that our proposed method achieves better performance.