This paper presents a new algorithm that can solve the problem of selecting appropriate update step size in the LMS algorithm. The proposed algorithm, called a Complementary Pair LMS (CP-LMS) algorithm, consists of two adaptive filters with different update step sizes operating in parallel, one filter re-initializing the other with the better coefficient estimates whenever possible. This new algorithm provides the faster convergence speed and the smaller steady-state error than those of a single filter with a fixed or variable step size.