A new class of least-squares algorithms is presented for adaptive filtering. The idea is to use a fixed set of directions and perform line search with one direction at a time in a cyclic fashion. These algorithms are called Euclidean Direction Search (EDS) algorithms. The fast version of this class is called the Fast-EDS or FEDS algorithm. It is shown to have O(N) computational complexity and a convergence rate comparable to that of the RLS algorithm. Computer simulations are presented to illustrate the performance of the new algorithm.