A new steepest descent linear adaptive algorithm, called the proportion-sign algorithm (PSA), is introduced and its performance analysis is presented when the signals are from zero-mean jointly stationary Gaussian processes. The PSA improves the convergence speed over the least mean square (LMS) algorithm without overly degrading the steady-state error performance and has the robustness to impulsive interference occurring in the desired response by adding a minimal amount of computational complexity. Computer simulations are presented that show these advantages of the PSA over the LMS algorithm and demonstrate a close match between theoretical and empirical results to verify our analysis.