An on-channel repeater (OCR) performing simultaneous reception and transmission at the same frequency is beneficial to improve spectral efficiency and coverage. In an OCR, it is important to cancel the feedback interference caused by imperfect isolation between the transmit and receive antennas, and least mean square (LMS) based adaptive filters are commonly used for this purpose. In this paper, we analyze the performance of the LMS based adaptive feedback canceller in terms of its transient behavior and the steady-state mean square error (MSE). Through a theoretical analysis, we derive iterative equations to compute transient MSEs and provide a procedure to simply evaluate steady-state MSEs for the adaptive feedback canceller. Simulation results performed to verify the theoretical MSEs show good agreement between the proposed theoretical analysis and the empirical results.