This letter reveals that whole link reciprocity does not exist in general amplify-and-forward (AF) time division duplex (TDD) relay systems due to the gain matrix. To resolve this problem, a novel gain matrix design method is proposed. Any existing gain matrix design criterion can be adopted in the downlink (uplink) to ensure optimal performance, and the proposed scheme is used in the uplink (downlink), with small adjustment, to keep whole link reciprocity. Simulation results show that, the proposed method can maintain whole link reciprocity without performance loss.