Conventional system models such as the finite impulse response (FIR) model, autoregressive external input (ARX) model, time delay neural network (TDNN), and recurrent neural network (RNN) depend on short-term memory when modeling a discrete time system. However, short-term memory can be inefficient with a varying appearance speed of I/O data. This inefficiency is referred to herein as the Varying Appearance Speed Problem (VASP) and demonstrated by analyzing impulse and frequency responses. Simulation results indicate that the varying appearance speed leads to asymmetrical cycles. Unable to prevent the memory effect from extensively disturbing the next output cycle, conventional models simulate the systems inaccurately. A solution using rate independent memory is then proposed. Only concerned with the previous extreme inputs, rate independent memory differs from short-term memory and potentially prevents a system model from the impact of varying appearance speeds. To demonstrate the VASP and verify the proposed model, this study conducts three experiments, i.e. (a) learning random step trajectories of circular and trefoil shapes, (b) modeling the relationship between the economic leading and coincident indexes, (c) simulating the connection between the ground-water level and land subsidence. In contrast to conventional models, the model presented here performs better in terms of mean square errors.