An efficient algorithm is proposed for the adaptive sizing of a tracking window in correlation-based video tracking. Since a tracking window specifies a support region when estimating a target displacement, the ability to adapt the window size relative to a moving target significantly influences the performance of video tracking. The basic strategy of the proposed algorithm is to maintain the occupancy rate of the target in the tracking window within a specified range. As such, the proposed algorithm measures the occupancy rate using the ratio of the power of the spatial gradients in the edge subwindows, which edge the tracking window, to that in the tracking window. In addition, the level of any complex background and additive white noise is also evaluated to reduce their effect on the gradients. Experimental results using various artificial and real sequences confirm that the proposed algorithm can effectively adjust a tracking window to a moving target and is robust to a complex background and noise.