The quadtree-based variable block sized prediction makes the biggest contribution for dramatically improved coding efficiency in the new video coding standard named HEVC. However, this technique takes about 75–80% computational complexity of an HEVC encoder. This paper brings forward an adaptive scheme that exploits temporal, spatial and transform-domain features to speed up the original quadtree-based prediction, targeting at high resolution videos. Before encoding starts, analysis on utilization ratio of each coding depth is performed to skip rarely adopted coding depths at frame level. Then, texture complexity (TC) measurement is applied to filter out none-contributable coding blocks for each largest coding unit (LCU). In this step, a dynamic threshold setting approach is proposed to make filtering adaptable to videos and coding parameters. Thirdly, during encoding process, sum of absolute quantized residual coefficient (SAQC) is used as criterion to prune useless coding blocks for both LCUs and 3232 blocks. By using proposed scheme, motion estimation is performed for prediction blocks within a narrowed range. Experiments show that proposed scheme outperforms existing works and speeds up original HEVC by a factor of up to 61.89% and by an average of 33.65% for 4kx2k video sequences. Meanwhile, the peak signal-to-noise ratio (PSNR) degradation and bit increment are trivial.