Video quality generally suffers from packet losses caused by an unreliable channel when video is transmitted over an error-prone wireless channel. This quality degradation is the main reason that a video compression encoder uses error-resilient coding to deal with the high packet-loss probability. The use of adequate error resilience can mitigate the effects of channel errors, but the coding efficiency for bit reduction will be decreased. On the other hand, H.264/AVC uses multiple reference frame (MRF) motion compensation for a higher coding efficiency. However, an increase in the number of reference frames in the H.264/AVC encoder has been recently observed, making the received video quality worse in the presence of transmission errors if the cyclic intra-refresh is used as the error-resilience method. This is because the reference-block selection in the MRF chooses blocks on the basis of the rate distortion optimization, irrespective of the intra-refresh coding. In this paper, a new error-resilient reference selection method is proposed to provide error resilience for MRF based motion compensation. The proposed error-resilient reference selection method achieves an average PSNR enhancement up to 0.5 to 2dB in 10% packet-loss-ratio environments. Therefore, the proposed method can be valuable in most MRF-based interactive video encoding system, which can be used for video broadcasting and mobile video conferencing over an erroneous network.

In transcoding, it is well known that refinement of the motion vectors is critical to enhance the quality of transcoded video while significantly reducing transcoding complexity. This paper proposes a novel cost model to estimate the rate-distortion cost of motion vector composition in order to develop a reliable motion vector re-estimation method that has reasonable computation cost. Based on a statistical analysis of motion compensated prediction errors, we design a basic form of the proposed cost model as a function of distance from the optimal motion vector. Simulations with a transcoder employing the proposed cost model demonstrate a significant quality gain over representative video transcoding schemes with no complexity increase.

The mode of intra prediction in H.264/AVC is encoded based on the most probable mode (MPM). To increase coding efficiency, the probability of the case that MPM is equal to coding mode of the current block should increase. In this paper we propose an efficient scheme to make MPM which is matched for the spatial direction property of pixels in the current block. Simulation results show that the proposed scheme gives significant coding gains when compared with the conventional techniques.

We present a new method for generating thumbnail images from H.264/AVC coded bit streams. What distinguishes our approach from previous works is that it determines the thumbnail image pixels by summing the residual and estimate block averages. The residual block averages are directly acquired in the transform domain and the estimated block averages are calculated in the spatial domain. Due to the construction of the reference pixels in the spatial domain, the proposed method eliminates the source of mismatch error, thus the result does not suffer any degradation. The thumbnail images produced by the proposed method are indistinguishable to the ones by the method that decodes the H.264/AVC intra coded bit streams and then scales them down. For most images, the proposed method also executes almost 3 times faster than the down-scaling method at frequently used bandwidths.

Rate control that is required to regulate the bitrate of video coding is critical to time-sensitive video applications used over networks. However, the H.264/AVC standard does not respond to scene changes, and this causes the transmission quality to deteriorate as a scene change occurs. In this work, a scene change is detected by comparing the ratio of the sum of absolute difference (SAD) between two consecutive frames. As the scene change is detected, the proposed method, which is modified from the reference software of H.264/AVC, re-assigns a quantization parameter (QP) value to regulate the bitrate. Because the inter-prediction works poorly for the scene-changed frame, the proposed method estimates its frame complexity based on the content, and further creates another Q-R model to assign QP. The adaptive rate control mechanism presented in this study can quickly respond to the heavy bitrate increment caused by a change of scene. Simulation results show that the proposed method improves the average peak signal noise ratio (PSNR) to approximately 1.1dB, with a smaller buffer size compared with the performance of the reference software JM version 17.2.

H.264/AVC estimates the prediction mode from the modes of neighboring blocks in order to reduce the quantity of bits needed to represent the best mode. The estimated mode is known as Most Probable Mode (MPM). When QP changes, the probability of MPM increases. In this paper, we propose an adaptive mode signaling method based on the probability of MPM. According to the probability of MPM, a conventional MPM method and the proposed merging method are adaptively used to signal the intra prediction mode. Experimental results show that the proposed method achieves an average bit savings of 0.727% and an average PSNR gain of 0.041dB. In the low bit rate condition, the proposed method achieves an average bit savings of 1.795%, and a PSNR gain of 0.103dB, compared to H.264/AVC.

The emerging high-efficiency video coding (HEVC) standard attempts to improve the coding efficiency by a factor of two over H.264/AVC through the use of new compression tools such as various block sizes with multiple directions. Although multiple-directional predictions are among the features contributing to the improved compression efficiency, its high computational complexity keeps it from being used widely. This paper presents an algorithm to skip backward and bi-directional predictions when merge or forward prediction modes are likely to be determined as the best mode. The proposed algorithm takes advantage of the fact that there is a cost relationship among multi-directional predictions and that the results of backward and bi-directional predictions are therefore predictable before the actual operations. After merge and forward predictions, if the expected results of backward and bi-directional predictions are worse than the results up to that point, then additional backward and bi-directional predictions to search for more accurate motion vectors are not performed. A simulation shows that the encoding time is reduced by about 15.18% with a marginal degradation in compression efficiency.

In this paper, we propose a novel joint rate distortion optimization (JRDO) model for intra prediction coding. The spatial prediction dependency is exploited by modeling the distortion propagation with a linear fitting function. A novel JRDO based Lagrange multiplier (LM) is derived from this model. To adapt to different blocks' distortion propagation characteristics, we also introduce a generalized multiple Lagrange multiplier (MLM) framework where some candidate LMs are used in the RDO process. Experiment results show that our proposed JRDO-MLM scheme is superior to the H.264/AVC encoder.

As a next-generation video compression standard, High Efficiency Video Coding (HEVC) achieves enhanced coding performance relative to prior standards such as H.264/AVC. In the new standard, the improved intra prediction plays an important role in bit rate saving. Meanwhile, it also involves significantly increased complexity, due to the adoption of a highly flexible coding unit structure and a large number of angular prediction modes. In this paper, we present a low-complexity intra prediction algorithm for HEVC. We first propose a fast preprocessing stage based on a simplified cost model. Based on its results, a fast prediction unit selection scheme reduces the number of prediction unit (PU) levels that requires fine processing from 5 to 2. To supply PU size decision with appropriate thresholds, a fast training method is also designed. Still based on the preprocessing results, an efficient mode selection scheme reduces the maximum number of angular modes to evaluate from 35 to 8. This achieves further algorithm acceleration by eliminating the necessity to perform fine Hadamard cost calculation. We also propose a 32×32 PU compensation scheme to alleviate the mismatch of cost functions for large transform units, which effectively improves coding performance for high-resolution sequences. In comparison with HM 7.0, the proposed algorithm achieves over 50% complexity reduction in terms of encoding time, with the corresponding bit rate increase lower than 2.0%. Moreover, the achieved complexity reduction is relatively stable and independent to sequence characteristics.

Video coding plays an important role in human life especially in communications. H.264/AVC is a prominent video coding standard that has been used in a variety of applications due to its high efficiency comes from several new coding techniques. However, the extremely high encoding complexity hinders itself from real-time applications. This paper presents a new encoding algorithm that makes use of particle swarm optimization (PSO) to train discriminant functions for classification based fast mode decision. Experimental results show that the proposed algorithm can successfully reduce encoding time at the expense of negligible quality degradation and bitrate increases.

H.264 video codec system requires big capacity and high bandwidth of Frame Store (FS) for buffering reference frames. The up-to-date three dimensional (3D) stacked Phase change Random Access Memory (PRAM) is the promising approach for on-chip caching the reference signals, as 3D stacking offers high memory bandwidth, while PRAM possesses the advantages in terms of high density and low leakage power. However, the write endurance problem, that is a PRAM cell can only tolerant limited number of write operations, becomes the main barrier in practical applications. This paper studies the wear reduction techniques of PRAM based FS in H.264 codec system. On the basis of rate-distortion theory, the content oriented selective writing mechanisms are proposed to reduce bit updates in the reference frame buffers. With the proposed control parameter a, our methods make the quantitative trade off between the quality degradation and the PRAM lifetime prolongation. Specifically, taking a in the range of [0.2,2], experimental results demonstrate that, our methods averagely save 29.9–35.5% bit-wise write operations and reduce 52–57% power, at the cost of 12.95–20.57% BDBR bit-rate increase accordingly.

As the successive video compression standard of H.264/AVC, High Efficiency Video Codec (HEVC) will play an important role in video coding area. In the deblocking filter part, HEVC inherits the basic property of H.264/AVC and gives some new features. Based on this variation, this paper introduces a novel dual-mode deblocking filter architecture which could support both of the HEVC and H.264/AVC standards. For HEVC standard, the proposed symmetric unified-cross unit (SUCU) based filtering scheme greatly reduces the design complexity. As a result, processing a 1616 block needs 24 clock cycles. For H.264/AVC standard, it takes 48 clock cycles for a 1616 macro-block (MB). In synthesis result, the proposed architecture occupies 41.6k equivalent gate count at frequency of 200 MHz in SMIC 65 nm library, which could satisfy the throughput requirement of super hi-vision (SHV) on 60 fps. With filter reusing scheme, the universal design for the two standards saves 30% gate counts than the dedicated ones in filter part. In addition, the total power consumption could be reduced by 57.2% with skipping mode when the edges need not be filtered.

A motion estimation (ME) multimedia processor was developed by employing dynamic voltage and frequency scaling (DVFS) technique to greatly reduce the power dissipation. To make full use of the advantages of DVFS technique, a fast motion estimation (ME) algorithm was also developed. It can adaptively predict the optimum supply voltage and the optimum clock frequency before ME process starts for each macro-block for encoding. Power dissipation of the 90-nm CMOS DVFS controlled multimedia processor, which contained an absolute difference accumulator as well as a small on-chip DC/DC level converter, a minimum value detector and DVFS controller, was reduced to 38.48 µW, which was only 3.261% that of a conventional multimedia processor.

The emerging High Efficiency Video Coding (HEVC) standard attempts to improve the coding efficiency by a factor of two over H.264/AVC through the use of new compression tools with high computational complexity. Although multipledirectional prediction is one of the features contributing to the improved compression efficiency, the computational complexity for prediction increases significantly. This paper presents an early uni-directional prediction decision algorithm. The proposed algorithm takes advantage of the property of HEVC that it supports a deep quad-tree block structure. Statistical observation shows that the correlation of prediction direction among different blocks which share same area is very high. Based on this observation, the mode of the current block is determined early according to the mode of upper blocks. Bi-directional prediction is not performed when the upper block is encoded as the uni-directional prediction mode. A simulation shows that it reduces ME operation time by about 22.7% with a marginal drop in compression efficiency.

In this letter, we propose a novel intra prediction coding scheme for H.264/AVC. Based on our proposed minimum distance prediction (MDP) scheme, the optimal reference samples for predicting the current pixel can be adaptively updated corresponding to different video contents. The experimental results show that up to 2 dB and 1 dB coding gains can be achieved with the proposed method for QCIF and CIF sequences respectively.

REMUS-II (REconfigurable MUltimedia System 2) is a coarse-grained dynamically reconfigurable computing system for multimedia and communication baseband processing. This paper proposes a real-time H.264 baseline profile encoder on REMUS-II. First, we propose an overall mapping flow for mapping algorithms onto the platform of REMUS-II system and then illustrate it by implementing the H.264 encoder. Second, parallel and pipelining techniques are considered for fully exploiting the abundant computing resources of REMUS-II, thus increasing total computing throughput and solving high computational complexity of H.264 encoder. Besides, some data-reuse schemes are also used to increase data-reuse ratio and therefore reduce the required data bandwidth. Third, we propose a scheduling scheme to manage run-time reconfiguration of the system. The scheduling is also responsible for synchronizing the data communication between tasks and handling conflict between hardware resources. Experimental results prove that the REMUS-MB (REMUS-II version for mobile applications) system can perform a real-time H.264/AVC baseline profile encoder. The encoder can encode CIF@30 fps video sequences with two reference frames and maximum search range of [-16,15]. The implementation, thereby, can be applied to handheld devices targeted at mobile multimedia applications. The platform of REMUS-MB system is designed and synthesized by using TSMC 65 nm low power technology. The die size of REMUS-MB is 13.97 mm2. REMUS-MB consumes, on average, about 100 mW while working at 166 MHz. To my knowledge, in the literature this is the first implementation of H.264 encoding algorithm on a coarse-grained dynamically reconfigurable computing system.

Blocking artifacts are introduced in many block-based coding systems, and its reduction can significantly improve the subjective quality of compressed video. The H.264/AVC uses an in-loop deblocking filter to remove the blocking artifacts. The filter considers some coding conditions in its adaptive deblocking filtering such as coded block pattern (CBP), motion vector, macroblock type, etc. for inter-predicted blocks, however, it does not consider much for intra-coded blocks. In this paper, we utilize the human visual system (HVS) characteristic and the local characteristic of image blocks to modify the boundary strength (BS) of the intra-deblocking filter in order to gain improvement in the subjective quality and also to reduce the complexity in filtering intra coded slices. In addition, we propose a low-complexity deblocking method which utilizes the correlation between vertical and horizontal boundaries of a block in inter coded slices. Experimental results show that our proposed method achieves not only significant gain in the subjective quality but also some PSNR gain, and reduces the computational complexity of the deblocking filter by 36.23% on average.

In this paper, an improved B-picture coding algorithm based on the symmetric bi-directional motion estimation (ME) is proposed. In addition to the block match error between blocks in the forward and backward reference frames, the proposed method exploits the previously-reconstructed template regions in the current and reference frames for bi-directional ME. The side match error between the predicted target block and its template is also employed in order to alleviate block discontinuities. To efficiently perform ME, an initial motion vector (MV) is adaptively derived by exploiting temporal correlations. Experimental results show that the number of generated bits is reduced by up to 9.31% when the proposed algorithm is employed as a new macroblock (MB) coding mode for the H.264/AVC standard.

High-definition (HD) videos become more and more popular on portable devices these years. Due to the resolution mismatch between the HD video sources and the relative low-resolution screens of portable devices, the HD videos are usually fully decoded and then down-sampled (FDDS) for the displays, which not only increase the cost of both computational power and memory bandwidth, but also lose the details of video contents. In this paper, an encoder-unconstrained partial decoding scheme for H.264/AVC is presented to solve the problem by only decoding the object of interest (OOI) related region, which is defined by users. A simplified compression domain tracking method is utilized to ensure that the OOI locates in the center of the display area. The decoded partial area (DPA) adaptation, the reference block relocation (RBR) and co-located temporal Intra prediction (CTIP) methods are proposed to improve the visual quality for the DPA with low complexity. The simulation results show that the proposed partial decoding scheme provides an average of 50.16% decoding time reduction comparing to the fully decoding process. The displayed region also presents the original HD granularity of OOI. The proposed partial decoding scheme is especially useful for displaying HD video on the devices of which the battery life is a crucial factor.

Scalable video coding offers efficient video transmission to a variety of display devices over heterogeneous and error-prone networks. Scalable video coding has been strenuously researched in recent years and state-of-the-art international coding with scalability has been standardized as SVC, which is an extension of H.264/AVC. This paper summarizes the recent advanced research that has been done for improving the quality and reducing the complexity of scalable video coding (including SVC), as well as for improving the quality assessment techniques. It is intended to give researchers a critical, technical overview of what is required to develop more efficient scalable video coding in the future.