The H.264/AVC standard provides several new error-resilient features to enable the reliable transmission of compressed video signals over lossy packet networks. Flexible Macroblock Ordering (FMO) is one of the most interesting resilient features within the H.264/AVC standard. Unlike former standards, in which slices were constructed out of consecutive raster scan macroblocks, FMO suggests new slices composed of spatially distributed Macroblocks (MBs), and organized in a mixed-up fashion. H.264/AVC specifies seven types of FMO. The standard defines also an explicit FMO type (Type 6), which allows explicitly assignment of each MB within the frame to any available slice groups. Therefore new FMO types can be used and integrated into H264/AVC without violating the standard. In this paper we propose a new Explicit Chessboard-Wipe (ECW) Flexible Macroblocks Ordering (FMO) technique, which outperforms all other FMO types. The new ECW ordering results in effective error scattering which maximizes the number of correctly received macroblocks located around corrupted macroblocks, leading to better error concealment. Performance evaluations demonstrate that the proposed Explicit FMO approach outperforms all the FMO types. Both subjective and objective visual quality comparative study has been also carried out in order to validate the proposed approach.

The intra-prediction unit is an essential part of H.264 codec, since it reduces the amount of data to be encoded by predicting pixel values (luminance and chrominance) from their neighboring blocks. A dedicated hardware implementation for the intra-prediction unit is required for real-time encoding and decoding of high resolution video data. To develop a cost-effective intra-prediction unit this paper proposes a novel architecture of intra-predictor generator, the core part of intra-prediction unit. The proposed intra-predictor generator enables the intra-prediction unit to achieve significant clock cycle reduction with approximately the same gate count, as compared to Huang's work [3].

In order to reduce spatial redundancies, the H.264/AVC Intra coding provides nine directional prediction modes including DC prediction for every 44 block, but it needs a lot of overhead bits to represent the nine directional prediction modes for every 44 block. To compress the directional mode bits efficiently, the most probable mode is estimated by using the correlation between the prediction mode of spatially adjacent blocks and that of the current block. In this paper, a new method for estimating the most probable mode is proposed by using the directional information of the prediction mode of the adjacent blocks. Experimental results show that the proposed method is able to achieve a coding gain of about 0.2 dB on average at low bit rate.

This letter proposes a new algorithm of refining the quantization parameter in H.264 real-time encoding. In the H.264 encoding, the quantization parameter computed according to the quadratic rate model is not accurate in meeting the target bit rate. In order to make the actual encoded bit rate closer to the target bit rate, ρ-domain rate model is introduced in our proposed quantization parameter refinement algorithm. Simulation results show that the proposed algorithm achieves obvious gain in PSNR and has stabler encoded bit rate compared to Jiang's algorithm.

A bit-depth scalability is proposed in an adaptive way based on modified inter-layer predictions of the spatial scalability. A simple prediction for high dynamic range (HDR) sequences is implemented to reduce the redundancy of the residual signals between the base layer which contains low dynamic range (LDR) sequences and the enhancement layer which contains HDR sequences by using scaling and offset values.

In this paper, we propose a method of resizing images in the integer DCT domain employed by H.264/AVC. To accomplish this, we first derive the matrix scaling the image resolution, and then factorize the scaled DCT blocks and the post scaling factors (PF) from the matrix obtained from the multiplication of the scaling matrix and the original integer DCT blocks. Then, we separate the scaled DCT blocks into the integer scaling matrix and the scaled integer DCT blocks. The experiments show that the proposed method produces nearly the same performance as those operating in the real DCT domain.

One hardware efficient and high speed architecture for variable block size motion estimation (VBSME) in H.264 is presented in this paper. By improving the pipeline structure and processing element (PE) circuits, the system latency and hardware cost is reduced, which makes this structure more hardware efficient than the original Propagate Partial SAD architecture. For small and middle frame size picture's coding, the proposed structure can save 12.1% hardware cost compared with original Propagate Partial SAD structure. In the case of HDTV, since small inter modes trivially contribute to the coding quality, we remove modes below 88 in our design. By adopting mode reduction technique, when the set number of PE array is less than 8, the proposed mode reduction based Propagate Partial SAD structure can work at faster clock speed and consume less hardware cost than widely used SAD Tree architecture. It is more robust to the high speed timing constraint when parallel processing is considered. With TSMC 0.18 µm technology in worst work conditions (1.62 V, 125), its peak throughput of 8-set PE array structure is 720p@30 Hz with 12864 search range and 5 reference frames. 12 k gates hardware cost can be reduced by our design compared with the parallel SAD Tree architecture.

We propose a low cost and stand-alone platform-based SoC for H.264/AVC decoder, whose target is practical mobile applications such as a handheld video player. Both low cost and stand-alone solutions are particularly emphasized. The SoC, consisting of RISC core and decoder core, has advantages in terms of flexibility, testability and various I/O interfaces. For decoder core design, the proposed H.264/AVC coprocessor in the SoC employs a new block pipelining scheme instead of a conventional macroblock or a hybrid one, which greatly contribute to reducing drastically the size of the core and its pipelining buffer. In addition, the decoder schedule is optimized to block level which is easy to be programmed. Actually, the core size is reduced to 138 KGate with 3.5 kbyte memory. In our practical development, a single external SDRAM is sufficient for both reference frame buffer and display buffer. Various peripheral interfaces such as a compact flash, a digital broadcast receiver and a LCD driver are also provided on a chip.

In this paper, we propose memory and performance optimized architecture to accelerate the operation speed of adaptive deblocking filter for H.264/JVT/AVC video coding. The proposed deblocking filter executes loading/storing and filtering operations with only 192 cycles for 1 macroblock. Only 244 internal buffers and 3216 internal SRAM are adopted for the buffering operation of deblocking filter with I/O bandwidth of 32 bit. The proposed architecture can process the filtering operation for 1 macroblock with less filtering cycles and lower memory sizes than some conventional approaches of realizing deblocking filter. The efficient hardware architecture is implemented with novel data arrangement, hybrid filter scheduling and minimum number of buffer. The proposed architecture is suitable for low cost and real-time applications, and the real-time decoding with 1080HD (19201088@30 fps) can be easily achieved when working frequency is 70 MHz.

Comparing with search pattern motion estimation (ME) algorithms, adaptive search range (ASR) algorithms are more fundamental, regular and flexible. In variable block size motion estimation (VBSME), ASR algorithms can be applied whether on a whole frame (frame level), or on an entire macroblock which includes up to forty-one blocks (macroblock level), or just on a single block (block level). In the other hand, in H.264/AVC, not the motion vectors (MVs) but the motion vector differences (MVDs) are coded and the median motion vector predictors (median-MVPs) are used to place the search centers. In this sense, it can be thought that the search windows (SWs) are centered at the positions pointed by median-MVPs, the search ranges (SRs) play the role of limiting MVDs. Thus it is reasonable for considering using MVDs to predict SRs. In this paper, one of the MB level and two of the block level, at all three MVD based SR prediction algorithms are proposed. VBSME based experiments are carried out to assess the proposed algorithms. Comparisons between the proposed three algorithms and the previously proposed one given in [8] are done in terms of encoding quality and computational complexity.

We describe a sub 100-mW H.264 MP@L4.1 integer-pel motion estimation processor core for low power video encoder. It supports macro block adaptive frame field (MBAFF) encoding and bi-directional prediction for a resolution of 19201080 pixels at 30 fps. The proposed processor features a novel hierarchical algorithm, reconfigurable ring-connected systolic array architecture and segmentation-free, rectangle-access search window buffer. The hierarchical algorithm consists of a fine search and a coarse search. A complementary recursive cross search is newly introduced in the coarse search. The fine search is adaptively carried out, based on an image analysis result obtained by the coarse search. The proposed systolic array architecture minimizes the amount of transferred data, and lowers computation cycles for the coarse and fine searches. In addition, we propose a novel search window buffer SRAM that has instantaneous accessibility to a rectangular area with arbitrary location. The processor core has been designed with a 90 nm CMOS design rule. Core size is 2.52.5 mm2. One core supports one-reference-frame and dissipates 48 mW at 1 V. Two core configuration consumes 96 mW for two-reference-frame search.

Variable Block Size Motion Estimation (VBSME) costs a lot of computation during video coding. Search range reduction algorithm is widely used to reduce computational cost of motion estimation. Current VBSME designs are not suitable for this algorithm. This paper proposes a reconfigurable design of VBSME which can be efficiently used with search range reduction algorithm. While using proposed design, nm reference MBs form an MB array which can be processed in parallel. n and m can be configured according to the new search range shape calculated by algorithm. In this way, the parallelism of proposed design is very flexible and can be adapted to any search range shape. The hardware resource is also fully used while performing VBSME. There are two primary reconfigurable modules in this design: PEGA (PE Group Array) and SAD comparator. By using TSMC 0.18 µm standard cell library, the implementation results show that the hardware cost of design which uses 16 PEGs (PE Groups) is about 179 K Gates, the clock frequency is 167 MHz.

Motion estimation (ME) is a computation-intensive module in video coding system. In MPEG-2 to H.264 transcoding, motion vector (MV) from MPEG-2 reused as search center in H.264 encoder is a simple but effective technique to simplify ME processing. However, directly applying MPEG-2 MV as search center will bring difficulties on application of data reuse method in hardware design, because the irregular overlapping of search windows between successive macro block (MB). In this paper, we propose a search window reuse scheme for transcoding, especially for HDTV application. By utilizing the similarity between neighboring MV, overlapping area of search windows can be regularized. Experiment results show that our method achieves average 93.1% search window reuse-rate in HDTV720p sequence with almost no video quality degradation. Compared to transcoding method without any data reuse scheme, bandwidth of the proposed method can be reduced to 40.6% of that.

In this paper, a fast mode decision method for intra-prediction is proposed to reduce the computational complexity of H.264/AVC encoders. With edge information, we propose a novel fast estimation algorithm to reduce the computation overhead of H.264/AVC for mode selection, where the edge direction of each coding block is detected from only part of the transformed coefficients. Hence, the computation complexity is greatly reduced. Experimental results show that the proposed fast mode decision method can eliminate about 81.34% encoding time for all intra-frame sequences with acceptable degradation of averaged PSNR and bitrates.

This paper introduces an adaptive low complexity pre-processing filter to improve the coding performance of seriously degraded video sequences that is caused by the additive noise. The additive noise leads to a decrease in coding performance due to the high frequency components. By incorporating local statistics and quantization parameter into filtering process, the spurious noise is significantly attenuated and coding efficiency is improved for given quantization step size. In order to reduce the complexity of the pre-processing filter, the simplified local statistics and quantization parameter are introduced. The simulation results show the capability of the proposed algorithm.

H.264/AVC is the newest video coding standard. There are many new features in it which can be easily used for video encryption. In this paper, we propose a new scheme to do video encryption for H.264/AVC video compression standard. We define Unequal Secure Encryption (USE) as an approach that applies different encryption schemes (with different security strength) to different parts of compressed video data. This USE scheme includes two parts: video data classification and unequal secure video data encryption. Firstly, we classify the video data into two partitions: Important data partition and unimportant data partition. Important data partition has small size with high secure protection, while unimportant data partition has large size with low secure protection. Secondly, we use AES as a block cipher to encrypt the important data partition and use LEX as a stream cipher to encrypt the unimportant data partition. AES is the most widely used symmetric cryptography which can ensure high security. LEX is a new stream cipher which is based on AES and its computational cost is much lower than AES. In this way, our scheme can achieve both high security and low computational cost. Besides the USE scheme, we propose a low cost design of hybrid AES/LEX encryption module. Our experimental results show that the computational cost of the USE scheme is low (about 25% of naive encryption at Level 0 with VEA used). The hardware cost for hybrid AES/LEX module is 4678 Gates and the AES encryption throughput is about 50 Mbps.

We propose a method to prevent the degradation of decoded MPEG pictures caused by video transmission over error-prone networks. In this paper, we focus on the error concealment that is processed at the decoder without using any backchannels. Though there have been various approaches to this problem, they generally focus on minimizing the degradation measured frame by frame. Although this frame-level approach is effective in evaluating individual frame quality, in the sense of human perception, the most noticeable feature is the spatio-temporal discontinuity of the image feature in the decoded video image. We propose a novel error concealment algorithm comprising the combination of i) A spatio-temporal error recovery function with low processing cost, ii) A MB-based image fidelity tracking scheme, and iii) An adaptive post-filter using the fidelity information. It is demonstrated by experimental results that the proposed algorithm can significantly reduce the subjective degradation of corrupted MPEG video quality with about 30 % of additional decoding processing power.

This paper proposes a flexible VLSI architecture design for motion estimation in H.264/AVC using a high-efficiency variable block-size decision (VBSD) approach. The proposed VBSD approach can perform a full motion search on integrating the 44 block sizes into 48, 84, 88, 816, 168, or 1616 block sizes and then appropriately select the optimal modes for motion compensation operating. In other words, the proposed architecture based on the VBSD approach can effectively reduce the encoding time of the motion estimation by dealing with different block sizes under 1616 searching range. Using the TSMC 0.18-µm CMOS technology, the proposed architecture has been successfully realized. Simulation and verification results show that the proposed architecture has significant bit-rate reduction and small PSNR degradation. Also, the physical chip design revealed that the maximum frame rate of this work can process 704 fps with QCIF (176144), 176 fps with CIF (352288) and 44 fps with 4CIF (704576) video resolutions under lower gate counts and higher working frequency.

In this letter, we propose an enhanced method for inter-layer motion prediction in scalable video coding (SVC). For inter-layer motion prediction, the use of refined motion data in the Fine Granular Scalability (FGS) layer is proposed instead of the conventional use of motion data in the base quality layer to reduce the inter-layer redundancy efficiently. Experimental results show that the proposed method enhances coding efficiency without increasing the computational complexity of the decoder.

This paper presents an adaptive transform coefficient scan method that effectively improves intra coding efficiency of H.264. Instead of applying one zig-zag scan to all transform blocks, the proposed method applies a field scan to a horizontally predicted block, a horizontal scan to a vertically predicted block, and a zig-zag scan to blocks predicted in other prediction modes. Experiments based on JM9.6 were performed using only intra coding. Results of the experiments show that the proposed method yields an average PSNR enhancement of 0.16 dB and a maximum PSNR enhancement of 0.31 dB over the current H.264 using zig-zag scan.