Video quality generally suffers from packet losses caused by an unreliable channel when video is transmitted over an error-prone wireless channel. This quality degradation is the main reason that a video compression encoder uses error-resilient coding to deal with the high packet-loss probability. The use of adequate error resilience can mitigate the effects of channel errors, but the coding efficiency for bit reduction will be decreased. On the other hand, H.264/AVC uses multiple reference frame (MRF) motion compensation for a higher coding efficiency. However, an increase in the number of reference frames in the H.264/AVC encoder has been recently observed, making the received video quality worse in the presence of transmission errors if the cyclic intra-refresh is used as the error-resilience method. This is because the reference-block selection in the MRF chooses blocks on the basis of the rate distortion optimization, irrespective of the intra-refresh coding. In this paper, a new error-resilient reference selection method is proposed to provide error resilience for MRF based motion compensation. The proposed error-resilient reference selection method achieves an average PSNR enhancement up to 0.5 to 2dB in 10% packet-loss-ratio environments. Therefore, the proposed method can be valuable in most MRF-based interactive video encoding system, which can be used for video broadcasting and mobile video conferencing over an erroneous network.

In transcoding, it is well known that refinement of the motion vectors is critical to enhance the quality of transcoded video while significantly reducing transcoding complexity. This paper proposes a novel cost model to estimate the rate-distortion cost of motion vector composition in order to develop a reliable motion vector re-estimation method that has reasonable computation cost. Based on a statistical analysis of motion compensated prediction errors, we design a basic form of the proposed cost model as a function of distance from the optimal motion vector. Simulations with a transcoder employing the proposed cost model demonstrate a significant quality gain over representative video transcoding schemes with no complexity increase.

The mode of intra prediction in H.264/AVC is encoded based on the most probable mode (MPM). To increase coding efficiency, the probability of the case that MPM is equal to coding mode of the current block should increase. In this paper we propose an efficient scheme to make MPM which is matched for the spatial direction property of pixels in the current block. Simulation results show that the proposed scheme gives significant coding gains when compared with the conventional techniques.

Digital video signals are subject to several distortions due to compression processes, transmission over noisy channels or video processing. Therefore, the video quality evaluation has become a necessity for broadcasters and content providers interested in offering a high video quality to the customers. Thus, an objective no-reference video quality assessment metric is proposed based on the sigmoid model using spatial-temporal features weighted by parameters obtained through the solution of a nonlinear least squares problem using the Levenberg-Marquardt algorithm. Experimental results show that when it is applied to MPEG-2 streams our method presents better linearity than full-reference metrics, and its performance is close to that achieved with full-reference metrics for H.264 streams.

We present a new method for generating thumbnail images from H.264/AVC coded bit streams. What distinguishes our approach from previous works is that it determines the thumbnail image pixels by summing the residual and estimate block averages. The residual block averages are directly acquired in the transform domain and the estimated block averages are calculated in the spatial domain. Due to the construction of the reference pixels in the spatial domain, the proposed method eliminates the source of mismatch error, thus the result does not suffer any degradation. The thumbnail images produced by the proposed method are indistinguishable to the ones by the method that decodes the H.264/AVC intra coded bit streams and then scales them down. For most images, the proposed method also executes almost 3 times faster than the down-scaling method at frequently used bandwidths.

Rate control that is required to regulate the bitrate of video coding is critical to time-sensitive video applications used over networks. However, the H.264/AVC standard does not respond to scene changes, and this causes the transmission quality to deteriorate as a scene change occurs. In this work, a scene change is detected by comparing the ratio of the sum of absolute difference (SAD) between two consecutive frames. As the scene change is detected, the proposed method, which is modified from the reference software of H.264/AVC, re-assigns a quantization parameter (QP) value to regulate the bitrate. Because the inter-prediction works poorly for the scene-changed frame, the proposed method estimates its frame complexity based on the content, and further creates another Q-R model to assign QP. The adaptive rate control mechanism presented in this study can quickly respond to the heavy bitrate increment caused by a change of scene. Simulation results show that the proposed method improves the average peak signal noise ratio (PSNR) to approximately 1.1dB, with a smaller buffer size compared with the performance of the reference software JM version 17.2.

H.264/AVC estimates the prediction mode from the modes of neighboring blocks in order to reduce the quantity of bits needed to represent the best mode. The estimated mode is known as Most Probable Mode (MPM). When QP changes, the probability of MPM increases. In this paper, we propose an adaptive mode signaling method based on the probability of MPM. According to the probability of MPM, a conventional MPM method and the proposed merging method are adaptively used to signal the intra prediction mode. Experimental results show that the proposed method achieves an average bit savings of 0.727% and an average PSNR gain of 0.041dB. In the low bit rate condition, the proposed method achieves an average bit savings of 1.795%, and a PSNR gain of 0.103dB, compared to H.264/AVC.

The emerging high-efficiency video coding (HEVC) standard attempts to improve the coding efficiency by a factor of two over H.264/AVC through the use of new compression tools such as various block sizes with multiple directions. Although multiple-directional predictions are among the features contributing to the improved compression efficiency, its high computational complexity keeps it from being used widely. This paper presents an algorithm to skip backward and bi-directional predictions when merge or forward prediction modes are likely to be determined as the best mode. The proposed algorithm takes advantage of the fact that there is a cost relationship among multi-directional predictions and that the results of backward and bi-directional predictions are therefore predictable before the actual operations. After merge and forward predictions, if the expected results of backward and bi-directional predictions are worse than the results up to that point, then additional backward and bi-directional predictions to search for more accurate motion vectors are not performed. A simulation shows that the encoding time is reduced by about 15.18% with a marginal degradation in compression efficiency.

In this paper, we propose a novel joint rate distortion optimization (JRDO) model for intra prediction coding. The spatial prediction dependency is exploited by modeling the distortion propagation with a linear fitting function. A novel JRDO based Lagrange multiplier (LM) is derived from this model. To adapt to different blocks' distortion propagation characteristics, we also introduce a generalized multiple Lagrange multiplier (MLM) framework where some candidate LMs are used in the RDO process. Experiment results show that our proposed JRDO-MLM scheme is superior to the H.264/AVC encoder.

New media processing applications such as image recognition and AR (Augment Reality) have become into practical on embedded systems for automotive, digital-consumer and mobile products. Many-core processors have been proposed to realize much higher performance than multi-core processors. We have developed a low-power many-core SoC for multimedia applications in 40nm CMOS technology. Within a 210mm2 die, two 32-core clusters are integrated with dynamically reconfigurable processors, hardware accelerators, 2-channel DDR3 I/Fs, and other peripherals. Processor cores in the cluster share a 2MB L2 cache connected through a tree-based Network-on-Chip (NoC). Its total peak performance exceeds 1.5TOPS (Tera Operations Per Second). The high scalability and low power consumption are accomplished by parallelized software for multimedia applications. In case of face detection, the performance scales up to 64 cores and the SoC consumes only 2.21W. Moreover, it can execute the 1080p 48fps H.264 decoding about 520mW by 28 cores and the 4K2K 15fps super resolution about 770mW by 32 cores in one cluster. Exploiting parallelism by low power processor cores, the many-core SoC provides several tens of times better energy efficiency than that of a high performance desk-top quad-core processor.

As a next-generation video compression standard, High Efficiency Video Coding (HEVC) achieves enhanced coding performance relative to prior standards such as H.264/AVC. In the new standard, the improved intra prediction plays an important role in bit rate saving. Meanwhile, it also involves significantly increased complexity, due to the adoption of a highly flexible coding unit structure and a large number of angular prediction modes. In this paper, we present a low-complexity intra prediction algorithm for HEVC. We first propose a fast preprocessing stage based on a simplified cost model. Based on its results, a fast prediction unit selection scheme reduces the number of prediction unit (PU) levels that requires fine processing from 5 to 2. To supply PU size decision with appropriate thresholds, a fast training method is also designed. Still based on the preprocessing results, an efficient mode selection scheme reduces the maximum number of angular modes to evaluate from 35 to 8. This achieves further algorithm acceleration by eliminating the necessity to perform fine Hadamard cost calculation. We also propose a 32×32 PU compensation scheme to alleviate the mismatch of cost functions for large transform units, which effectively improves coding performance for high-resolution sequences. In comparison with HM 7.0, the proposed algorithm achieves over 50% complexity reduction in terms of encoding time, with the corresponding bit rate increase lower than 2.0%. Moreover, the achieved complexity reduction is relatively stable and independent to sequence characteristics.

Many subjective assessment methods for video quality are provided by ITU-T and ITU-R recommendations, but the differences among these methods have not been sufficiently studied. We compare five subjective assessment methods using four quantitative performance indices for both HD and QVGA resolution video. We compare the Double-Stimulus Continuous Quality-Scale (DSCQS), Double-Stimulus Impairment Scale (DSIS), Absolute Category Rating method (ACR), and ACR with Hidden Reference (ACR-HR) as common subjective assessment methods for HD and QVGA resolution videos. Furthermore, we added ACR with an 11-grade scale (ACR11) for the HD test and Subjective Assessment of Multimedia Video Quality (SAMVIQ) for the QVGA test for quality scale variations. The performance indices are correlation coefficients, rank correlation coefficients, statistical reliability, and assessment time. For statistical reliability, we propose a performance index for comparing different quality scale tests. The results of the performance comparison showed that the correlation coefficients and rank correlation coefficients of the mean opinion scores between pairs of methods were high for both HD and QVGA tests. As for statistical reliability provided by the proposed index, DSIS of HD and ACR of QVGA outperformed the other methods. Moreover, ACR, ACR-HR, and ACR11 were the most efficient subjective quality assessment methods from the viewpoint of assessment time.

Video coding plays an important role in human life especially in communications. H.264/AVC is a prominent video coding standard that has been used in a variety of applications due to its high efficiency comes from several new coding techniques. However, the extremely high encoding complexity hinders itself from real-time applications. This paper presents a new encoding algorithm that makes use of particle swarm optimization (PSO) to train discriminant functions for classification based fast mode decision. Experimental results show that the proposed algorithm can successfully reduce encoding time at the expense of negligible quality degradation and bitrate increases.

One of the largest challenges for coarse-grained reconfigurable arrays (CGRAs) is how to efficiently map applications. The key issues for mapping are (1) how to reduce the memory bandwidth, (2) how to exploit parallelism in algorithms and (3) how to achieve load balancing and take full advantage of the hardware potential. In this paper, we propose a novel parallelism scheme, called ‘Hybrid partitioning’, for mapping a H.264 high definition (HD) decoder onto REMUS-II, a CGRA system-on-chip (SoC). Combining good features of data partitioning and task partitioning, our methodology mainly consists of three levels from top to bottom: (1) hybrid task pipeline based on slice and macroblock (MB) level; (2) MB row-level data parallelism; (3) sub-MB level parallelism method. Further, on the sub-MB level, we propose a few mapping strategies such as hybrid variable block size motion compensation (Hybrid VBSMC) for MC, 2D-wave for intra 44, parallel processing order for deblocking. With our mapping strategies, we improved the algorithm's performance on REMUS-II. For example, with a luma 1616 MB, the Hybrid VBSMC achieves 4 times greater performance than VBSMC and 2.2 times greater performance than fixed 44 partition approach. Finally, we achieve 1080p@33fps H.264 high-profile (HiP)@level 4.1 decoding when the working frequency of REMUS-II is 200 MHz. Compared with typical hardware platforms, we can achieve better performance, area, and flexibility. For example, our performance achieves approximately 175% improvement than that of a commercial CGRA processor XPP-III while only using 70% of its area.

In this paper, we propose a novel scheme for efficient video broadcasting over WLANs using the IEEE 802.11e HCCA MAC and H.264/SVC. We rearrange the outgoing sequence of H.264/SVC NAL units according to their dimension, temporal, and quality scalability. In addition, our proposed scheme broadcasts the NAL units at various data-rates by using the link adaptation function of IEEE 802.11 PHY. Our scheme is verified using NCTUns network simulator, and is evaluated in terms of throughput, delay, and quality of experience (QoE) using structural similarity (SSIM) rather than mean square error (MSE). We employee a real video clip to increase the reliability of the simulation in which the video clip is compressed as VBR with 24 scalable layers by JSVM reference codec of the H.264/SVC. In the simulation topology, a host broadcasts the video clip to 10 wireless stations which are within 150 meters from an AP. We present performance comparisons between our proposed scheme and the scheme provided by the IEEE 802.11e HCCA standard, which is to be referred to as the simple scheme in this paper. The proposed scheme noticeably enhances in the three performance metrics. All wireless stations by the proposed scheme receive more video data than the simple scheme around 2530% within a delay bound of 1 second. The proposed scheme controls the end-to-end delay to 510% under that of the simple scheme. As for the throughput and the delay performance, the proposed scheme enhances the video quality by up to 67% compared to the simple scheme in SSIM evaluation.

H.264 video codec system requires big capacity and high bandwidth of Frame Store (FS) for buffering reference frames. The up-to-date three dimensional (3D) stacked Phase change Random Access Memory (PRAM) is the promising approach for on-chip caching the reference signals, as 3D stacking offers high memory bandwidth, while PRAM possesses the advantages in terms of high density and low leakage power. However, the write endurance problem, that is a PRAM cell can only tolerant limited number of write operations, becomes the main barrier in practical applications. This paper studies the wear reduction techniques of PRAM based FS in H.264 codec system. On the basis of rate-distortion theory, the content oriented selective writing mechanisms are proposed to reduce bit updates in the reference frame buffers. With the proposed control parameter a, our methods make the quantitative trade off between the quality degradation and the PRAM lifetime prolongation. Specifically, taking a in the range of [0.2,2], experimental results demonstrate that, our methods averagely save 29.9–35.5% bit-wise write operations and reduce 52–57% power, at the cost of 12.95–20.57% BDBR bit-rate increase accordingly.

As the successive video compression standard of H.264/AVC, High Efficiency Video Codec (HEVC) will play an important role in video coding area. In the deblocking filter part, HEVC inherits the basic property of H.264/AVC and gives some new features. Based on this variation, this paper introduces a novel dual-mode deblocking filter architecture which could support both of the HEVC and H.264/AVC standards. For HEVC standard, the proposed symmetric unified-cross unit (SUCU) based filtering scheme greatly reduces the design complexity. As a result, processing a 1616 block needs 24 clock cycles. For H.264/AVC standard, it takes 48 clock cycles for a 1616 macro-block (MB). In synthesis result, the proposed architecture occupies 41.6k equivalent gate count at frequency of 200 MHz in SMIC 65 nm library, which could satisfy the throughput requirement of super hi-vision (SHV) on 60 fps. With filter reusing scheme, the universal design for the two standards saves 30% gate counts than the dedicated ones in filter part. In addition, the total power consumption could be reduced by 57.2% with skipping mode when the edges need not be filtered.

A motion estimation (ME) multimedia processor was developed by employing dynamic voltage and frequency scaling (DVFS) technique to greatly reduce the power dissipation. To make full use of the advantages of DVFS technique, a fast motion estimation (ME) algorithm was also developed. It can adaptively predict the optimum supply voltage and the optimum clock frequency before ME process starts for each macro-block for encoding. Power dissipation of the 90-nm CMOS DVFS controlled multimedia processor, which contained an absolute difference accumulator as well as a small on-chip DC/DC level converter, a minimum value detector and DVFS controller, was reduced to 38.48 µW, which was only 3.261% that of a conventional multimedia processor.

The emerging High Efficiency Video Coding (HEVC) standard attempts to improve the coding efficiency by a factor of two over H.264/AVC through the use of new compression tools with high computational complexity. Although multipledirectional prediction is one of the features contributing to the improved compression efficiency, the computational complexity for prediction increases significantly. This paper presents an early uni-directional prediction decision algorithm. The proposed algorithm takes advantage of the property of HEVC that it supports a deep quad-tree block structure. Statistical observation shows that the correlation of prediction direction among different blocks which share same area is very high. Based on this observation, the mode of the current block is determined early according to the mode of upper blocks. Bi-directional prediction is not performed when the upper block is encoded as the uni-directional prediction mode. A simulation shows that it reduces ME operation time by about 22.7% with a marginal drop in compression efficiency.

In this letter, we propose a novel intra prediction coding scheme for H.264/AVC. Based on our proposed minimum distance prediction (MDP) scheme, the optimal reference samples for predicting the current pixel can be adaptively updated corresponding to different video contents. The experimental results show that up to 2 dB and 1 dB coding gains can be achieved with the proposed method for QCIF and CIF sequences respectively.