Geometric partitioning mode (GPM) is a new inter prediction tool adopted in versatile video coding (VVC), which is the latest video coding of international standard developed by joint video expert team in 2020. Different from the regular inter prediction performed on rectangular blocks, GPM separates a coding block into two regions by the pre-defined 64 types of straight lines, generates inter predicted samples for each separated region, and then blends them to obtain the final inter predicted samples. With this feature, GPM improves the prediction accuracy at the boundary between the foreground and background with different motions. However, GPM has room to further improve the prediction accuracy if the final predicted samples can be generated using not only inter prediction but also intra prediction. In this paper, we propose a GPM with inter and intra prediction to achieve further enhanced compression capability beyond VVC. To maximize the coding performance of the proposed method, we also propose the restriction of the applicable intra prediction mode number and the prohibition of applying the intra prediction to both GPM-separated regions. The experimental results show that the proposed method improves the coding performance gain by the conventional GPM method of VVC by 1.3 times, and provides an additional coding performance gain of 1% bitrate savings in one of the coding structures for low-latency video transmission where the conventional GPM method cannot be utilized.

Cross-component linear model (CCLM) has been recently adopted as a chroma intra-prediction tool in Versatile Video Coding (VVC), which is being developed as a new video coding standard. CCLM predicts chroma components from luma components through a linear model based on assumption of linear correlation between both components. A linear model is derived from the reconstructed neighboring luma and chroma samples of the current coding block by linear regression. A simplified linear modeling method recently adopted in the test model of VVC (VTM) 3.0 significantly reduces computational complexity of deriving model parameters with considerable coding loss. This letter proposes a method of linear modeling to compensate the coding loss of the simplified linear model. In the proposed method, the model parameters which are quite roughly derived in the existing simplified linear model are refined more accurately using individual method to derive each parameter. Experimental results show that, compared to VTM 3.0, the proposed method gives 0.08%, 0.52% and 0.55% Bjotegaard-Delta (BD)-rate savings, for Y, Cb and Cr components, respectively, in the All-Intra (AI) configuration with negligible computational complexity increase.

It is essential to improve intra prediction performance to raise the efficiency of video coding. In video coding standards such as H.265/HEVC, intra prediction is seen as an extension of directional prediction schemes, examples include refinement of directions, planar extension, filtering reference sampling, and so on. From the view point of reducing prediction error, some improvements on intra prediction for standardized schemes have been suggested. However, on the assumption that the correlation between neighboring pixels are static, these conventional methods use pre-defined predictors regardless of the image being encoded. Therefore, these conventional methods cannot reduce prediction error if the images break the assumption made in prediction design. On the other hand, adaptive predictors that change the image being encoded may offer poor coding efficiency due to the overhead of the additional information needed for adaptivity. This paper proposes an adaptive intra prediction scheme that resolves the trade-off between prediction error and adaptivity overhead. The proposed scheme is formulated as a constrained optimization problem that minimizes prediction error under sparsity constraints on the prediction coefficients. In order to solve this problem, a novel solver is introduced as an extension of LARS for multi-class support. Experiments show that the proposed scheme can reduce the amount of encoded bits by 1.21% to 3.24% on average compared to HM16.7.

This paper proposes a lossy image coding method for still images. In this method, recursive and non-recursive type intra prediction techniques are adaptively selected on a block-by-block basis. The recursive-type intra prediction technique applies a linear predictor to each pel within a prediction block in a recursive manner, and thus typically produces smooth image values. In this paper, the non-recursive type intra prediction technique is extended from the angular prediction technique adopted in the H.265/HEVC video coding standard to enable interpolative prediction to the maximum possible extent. The experimental results indicate that the proposed method achieves better coding performance than the conventional method that only uses the recursive-type prediction technique.

Compressed Sensing based CMOS image sensor (CS-CIS) is a new generation of CMOS image sensor that significantly reduces the power consumption. For CS-CIS, the image quality and data volume of output are two important issues to concern. In this paper, we first proposed an algorithm to generate a series of deterministic and ternary matrices, which improves the image quality, reduces the data volume and are compatible with CS-CIS. Proposed matrices are derived from the approximate DCT and trimmed in 2D-zigzag order, thus preserving the energy compaction property as DCT does. Moreover, we proposed matrix row operations adaptive to the proposed matrix to further compress data (measurements) without any image quality loss. At last, a low-cost VLSI architecture of measurements compression with proposed matrix row operations is implemented. Experiment results show our proposed matrix significantly improve the coding efficiency by BD-PSNR increase of 4.2 dB, comparing with the random binary matrix used in the-state-of-art CS-CIS. The proposed matrix row operations for measurement compression further increases the coding efficiency by 0.24 dB BD-PSNR (4.8% BD-rate reduction). The VLSI architecture is only 4.3 K gates in area and 0.3 mW in power consumption.

Scalable high efficiency video coding (SHVC) can provide variable video quality according to terminal devices. However, the computational complexity of SHVC is increased by introducing new techniques based on high efficiency video coding (HEVC). In this paper, a hardware oriented low complexity algorithm is proposed. The hardware oriented proposals have two key points. Firstly, the coding unit depth is determined by analyzing the boundary correlation between coding units before encoding process starts. Secondly, the redundant calculation of R-D optimization is reduced by adaptively using the information of the neighboring coding units and the co-located units in the base layer. The simulation results show that the proposed algorithm can achieve over 62% computation complexity reduction compared to the original SHM11.0. Compared with other related work, over 11% time saving have been achieved without PSNR loss. Furthermore, the proposed algorithm is hardware friendly which can be implemented in a small area.

This paper presents a measurement-domain intra prediction coding framework that is compatible with compressive sensing (CS)-based image sensors. In this framework, we propose a low-complexity intra prediction algorithm that can be directly applied to measurements captured by the image sensor. We proposed a structural random 0/1 measurement matrix, embedding the block boundary information that can be extracted from the measurements for intra prediction. Furthermore, a low-cost Very Large Scale Integration (VLSI) architecture is implemented for the proposed framework, by substituting the matrix multiplication with shared adders and shifters. The experimental results show that our proposed framework can compress the measurements and increase coding efficiency, with 34.9% BD-rate reduction compared to the direct output of CS-based sensors. The VLSI architecture of the proposed framework is 9.1 Kin area, and achieves the 83% reduction in size of memory bandwidth and storage for the line buffer. This could significantly reduce both the energy consumption and bandwidth in communication of wireless camera systems, which are expected to be massively deployed in the Internet of Things (IoT) era.

High Efficiency Video Coding is a new video coding standard after H.264/AVC. By introducing a flexible coding unit, which can be recursively divided from 64×64 to 8×8 blocks in a Quadtree-Structure, HEVC achieves significantly higher coding efficiency than the previous standards. With the flexible CU structure, HEVC can effectively adapt to highly varying contents with a smaller CU or to flat contents with a larger CU, making it suitable for applications from mobile video to super high definition television. On the other hand, CU division does incur high computational cost for HEVC. In this paper, we propose a simple and fast CU division algorithm by using only a subset of pixels to determine when CU division happens. Experiment results show that our algorithm can achieve prediction quality close to HEVC Test Model with much lower computational cost.

High Efficiency Video Coding (HEVC) is the latest coding standard. Compared with Advanced Video coding (H.264/AVC), HEVC offers about a 50% bitrate reduction at the same reconstructed video quality. However, this new coding standard leads to enormous computational complexity, which makes it difficult to encode video in real time. Therefore, in this paper, aiming at the high complexity of intra coding in HEVC, a new fast coding unit (CU) splitting algorithm is proposed based on the decision tree. Decision tree, as a method of machine learning, can be designed to determine the size of CUs adaptively. Here, two significant features, Just Noticeable Difference (JND) values and coding bits of each CU can be extracted to train the decision tree, according to their relationships with the CUs' partitions. The experimental results have revealed that the proposed algorithm can save about 34% of time, on average, with only a small increase of BD-rate under the “All_Intra” setting, compared with the HEVC reference software.

This paper proposes a fast mode decision technique for intra prediction of High Efficiency Video Coding (HEVC) based on a reliability metric for motion vectors (RMMV). Since such a decision problem can be regarded as a kind of pattern classification, an efficient classifier is required for the reduction of computation complexity. This paper employs the RMMV as a classifier because the RMMV can efficiently categorize image blocks into flat(uniform), active, and edge blocks, and can estimate the direction of an edge block as well. A local search for angular modes is introduced to further speed up the decision process. An experiment shows the advantage of our technique over other techniques.

8K Ultra High Definition Television (UHDTV) requires extremely high throughput for video decoding based on H.265. In H.265, intra coding could significantly enhance video compression efficiency, at the expense of an increased computational complexity compared with H.264. For intra prediction of 8K UHDTV real-time H.265 decoding, the joint complexity and throughput issue is more difficult to solve. Therefore, based on the divide-and-conquer strategy, we propose a new VLSI architecture in this paper, including two techniques, in order to achieve 8K UHDTV H.265 intra prediction decoding. The first technique is the LUT based Reference Sample Fetching Scheme (LUT-RSFS), reducing the number of reference samples in the worst case from 99 to 13. It further reduces the circuit area and enhances the performance. The second one is the Hybrid Block Reordering and Data Forwarding (HBRDF), minimizing the idle time and eliminating the dependency between TUs by creating 3 Data Forwarding paths. It achieves the hardware utilization of 94%. Our design is synthesized using Synopsys Design Compiler in 40nm process technology. It achieves an operation frequency of 260MHz, with a gate count of 217.8K for 8-bit design, and 251.1K for 10-bit design. The proposed VLSI architecture can support 4320p@120fps H.265 intra decoding (8-bit or 10-bit), with all 35 intra prediction modes and prediction unit sizes ranging from 4×4 to 64×64.

The high efficiency video coding (HEVC) standard has significantly improved compression performance for many applications, including remote desktop and desktop sharing. Screen content video coding is widely used in applications with a high demand for real-time performance. HEVC usually introduces great computational complexity, which makes fast algorithms necessary to offset the limited computing power of HEVC encoders. In this study, a statistical analysis of several screen content sequences is first performed to better account for the completely different statistics of natural images and videos. Second, a fast coding unit (CU) splitting method is proposed, which aims to reduce HEVC intra coding computational complexity, especially in screen content coding. In the proposed scheme, CU size decision is made by checking the smoothness of the luminance values in every coding tree unit. Experiments demonstrate that in HEVC range extension standard, the proposed scheme can save an average of 29% computational complexity with 0.9% Bjøntegaard Delta rate (BD-rate) increase compared with HM13.0+RExt6.0 anchor for screen content sequences. For default HEVC, the proposed scheme can reduce encoding time by an average of 38% with negligible loss of coding efficiency.

We present a new method for generating thumbnail images from H.264/AVC coded bit streams. What distinguishes our approach from previous works is that it determines the thumbnail image pixels by summing the residual and estimate block averages. The residual block averages are directly acquired in the transform domain and the estimated block averages are calculated in the spatial domain. Due to the construction of the reference pixels in the spatial domain, the proposed method eliminates the source of mismatch error, thus the result does not suffer any degradation. The thumbnail images produced by the proposed method are indistinguishable to the ones by the method that decodes the H.264/AVC intra coded bit streams and then scales them down. For most images, the proposed method also executes almost 3 times faster than the down-scaling method at frequently used bandwidths.

High efficiency video coding (HEVC) is a video compression standard that outperforms the predecessor H.264/AVC by doubling the compression efficiency. To enhance the intra prediction accuracy, 35 intra prediction modes were used in the prediction units (PUs), with partition sizes ranging from 4 × 4 to 64 × 64 in HEVC. However, the manifold prediction modes dramatically increase the encoding complexity. This paper proposes a fast mode- and depth-decision algorithm based on edge detection and reconfiguration to alleviate the large computational complexity in intra prediction with trivial degradation in accuracy. For mode decision, we propose pixel gradient statistics (PGS) and mode refinement (MR). PGS uses pixel gradient information to assist in selecting the prediction mode after rough mode decision (RMD). MR uses the neighboring mode information to select the best PU mode (BPM). For depth decision, we propose a partition reconfiguration algorithm to replace the original partitioning order with a more reasonable structure, by using the smoothness of the coding unit as a criterion in deciding the prediction depth. Smoothness detection is based on the PGS result. Experiment results show that the proposed method saves about 41.50% of the original processing time with little degradation (BD bitrate increased by 0.66% and BDPSNR decreased by 0.060dB) in the coding gain.

H.264/AVC estimates the prediction mode from the modes of neighboring blocks in order to reduce the quantity of bits needed to represent the best mode. The estimated mode is known as Most Probable Mode (MPM). When QP changes, the probability of MPM increases. In this paper, we propose an adaptive mode signaling method based on the probability of MPM. According to the probability of MPM, a conventional MPM method and the proposed merging method are adaptively used to signal the intra prediction mode. Experimental results show that the proposed method achieves an average bit savings of 0.727% and an average PSNR gain of 0.041dB. In the low bit rate condition, the proposed method achieves an average bit savings of 1.795%, and a PSNR gain of 0.103dB, compared to H.264/AVC.

Screen content generally consists of text, images, and videos variously generated or captured by computers and other electronic devices. For the purpose of coding such screen content, we introduce alternative intra prediction (AIP) modes based on the emerging high efficiency video coding (HEVC) standard. With text and graphics, edges are much sharper and a large number of corners exist. These properties make it difficult to predict blocks using a one-directional intra prediction mode. The proposed method provides two-directional prediction by combining the existing vertical and horizontal prediction modes. Experiments show that our AIP modes provide an average BD-rate reduction of 2.8% relative to HEVC for general screen contents, and a 0.04% reduction for natural contents.

The development of 3D High Efficiency Video Coding (3D-HEVC) has resulted in a growing interest in the compression of depth-maps. To achieve better intra prediction performance, the Depth Modeling Mode (DMM) technique is employed as an intra prediction technique for depth-maps. However, the complexity and computation load have dramatically increased with the application of DMM. Therefore, in view of the limited colors in depth-maps, this paper presents a novel fast intra coding scheme based on Base Colors and Index Map (BCIM) to reduce the complexity of DMM effectively. Furthermore, the index map is remapped, and the Base Colors are coded by predictive coding in BCIM to improve compression efficiency. Compared with the intra prediction coding in DMM, the experimental results illustrate that the proposed scheme provides a decrease of approximately 51.2% in the intra prediction time. Meanwhile, the BD-rate increase is only 0.83% for the virtual intermediate views generated by Depth-Image-Based Rendering.

This letter proposes a pipelined architecture with prediction mode scheduling for high efficiency video coding (HEVC). An increased number of intra prediction modes in HEVC have introduced a new technique, named rough mode decision (RMD). This development, however, means that pipeline architectures for H.264 cannot be used in HEVC. The proposed scheme executes the RMD and the rate-distortion optimization (RDO) process simultaneously by grouping the intra prediction modes and changing the candidate selection method of the RMD algorithm. The proposed scheme reduces execution cycle by up to 26% with negligible coding loss.

High Efficiency Video Coding (HEVC) is the latest video coding standard that is supported by JCT-VC. In this letter, an encoding algorithm for early termination of Coding Unit (CU) and Prediction Unit (PU) based on the texture direction is proposed for the HEVC intra prediction. Experimental results show that the proposed algorithm provides an average 40% total encoding time reduction with the negligible loss of rate-distortion.

As a next-generation video compression standard, High Efficiency Video Coding (HEVC) achieves enhanced coding performance relative to prior standards such as H.264/AVC. In the new standard, the improved intra prediction plays an important role in bit rate saving. Meanwhile, it also involves significantly increased complexity, due to the adoption of a highly flexible coding unit structure and a large number of angular prediction modes. In this paper, we present a low-complexity intra prediction algorithm for HEVC. We first propose a fast preprocessing stage based on a simplified cost model. Based on its results, a fast prediction unit selection scheme reduces the number of prediction unit (PU) levels that requires fine processing from 5 to 2. To supply PU size decision with appropriate thresholds, a fast training method is also designed. Still based on the preprocessing results, an efficient mode selection scheme reduces the maximum number of angular modes to evaluate from 35 to 8. This achieves further algorithm acceleration by eliminating the necessity to perform fine Hadamard cost calculation. We also propose a 32×32 PU compensation scheme to alleviate the mismatch of cost functions for large transform units, which effectively improves coding performance for high-resolution sequences. In comparison with HM 7.0, the proposed algorithm achieves over 50% complexity reduction in terms of encoding time, with the corresponding bit rate increase lower than 2.0%. Moreover, the achieved complexity reduction is relatively stable and independent to sequence characteristics.