In transcoding, it is well known that refinement of the motion vectors is critical to enhance the quality of transcoded video while significantly reducing transcoding complexity. This paper proposes a novel cost model to estimate the rate-distortion cost of motion vector composition in order to develop a reliable motion vector re-estimation method that has reasonable computation cost. Based on a statistical analysis of motion compensated prediction errors, we design a basic form of the proposed cost model as a function of distance from the optimal motion vector. Simulations with a transcoder employing the proposed cost model demonstrate a significant quality gain over representative video transcoding schemes with no complexity increase.

In this paper, VLSI architecture design of unified motion vector (MV) and boundary strength (BS) parameter decoder (PDec) for 8K UHDTV HEVC decoder is presented. The adoption of new coding tools in PDec, such as Advanced Motion Vector Prediction (AMVP), increases the VLSI hardware realization overhead and memory bandwidth requirement, especially for 8K UHDTV application. We propose four techniques for these challenges. Firstly, this work unifies MV and BS parameter decoders for line buffer memory sharing. Secondly, to support high throughput, we propose the top-level CU-adaptive pipeline scheme by trading off between implementation complexity and performance. Thirdly, PDec process engine with optimizations is adopted for 43.2k area reduction. Finally, PU-based coding scheme is proposed for 30% DRAM bandwidth reduction. In 90nm process, our design costs 93.3k logic gates with 23.0kB line buffer. The proposed architecture can support real-time decoding for 7680x4320@60fps application at 249MHz in the worst case.

A new technology for video frame rate up-conversion (FRUC) is presented by combining a median filter and motion estimation (ME) with an occlusion detection (OD) method. First, ME is performed to obtain a motion vector. Then, the OD method is used to refine the MV in the occlusion region. When occlusion occurs, median filtering is applied. Otherwise, bidirectional motion compensated interpolation (BDMC) is applied to create the interpolated frames. The experimental results show that the proposed algorithm provides better performance than the conventional approach. The average gain in the PSNR (Peak Signal to Noise Ratio) is always better than the other methods in the Full HD test sequences.

In this paper, an improved B-picture coding algorithm based on the symmetric bi-directional motion estimation (ME) is proposed. In addition to the block match error between blocks in the forward and backward reference frames, the proposed method exploits the previously-reconstructed template regions in the current and reference frames for bi-directional ME. The side match error between the predicted target block and its template is also employed in order to alleviate block discontinuities. To efficiently perform ME, an initial motion vector (MV) is adaptively derived by exploiting temporal correlations. Experimental results show that the number of generated bits is reduced by up to 9.31% when the proposed algorithm is employed as a new macroblock (MB) coding mode for the H.264/AVC standard.

A novel approach for detecting anomaly in visual surveillance system is proposed in this paper. It is composed of three parts: (a) a dense motion field and motion statistics method, (b) motion directional PCA for feature dimensionality reduction, (c) an improved one-class SVM for one-class classification. Experiments demonstrate the effectiveness of the proposed algorithm in detecting abnormal events in surveillance video, while keeping a low false alarm rate. Our scheme works well in complicated situations that common tracking or detection modules cannot handle.

In this paper, we propose a new motion vector smoothing algorithm using weighted vector median filtering based on edge direction for frame interpolation. The proposed WVM (Weighted Vector Median) system adjusts the weighting values based on edge direction, which is derived from spatial coherence between the edge direction continuity of a moving object and motion vector (MV) reliability. The edge based weighting scheme removes the effect of outliers and irregular MVs from the MV smoothing process. Simulation results show that the proposed algorithm can correct wrong motion vectors and thus improve both the subjective and objective visual quality compared with conventional methods.

In this letter, we first study the impact of the basic reference frame jitter on the digital image stabilization. Next, a method for stabilizing the digital image sequence based on the correction for basic reference frame jitter is proposed. The experimental results show that our proposed method can effectively decrease the excessive undefined areas in the stable image sequence resulting from the basic reference frame jitter.

We proposed a new motion vector (MV) smoothing using fuzzy weighting and vector median filtering for frame rate up-conversion. A fuzzy reasoning system adjusts the weighting values based on the local characteristics of MV field including block difference and block boundary distortion. The fuzzy weighting removes the affect of outliers and irregular MVs from the MV smoothing process. The simulation results show that the proposed algorithm can efficiently correct wrong MVs and thus improve visual quality of the interpolated frames better than conventional methods.

In this paper, VLSI architecture of a joint parameter decoder is proposed to realize the calculation of motion vector (MV), intra prediction mode (IPM) and boundary strength (BS) for ultra high definition H.264/AVC applications. For this architecture, a 64-cycle-per-MB pipeline with simplified control modes is designed to increase system throughput and reduce hardware cost. Moreover, in order to save memory bandwidth, the data which includes the motion information for the co-located picture and the last decoded line, is pre-processed before being stored to DRAM. A partition based storage format is applied to condense the MB level data, while variable length coding based compression method is utilized to reduce the data size in each partition. Experimental results show our design is capable of real-time 38402160@60 fps decoding at less than 133 MHz, with 37.2 k logic gates. Meanwhile, by applying the proposed scheme, 85-98% bandwidth saving is achieved, compared with storing the original information for every 44 block to DRAM.

In the latest video coding frameworks, efficiency of motion vector (MV) coding is becoming increasingly important because of the growing bit rate portion of motion information. However, neither the conventional median predictor, nor the newer schemes such as the minimum bit rate prediction scheme and the hybrid scheme, can effectively eliminate the local redundancy of motion vectors. In this paper, we present the prioritized reference decision scheme for efficient motion vector coding, based on the H.264/AVC framework. This scheme makes use of a boolean indicator to specify whether the median predictor is to be used for the current MV or not. If not, the median prediction is considered not suitable for the current MV, and this information is used for refining the possible space of a group of reference MVs including 4 neighboring MVs and the zero MV. This group of MVs is organized to be a prioritized list so that the reference MV with highest priority is to be selected as the prediction value. Furthermore, the boolean indicators are coded into the modified code words of mb_type and sub_mb_type, so as to reduce the overhead. By applying the proposed scheme, the structure and the applicability problems with the state-of-the-art MBP scheme have been overcome. Experimental result shows that the proposed scheme achieves a considerable reduction of bits for MVDs, compared with the conventional median prediction algorithm. It also achieves a better and much stabler performance than MBP-based MV coding.

Current sub-pixel motion estimation algorithm is time and memory-consuming when performing image compression and communication. So we propose a selective interpolation method for sub-pixel motion estimation. We applied selective interpolations after estimating a candidate for sub-pixel accuracy motion vector from the simplest mathematical model. According to simulation results, the proposed method attains nearly the same performance as the full-search for half-pixel motion estimation with much lower computational complexity.

With the ubiquitous application of Internet and wireless networks, H.264 video communication becomes more and more common. However, due to the high-efficiently predictive coding and the variable length entropy coding, it is more sensitive to transmission errors. The current error concealment (EC) scheme, which utilizes the spatial and temporal correlations to conceal the corrupted region, produces unsatisfied boundary artifacts. In this paper, first we propose variable block size error concealment (VBSEC) scheme inspired by variable block size motion estimation (VBSME) in H.264. This scheme provides four EC modes and four sub-block partitions. The whole corrupted macro-block (MB) will be divided into variable block size adaptively according to the actual motion. More precise motion vectors (MV) will be predicted for each sub-block. Then MV refinement (MVR) scheme is proposed to refine the MV of the heterogeneous sub-block by utilizing three step search (TSS) algorithm adaptively. Both VBSEC and MVR are based on our directional spatio-temporal boundary matching algorithm (DSTBMA). By utilizing these schemes, we can reconstruct the corrupted MB in the inter frame more accurately. The experimental results show that our proposed scheme can obtain better objective and subjective EC quality, respectively compared with the boundary matching algorithm (BMA) adopted in the JM11.0 reference software, spatio-temporal boundary matching algorithm (STBMA) and other comparable EC methods.

Motion-compensated frame interpolation (MCFI) is widely used to smoothly display low frame rate video sequences by synthesizing and inserting new frames between existing frames. The temporal shift interpolation technique (TSIT) is popular for frame interpolation of video sequences that are encoded by a block-based video coding standard such as MPEG-4 or H.264/AVC. TSIT assumes the existence of a motion vector (MV) and may not result in high-quality interpolation for intra-mode blocks that do not have MVs. This paper proposes a new frame interpolation algorithm mainly designed for intra-mode blocks. In order to improve the accuracy of pixel interpolation, the new algorithm proposes sub-pixel interpolation and the reuse of MVs for their refinement. In addition, the new algorithm employs two different interpolation modes for inter-mode blocks and intra-mode blocks, respectively. The use of the two modes reduces ghost artifacts but potentially increases blocking effects between the blocks interpolated by different modes. To reduce blocking effects, the proposed algorithm searches the boundary of an object and interpolates all blocks in the object in the same mode. Simulation results show that the proposed algorithm improves PSNR by an average of 0.71 dB compared with the TSIT with MV refinement and also significantly improves the subjective quality of pictures by reducing ghost artifacts.

The quality of the Side-information frame (S frame) influences significantly the rate-distortion performance in the Distributed Video Coding (DVC). In this letter, we propose an efficient Side-Information Frame Generator (SIFG). It considers smoothness constraints of both the motion vector field and spatial adjacent pixels. Simulation results show that the proposed techniques provide potential rate-distortion performance advantages. Besides, the fine visual quality of the S frame is obtained.

Existing methods for inverse motion compensation (IMC) in the DCT domain have not considered the unrestricted motion vector (UMV). In the existing methods, IMC is performed to deal with the UMV in the spatial domain after the inverse DCT (IDCT). We propose an IMC method which can deal with the UMV directly in the DCT domain without the use of the IDCT/DCT required by the existing methods. The computational complexity of the proposed method can be reduced by about half of that of the brute-force method operating in the spatial domain. Experimental results show that the proposed method can efficiently reduce the processing time with similar visual quality.

In this paper, we propose a method to estimate affine motion parameters from consecutive images with the assumption that the motion in progress can be characterized by an affine model. The motion may be caused either by a moving camera or moving object. The proposed method first extracts motion vectors from a sequence of images and then processes them by adaptive robust estimation to obtain affine parameters. Typically, a robust estimation filters out outliers (velocity vectors that do not fit into the model) by fitting velocity vectors to a predefined model. To filter out potential outliers, our adaptive robust estimation defines a flexible weight function based on a sigmoid function. During the estimation process, we tune the sigmoid function gradually to its hard-limit as the errors between the input data and the estimation model are decreased, so that we can effectively separate non-outliers from outliers with the help of the finally tuned hard-limit form of the weight function. The experimental results show that the suggested approach is very effective in estimating affine parameters.

This paper proposes a fast motion estimation algorithm for variable block-sizes by utilizing motion vector bottom-up procedure for H.264. The refined motion vectors of adjacent small blocks are merged to predict the motion vectors of larger blocks for reducing the computation. Experimental results show that our proposed method has lower computational complexity than full search, fast full search and fast motion estimation of the H.264 reference software JM93 with slight quality decrease and little bit-rate increase.

When the frame size is downscaled for video transcoding, the new motion vector (MV) must be computed. This paper presents an algorithm to utilize the activity measurement by DC value and the number of non-zero quantized DCT coefficients in the residual macroblock to compose the motion vector. It can reduce the complexity for motion estimation and improve the performance of the spatial domain video transcoder.

A method of scrambling MPEG video by exchanging the motion vector (MV) in the MPEG bitstream is proposed. It deals directly with the MPEG bitstream and exclusive MPEG encoders are unnecessary. The size of the scrambled bitstream does not increase and image quality is maintained after descrambling. Moreover, the structure of the MPEG bitstream is maintained and can be decoded with a standard MPEG video decoder. We demonstrate the effectiveness of this method through simulation results that reveal unchanged image quality and size of bitstreams.

Compressed video bitstream is sensitive to errors that may degrade the reconstructed images severely even the bit error rate is small. One approach to combat the impact of error is error concealment at the decoder without increasing the bit rate and changing the encoder. We propose motion vector based error concealment algorithms to recover the motion vector per pixel instead of that per block according to the relation of neighboring motion vectors. The displacement per pixel can be estimated more accurately by using the tendency of neighboring motion vectors. Besides, we use not only the relation among motion vectors, but also the pixels. The pixels of the error block are divided into different parts according to their consistency with neighboring blocks and the displacement at each pixel of these parts is interpolated by relative motion vectors. From simulation results, the proposed motion vector based methods provide better reconstruction quality for damaged images than other methods.