This paper describes a novel 4K 120fps (frames per second) real-time HEVC (High Efficiency Video Coding) encoder for high-frame-rate video encoding and transmission. Motion portrayal problems such as motion blur and jerkiness may occur in video scenes containing fast-moving objects or quick camera panning. A high-frame-rate solves such problems and provides a more immersive viewing experience that can express even the fast-moving scenes without discomfort. It can also be used in remote operation for scenes with high motion, such as VAR (Video Assistant Referee) systems in sports. Real-time encoding of high-frame-rate videos with low latency and temporal scalability is required for providing such high-frame-rate video services. The proposed encoder achieves full 4K/120fps real-time encoding, which is twice the current 4K service frame rate of 60fps, by multichip configuration with two encoder LSI. Exchange of reference picture data near a spatially divided slice boundary provides cross-chip motion estimation, and maintains the coding efficiency. The encoder supports temporal-scalable coding mode, in which it output stream with temporal scalability transmitted over one or two transmission paths. The encoder also supports the other mode, low-delay coding mode, in which it achieves 21.8msec low-latency processing through motion vector restriction. Evaluation of the proposed encoder's multichip configuration shows that the BD-bitrate (the average rate of bitrate increase), compared to simple slice division without inter-chip transfer, is -2.86% at minimum and -2.41% on average in temporal-scalable coding mode. The proposed encoder system will open the door to the next generation of high-frame-rate UHDTV (ultra-high-definition television) services.

This paper describes a novel low-latency 4K 60 fps HEVC (high efficiency video coding)/H.265 multi-channel encoding system with content-aware bitrate control for live streaming. Adaptive bitrate (ABR) streaming techniques, such as MPEG-DASH (dynamic adaptive streaming over HTTP) and HLS (HTTP live streaming), spread widely on Internet video streaming. Live content has increased with the expansion of streaming services, which has led to demands for traffic reduction and low latency. To reduce network traffic, we propose content-aware dynamic and seamless bitrate control that supports multi-channel real-time encoding for ABR, including 4K 60 fps video. Our method further supports chunked packaging transfer to provide low-latency streaming. We adopt a hybrid architecture consisting of hardware and software processing. The system consists of multiple 4K HEVC encoder LSIs that each LSI can encode 4K 60 fps or up to high-definition (HD) ×4 videos efficiently with the proposed bitrate control method. The software takes the packaging process according to the various streaming protocol. Experimental results indicate that our method reduces encoding bitrates obtained with constant bitrate encoding by as much as 56.7%, and the streaming latency over MPEG-DASH is 1.77 seconds.

In this paper, we describe a novel low-delay 4K 120-fps real-time HEVC decoder with a parallel processing architecture that conforms to the HEVC main 4:2:2 10 profile. It supports the hierarchical temporal scalable streams required for Ultra High Definition high-frame-rate broadcasting and also supports low-delay and high-bitrate decoding for video transmission uses. To achieve this support, the decoding processes are parallelized and pipelined at the frame level, slice level, and coding tree unit row level. The proposed decoder was implemented on three FPGAs operated at 133 and 150 MHz, and it achieved 300-Mbps stream decoding and 37-msec end-to-end delay with our concurrently developed 4K 120-fps encoder.

This paper proposes an efficient reference image sharing method for the image-division parallel video encoding architecture. This method efficiently reduces the amount of data transfer by using pre-transfer with area prediction and on-demand transfer with a transfer management table. Experimental results show that the data transfer can be reduced to 19.8-35.3% of the conventional method on average without major degradation of coding performance. This makes it possible to reduce the required bandwidth of the inter-chip transfer interface by saving the amount of data transfer.

This paper proposes a very small on-chip multimedia real-time OS for embedded system LSIs, and demonstrates its usefulness on MPEG-2 multimedia applications. The real-time OS, which has a conditional cyclic task with suspend and resume for interacting hardware (HW) / software (SW) of embedded system LSIs, implements the minimum set of task, interrupt, and semaphore managements on the basis of an analysis of embedded software requirements. It requires only about 2.5 Kbytes memory on run-time, reduces redundant conventional cyclic task execution steps to about 1/2 for HW/SW interactions, and provides sufficient performance in real-time through implementing two typical embedded softwares for practical multimedia system LSIs: an MPEG-2 system protocol LSI and an MPEG-2 video encoder LSI. This on-chip multimedia real-time OS with 2.5 Kbyte memory will be acceptable for future multimedia embedded system LSIs.

An H.264/AVC encoder LSI (named “SARA”) that supports High422 profile, as well as 422 profile of MPEG-2, has been developed for HDTV broadcasting infrastructures. It contains three motion estimation and compensation (ME/MC) engines with wide search ranges of -217.75 to +199.75 horizontally, -109.75 to +145.75 vertically, which can utilize almost all H.264/AVC ME/MC coding tools, such as multiple reference frame, variable block size, quarter-pel prediction, macroblock adaptive field/frame prediction (MBAFF), spatial/temporal direct mode, and weighted prediction. Our evaluations show that it can encode fast moving scenes with 1.2 dB to 1.7 dB higher than the JM. It was successfully fabricated in a 90-nm technology, and integrates 140 million transistors.