This paper describes a novel 4K 120fps (frames per second) real-time HEVC (High Efficiency Video Coding) encoder for high-frame-rate video encoding and transmission. Motion portrayal problems such as motion blur and jerkiness may occur in video scenes containing fast-moving objects or quick camera panning. A high-frame-rate solves such problems and provides a more immersive viewing experience that can express even the fast-moving scenes without discomfort. It can also be used in remote operation for scenes with high motion, such as VAR (Video Assistant Referee) systems in sports. Real-time encoding of high-frame-rate videos with low latency and temporal scalability is required for providing such high-frame-rate video services. The proposed encoder achieves full 4K/120fps real-time encoding, which is twice the current 4K service frame rate of 60fps, by multichip configuration with two encoder LSI. Exchange of reference picture data near a spatially divided slice boundary provides cross-chip motion estimation, and maintains the coding efficiency. The encoder supports temporal-scalable coding mode, in which it output stream with temporal scalability transmitted over one or two transmission paths. The encoder also supports the other mode, low-delay coding mode, in which it achieves 21.8msec low-latency processing through motion vector restriction. Evaluation of the proposed encoder's multichip configuration shows that the BD-bitrate (the average rate of bitrate increase), compared to simple slice division without inter-chip transfer, is -2.86% at minimum and -2.41% on average in temporal-scalable coding mode. The proposed encoder system will open the door to the next generation of high-frame-rate UHDTV (ultra-high-definition television) services.

This paper describes a novel low-latency 4K 60 fps HEVC (high efficiency video coding)/H.265 multi-channel encoding system with content-aware bitrate control for live streaming. Adaptive bitrate (ABR) streaming techniques, such as MPEG-DASH (dynamic adaptive streaming over HTTP) and HLS (HTTP live streaming), spread widely on Internet video streaming. Live content has increased with the expansion of streaming services, which has led to demands for traffic reduction and low latency. To reduce network traffic, we propose content-aware dynamic and seamless bitrate control that supports multi-channel real-time encoding for ABR, including 4K 60 fps video. Our method further supports chunked packaging transfer to provide low-latency streaming. We adopt a hybrid architecture consisting of hardware and software processing. The system consists of multiple 4K HEVC encoder LSIs that each LSI can encode 4K 60 fps or up to high-definition (HD) ×4 videos efficiently with the proposed bitrate control method. The software takes the packaging process according to the various streaming protocol. Experimental results indicate that our method reduces encoding bitrates obtained with constant bitrate encoding by as much as 56.7%, and the streaming latency over MPEG-DASH is 1.77 seconds.

In this paper, we describe a novel low-delay 4K 120-fps real-time HEVC decoder with a parallel processing architecture that conforms to the HEVC main 4:2:2 10 profile. It supports the hierarchical temporal scalable streams required for Ultra High Definition high-frame-rate broadcasting and also supports low-delay and high-bitrate decoding for video transmission uses. To achieve this support, the decoding processes are parallelized and pipelined at the frame level, slice level, and coding tree unit row level. The proposed decoder was implemented on three FPGAs operated at 133 and 150 MHz, and it achieved 300-Mbps stream decoding and 37-msec end-to-end delay with our concurrently developed 4K 120-fps encoder.

We have proposed a method to detect and quantitatively extract anomalies from surveillance videos. Using our method, anomalies are detected as patterns based on spatio-temporal features that are outliers in new feature space. Conventional anomaly detection methods use features such as tracks or local spatio-temporal features, both of which provide insufficient timing information. Using our method, the principal components of spatio-temporal features of change are extracted from the frames of video sequences of several seconds duration. This enables anomalies based on movement irregularity, both position and speed, to be determined and thus permits the automatic detection of anomal events in sequences of constant length without regard to their start and end. We used a 1-class SVM, which is a non-supervised outlier detection method. The output from the SVM indicates the distance between the outlier and the concentrated base pattern. We demonstrated that the anomalies extracted using our method subjectively matched perceived irregularities in the pattern of movements. Our method is useful in surveillance services because the captured images can be shown in the order of anomality, which significantly reduces the time needed.