Screen Content Coding

Cloud based applications, such as virtual desktop interface, online gaming, shared screen collaboration, distance education, etc., have drawn more and more attention. Correspondingly, new opportunities and challenges are introduced on how to efficiently compress high-definition computer-generated screen content for limited bandwidth and computing resources. To better understand the natural statistics of screen contents and develop potential compression technologies, JCT-VC launched the development of screen content extension on top of HEVC in early 2014. Many new coding tools and algorithms (e.g., palette coding mode, intra block copy mode, adaptive color transform, etc.) were introduced during the standardization. The current testing model software (i.e., SCM) already outperforms conventional HEVC by over 40% bitrate reduction on typical screen contents.

In this project, we are utilizing machine learning techniques and statistical studies to develop fast encoding and transcoding solutions to significantly reduce encoder / transcoder complexity, while simultaneously preserve the Rate-Distortion performance. The sub-areas investigated or being investigated are (1) Screen Content Coding Standardization Contributions (collaborating with Huawei researchers), (2) Screen Content Fast Encoding Algorithms Design, and (3) Screen Content Fast Transcoding Algorithms Design.

Related Publications:

[J1] F. Duanmu, Z. Ma, and Y. Wang, “Fast Mode and Partition Decision Using Machine Learning for Intra-Frame Coding in HEVC Screen Content Coding Extension,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS), 2016 Aug; Vol: PP, Issue 99, Page:1-15. doi: 10.1109/JETCAS.2016.2597698

[C1] F. Duanmu, Z. Ma, and Y. Wang “A Novel Screen Content Fast Transcoding Framework Based on Statistical Study and Machine Learning”, in Proc. International Conference of Image Processing (ICIP), pp. 4205-4209, Phoenix, Arizona, USA, 2016.

[C2] F. Duanmu, Z. Ma, and Y. Wang, “Fast CU partition decision using machine learning for screen content compression”, Proc. IEEE International Conference on Image Processing (ICIP), pp. 4972 - 4976, Quebec City, Canada, 2015.

[C3] Y. Xu, W. Huang, W. Wang, F. Duanmu, and Z. Ma “2-D Index Map Coding for HEVC Screen Content Expression”, Proc. Data Compression Conference (DCC), pp. 263-272, Snowbird, Utah, USA, 2015.

[P1] “New SCC test sequence for consideration”, Doc. JCTVC-U0188, Warsaw, June, 2015.

[P2] “Non-CE1: Index Map Splitting (IMS) Mode for Palette Coding in HEVC SCC”, Doc. JCTVC-U0148, Warsaw, June, 2015.

[P3] “Non-CE1: Context model unification for palette run type flags in HEVC SCC”, Doc. JCTVC-U0149, Warsaw, June, 2015.

[P4] “Non-SCCE1: Analysis of Full Frame IBC Block Vector Distribution”, Doc. JCTVC-R0269, Sapporo, July, 2014.

[P5] “Non-SCCE3: Results of SCCE3 with support of 2x4 CTUs as reference buffer”, Doc. JCTVC-R0268, Sapporo, July, 2014.

[P6] “Non-SCCE3: Improvements for SCCE3 Test C3”, Doc. JCTVC-R0304, Sapporo, July, 2014.

Page last modified on January 12, 2017, at 11:43 AM EST