From NYU Video Lab

HomePage: 360-degree Video Coding

360-degree video has become popular in recent years with the advances in virtual reality (VR) and augmented reality (AR) technologies and has been rapidly commercialized in a variety of applications, such as immersive cinema, gaming, education, telepresence, healthcare, social media and 360-degree video streaming, etc. To provide the users with an immersive experience, 360-degree video requires much higher bandwidth compared with conventional 2D video, due to the increase in resolution, frame rate, and quality of experience (QoE) requirements. For example, a premium quality 360-degree stereo video with 90 frames per second (fps) at 8K resolution can easily consume bandwidth up to multiple gigabits-per-second (Gbps). Therefore, efficient compression and delivery of ultra-high quality 360-degree video becomes important for the wide adoption of VR and AR applications.

In this project, we are collaborating with InterDigital researchers and exploring efficient 360-degree video sampling and projection solutions to better compress 360-degree videos. A hybrid cubemap projection (HCP) format is proposed to generalize prior CMP and other CMP-like projections (e.g., adjusted cubemap or equi-angular cubemap), and allow adaptive selections of optimal transform functions for each direction over each cube face based on the specific content characteristics inside that face. Experimental results demonstrate that the proposed HCP format achieves on average luma (Y) BD-rate reduction of 11.51%, 8.0%, and 0.54% compared to the equirectangular projection (ERP) format, cubemap projection (CMP) format, and adjusted cubemap projection (ACP) format, respectively, in terms of end-to-end WS-PSNR.

Related Publications:

F. Duanmu, Y. He, X. Xiu, P. Hanhart, Y. Ye, and Y. Wang, “Content-Adaptive 360-degree Video Coding Using Hybrid Cubemap Projection,” IEEE Picture Coding Symposium (PCS), San Francisco, California, USA, 2018 (Under Review).

F. Duanmu, Y. He, X. Xiu, P. Hanhart, Y. Ye, and Y. Wang, “Hybrid Cubemap Projection Format for 360-Degree Video Coding,” in Proc. of IEEE Data Compression Conference (DCC), Snowbird, Utah, USA, 2018 (Accepted).

Last Update by Fanyi Duanmu, 02/14/2018

Retrieved from
Page last modified on February 20, 2018, at 12:05 PM EST