From NYU Video Lab

HomePage: Perceptual Quality Maximization for Video Calls with Packet Losses by Optimizing FEC, Frame Rate and Quantization

We consider video calls over networks with bursty packet losses and varying bitrate, where loss resiliency is provided through frame-level forward error correction (FEC) due to tight delay constraints. For this problem, both encoding frame rate (eFR) and decoded frame rate (dFR) are crucial; a high eFR at low bitrates leads to a larger quantization stepsize (QS), smaller frames and thus suboptimal FEC, while a low eFR at high bitrates diminishes the perceptual quality. At the receiver, damaged frames and others predicted from them are typically discarded, reducing dFR. To mitigate frame losses, hierarchical-P coding (hPP) may be used at the cost of lower coding efficiency than IPP..I coding (IPP), which results in irregular frame freezes in case of loss.

In this paper, we study the received video call quality maximization for both hPP and IPP by jointly optimizing eFR, QS and the FEC redundancy rates, under the sending bitrate constraint. We use a perceptual quality model, Q-STAR, which depends on average dFR and QS, along with a rate model, R-STAR, which depends on eFR and QS. We cast the problem as a combinatorial optimization problem, and after replacing QS with the video bitrate, employ exhaustive search and hill climbing methods to solve for the eFR and the video bitrate. We also use a greedy FEC packet distribution algorithm to determine the FEC redundancy rate for each frame.

Evaluating the results for both hPP and IPP, we show, for iid losses,

For bursty losses, we show that


Retrieved from
Page last modified on April 04, 2017, at 11:25 PM EST