Multimedia data transmitted over the Internet often suffers from delay, jitter, and data loss. Data loss in particular can be extremely high on the Internet, often as high as 40%. Unlike traditional applications, multimedia applications can tolerate some data loss. A small gap in a video stream may not significantly impair media quality, and may not even be noticeable to users. However, too much data loss can result in unacceptable media quality. A number of techniques exist to repair packet loss in a media stream. These techniques have proven to be effective for audio stream data loss, but many have yet to be applied to video.
Interleaving assumes that better perceptual quality can be achieved by spreading out bursty packet losses in a media flow. In other words, several small gaps are better than a big gap in multimedia flow. For example, assume there is a frame consisting of several characters of information:
Assume that during transmission several characters in the frame get lost:
The first word is then very hard to reconstruct. However, the original frame can be interleaved as:
After applying the same loss to the interleaved frame, the frame can be reconstructed as:
It is much easier to "interpolate" the missing letters. The same idea has been applied to audio streams as a loss recovery technique. However, it is known that the human visual system is less sensitive than the human auditory system, thus a small gap in a video stream maybe less noticeable than a small gab in an audio stream, suggesting that interleaving may be more effective for video than for audio.
Multimedia streams are compressed before being transmitted over the network. The MPEG (Motion Picture Expert Group) achieves high compression rate by exploiting temporal redundancies of subsequent pictures. MPEG distinguishes 3 main frame types for image coding: I-frame, P-frame, and B-frame. I-frame stands for Intra-coded frame which are self-contained. P-frame stands for Predictive-coded frame. The encoding and decoding of P-frames requires the information of previous I-frames and/or all previous P-frames. B-frame stands for Bi-directionally predictive-coded frames. The encoding and decoding of B-frames requires the information of the previous and following I- and/or P-frame.
In our research, we propose a video interleaving approach to enhance repair from loss. We design and implement an interleaving strategy for video streams. The sender re-sequences the video stream before transmitting, so that original adjacent units are separated by a guaranteed distance in the transmitted stream, and the receiver returns them to their original order. The basic idea of interleaving is to spread out one big gap in the media stream into several small gaps. In this way the effect of the loss of multiple consecutive frames will be ameliorated, and the perceptual quality will be increased. Our interleaving approach is also combined with repetition error-recover, in which a lost frame is recovered by repeating the previous consecutive frame. For example, with the Group of Picture (GOP) patter IBBPBBPBB, Figure 1 demonstrates interleaving and the result of the first P-frame lost within a video stream, and Figure 2 shows the situation of the same P-frame lost in the same video stream without interleaving.
, Claypool, Mark
(2001). Demonstration of Ameliorating Packet Loss in a Video Stream by Using Interleaving. .
Retrieved from: https://digitalcommons.wpi.edu/computerscience-pubs/105