Fehlertolerante Videokommunikation über verlustbehaftete

Traffic Charaterization and Fault Tolerance of H.264/AVC Encoded
Video Streams for QoS-Management in Computer Networks
Klaus Heidtmann, Michael Kiritz, Jens Norgall
Telecommunications and Computer Networks Division
Department of Computer Science, Hamburg University,
Vogt-Kölln-Str. 30, D-22527 Hamburg, Germany
[email protected]
Abstract Distributed video applications are an emerging area of our modern information society.
They are frequently integrated into multimedia-applications and process video data streams
produced by video encoders. This processing imposes high storage and throughput requirements
as well as temporal constrains due to the real-time aspect on communication systems. For instance
real-time video communication like video conferencing does not perform very favourable when
run over best-effort networks without quality-of-service management. Because of data losses or
high delays as consequences of congestions, bottlenecks or radio-interference in WLANs the
service offered by such a video application can deteriorate. To support the solution of such
problems this paper presents a study of the output process of video encoders to quantify and
characterize the requirements of video applications on networks. In particular we measure and
investigate the stochastic traffic characteristics of H.264/AVC-encoded video streams as network
load. This can support the solution of the problem to dimension, configure or parameterize video
applications or required services of underlying communication systems. We used the presented
results especially to derive and calibrate our analytical and simulation models and tools for load
generation as well as reliability and performance evaluation of real-time video communication.
Key words: multimedia system, quality of service, real-time video encoding, video stream, frame
length, compression, traffic characterization, network load, H.264, video communication
1. INTRODUCTION
Video applications like video telephony, video conferencing, video mail and videoon-demand are emerging areas and are already resp. will soon be part of our working
and private life. These applications impose high throughput requirements on computer
and communication systems, which range from high data rates, due to the voluminous
nature of video data, to temporal constraints, due to the continuous resp. real-time
aspects of video presentation and communication. To quantify and characterize these
requirements we study the output process of video encoders, which represent an
integral subsystem of video applications and produce video streams. As results we
present detailed measurements and traffic characteristics of H.264/AVC-encoded
video streams when leaving the video encoder as network load in order to be
transmitted by a computer network or communication system.
During the last decade we carried out comprehensive and detailed measurements
based on many different video sequences and the well-established H.261, H.263 and
H.264/AVC standards for video encoding [3,4,5,9,10,16]. In the following we present
some examples of the latest results [8,11]. So in section 2 this paper starts with the
presentation of measurements and comparison of the frame lengths of H.263- and
H.264/AVC-encoded video streams as their characteristic attribute. In the following
section 3 the previous measurements are used to look for distributions which reflect
the main characteristics of the observed video application. It follows an investigation
of autocorrelations of frame lengths in section 4. Finally we investigate the influences
of fault tolerance mechanisms integrated into the H.264/AVC-standard. These
mechanisms can be used to reconcile the size of the video stream and the picture
quality of the video to integrate quality of service in computer networks, especially in
case of faulty real-time video communication.
The presented results give some insight into the stochastic process of video
streams as produced by video encoders and can be applied to design and manage video
systems as subsystems of computer and communication systems skilfully. Video
systems as an integral part of multimedia systems are characterized by the computercontrolled integrated generation, manipulation, representation, storage and communication of digital video information. With the derived traffic characteristics one can
develop efficient mechanisms to control the video encoder so that it produces an
appropriate video data stream as network load for transmission. Based on the
characterization of a single video source introduced in this paper one can characterize
any mix of video streams by means of overlaying single stream models. Furthermore
the results are useful to provide a suitable or optimal quality of service of the
communication system for distributed video applications. The measurements and
traffic characterization can support the solution of the problem to dimension,
configure or parameterize video applications or required services of underlying
communication systems. Here, parameterization comprises a careful tuning of video
coding parameters. In a distributed environment with fixed resources our results can
be used to provide the best video quality from a single user point of view or from an
overall network point of view, where a maximum number of distributed users has to
be served. We use the presented results especially to derive and calibrate our
analytical and simulation models and tools for reliability and performance evaluation
of real-time video communication [3,6,13,15,16], especially for our eLearning tool
VideoExplorativ [2] and our load generator UniLoG (Universal Load Generator) [1].
2. MEASUREMENT OF FRAME LENGTHS
The standard H.261 defines two types of frames, i.e. I- and P-frames, while H.263
and H.264 define additional optional frame types [7]. The relative frequency of these
types of frames within a video stream is a considerable parameter and at least the first
picture of a video stream must be an intraframe. More interframes means more
computation, less data and a stronger dependence among different frames. Important
2
aspects for the picture sequence structure in an implementation are time constraints,
buffer size or the desired compression and quality trade-off. Real-time services
distinguish two different coding options, constant bit rate (CBR) and variable bit rate
(VBR). VBR uses fixed coding options (as opposed to CBR), e.g. fixed quantization
level, so the induced data rate is dependent on the motion intensity and entropy of the
coded video. Because of these fluctuations and their effects, e.g. on network
congestion, it is very important to derive adequate models for this type of traffic. So,
our following studies will refer to VBR encoded video streams with a beginning Iframe followed exclusively by P-frames and baseline profile in case of H.264/AVC.
In the following we want to observe the compressed video stream as induced by the
video encoder just using I- and P-frames. As we refer to interactive video applications
with real-time constraints the arrival process of the isochronous video streams is very
regular, so we can restrict ourselves to measurements characterizing the attributes of
interest of the compressed video frames to be transmitted. As is usual in modeling
video sources in the following we assume that length of frames is the only attribute of
interest for the requests. Therefore, we have to measure the length xi (in Bit resp. Byte)
of the i-th video frame leaving the video encoder. Thus, the collected trace of frame
lengths X = {xi | i=1,2,...,n} with n observed frames describes the video stream. This
trace leads to an empirical distribution function, which can be considered as traffic
characterization concerning the marginal distribution function of frame lengths.
Evidently, the distributions of lengths of data units have a strong impact in case of
static resource reservations during data transmission as well as in case of an adaptive
model-based quality-of-service management of communication systems.
During the last decade we carried out comprehensive measurements based on many
different video sequences and the well-established H.261, H.263 and H.264 standards
for video encoding. So now we want to exemplify the latest results [8,11] by
discussing two series of experiments in some more detail, in particular choosing
mainly H.264/AVC encoding of the sequences
 claire, a news announcer in a sequence with very low motion intensity.
 mobile, a video-recording taken from within a driving car and representing a
sequence with periods of rat her high motion intensity.
The frame lengths of the videos with H.264/AVC-encoding show the same
structure as H.263, but are 40-70% smaller. Two representative example traces of Pframe lengths are shown in Fig. 1. The traces of H.264-encodings with different
quantization are nearly interchangeable. For the same quantization steps size H.264encoded videos have a 2-4 dB higher mean PSNR compared to H.263. Detailed
measurements show that this advantage is based on the combination of the deblocking
filter, the extended intra-prediction, the variable size of the predicted blocks and the
motion compensation at subpixel level. The measurements to compare the
compression of the different encoding standards presume the same video quality, e.g.
identical PSNR.
3
Fig. 1. P-Frame lengths for video claire (left) and mobile (right), H263 resp. H.264 encoding
Table 1 shows the different compression achieved by the various encodings. One
can see that from one standard to the next there is nearly a doubling of the
compression factor. For a communication system with a fixed bandwidth this means
that it cam support twice as much video connections from one encoding to the next
higher one.
H.261
H.263
H.264
claire
compression factor
51,2
111,4
216,6
mobile
compression factor
3,1
6,4
14,6
Tab. 1: Compression factors for 261, H.263 and H.264 for the two example videos
The drawings of the empirical frame length distribution for the measured traces
look like astonishingly good approximation of Gaussian distributions. This leads to the
hypothesis that lengths of P-frames produced as a result of H.263- resp. H.264/AVCencodings can be closely approximated by normal distributions.
3. APPROXIMATE DISTRIBUTION OF FRAME LENGTH
In order to investigate the validity of the above hypothesis we repeated
approximation of observed empirical lengths distributions by Gaussian distribution for
all these standards and a variety of video sequences. The level of accuracy achievable
by the approximation was very satisfactory for all samples. As a graphical illustration
of typical observations we refer to Fig. 2 with the H.264/AVC-encoded video
sequences claire and mobile.
4
Fig. 2. Length distribution of P-frames for Claire (left) and mobile (right)
In order to judge the accuracy of the maximum likelihood estimates quantitatively,
by means of a ²-test, we tested the empirical distribution for normal distribution.
Here, classification into 13 partitions (10 degrees of freedom with 2 estimated
parameters) has been carried out. The significance level has been chosen to be =0.01
leading to a significance size of ²0.01,10 = 23.209 which is considerably higher than
the values reached by any of the video sequences observed. So we can accept the
hypothesis of normal distribution and even some more restrictive values of the
significance size would not directly lead to rejection of the hypothesis.
Thus, it seems acceptable to characterize the marginal distribution function of
video frame lengths by approximate normal distributions. An important advantage of
this approach results from the fact that the normal distribution is determined by only
two parameters and it allows a straight-forward derivation of quantiles and other
statistical quantities of the distribution function.
4. AUTOCORRELATION
Based on the results of modeling the one-dimensional marginal distribution of the
frame length we now want to take a closer look at their autocorrelations. Although the
marginal distribution of lengths is a basic and very important measure to characterize
the data streams of video applications, it is necessary to evaluate the autocorrelation
coefficients to get an adequate characterization. This is a result of the fact, that
independence assumptions induce a smoother stream than highly correlated processes
such as fractal traffic processes, and therefore independence assumptions can be
misleading in many cases. So we have to take a closer look at the autocorrelation
function of the traces. The measurements of autocorrelation coefficients (autocorrelation function) are shown in Fig. 3.
5
Fig. 3: The empirical autocorrelation function of the P-frame length traces of H.264 for Claire and mobile
Analyzing the measurement results in Fig. 3 leads to the conclusion, that a non
negligible autocorrelation up to lag  = 30 does exist in the trace mobile, while the
video claire, whose picture contents show the strongest correlation as consequence of
low motions, shows the lowest autocorrelation regarding the frame lengths. The
oscillating behavior of the autocorrelation function can be explained by low motion,
so that the coding decisions of producing predictive difference pictures alternate
periodically. A closer look at the more strongly correlated frame length traces of
mobile and other video sequences reveal changes in the level of required bandwidth,
corresponding to the level of motion within these videos. This leads to the assumption,
that correlation is based on changes in bandwidth requirements and finally on
alteration of motion and picture contents. In most video sequences the major fraction
of a picture content persists for a longer while and the motion intensity is maintained
over several frames, so that in these subintervals no rapid changes in bandwidth
requirements can be observed.
5. FAULT TOLERANCE
Finally we show the influence of the fault tolerance mechanism within H.264/AVC,
where 11% additional I-macroblocks in P-frames were enforced. The effects of single
packet losses at equidistant points of time for such videos can be subdivided into two
categories: If the number of packet losses is small and they have a great distance, the
video quality can recover the faults between the lost frames. Then the faults behave
like a sequence of single losses, which do not affect one another. In the second
situation the number of packet losses is higher, so that the distance between lost
frames is smaller than what is needed to recover from the faults. So the negative
influence of a loss on the video quality lasts longer than the time until the next loss.
Hence, the negative influences aggregate and prevent the video from recovering. Fig.
4 illustrates these two different situations in case of 1% and 10 % packet losses. We
did a lot of measurements concerning various fault tolerance mechanisms in
6
H.246/AVC and many different situations of packet loss [8], e.g. bursts, and these
investigations are still in progress.
Fig. 4. Quality of received video after regular single packet losses
6. SUMMARY AND OUTLOOK
In this paper we presented a selection of our detailed measurements video streams,
especially for H.264/AVC. First we showed traces of P-frame lengths. The observed
distributions of their lengths lead to the hypothesis that frame lengths can be approximated by a normal distribution. So we approximated the observed empirical lengths
distributions by Gaussian distribution for a variety of different video sequences
achieving a very satisfactory level of accuracy in all cases. As independence induces a
smoother stream than highly correlated processes we took a closer look at the autocorrelation noticing that a non negligible autocorrelation exists. Furthermore we
realized that correlation is based on changes in bandwidth requirements and finally on
alteration of motion and picture contents. Hence, the correlation structure is a result of
long-term correlations. Consequently, modeling of the frame length process over a
long time has to regard non negligible autocorrelation structure. Finally we investigated the influence of a fault tolerance mechanism on the video quality.
The derived characteristics of video streams can be applied to develop efficient
mechanisms to control the video codex so that it produces an appropriate video data
stream to be transmitted. They can be used straight forward to compute characteristics
of network load and traffic induced by video communication [15,16]. Moreover, our
results can be applied to manage networks for this type of real-time traffic or to
develop new components for suitable communication systems. In the case of computer
network analyses, our realistic and manageable load characterization is useful, e.g.
when applying analytical models or when executing simulation experiments
[1,6,11,15,16]. Currently the measured traces and distributions regarding H.264/AVC
are integrated into our load generator UniLoG, which can be used within our network
emulator NetEmu [12].
7
REFERENCES
[1] CONG J., Load Specification and Load Generation for Multimedia Traffic Load in Computer
Networks, PhD thesis, Dept. Comp. Science, Univ. Hamburg, 2006, published as: WOLFINGER B.
(Ed.) Berichte aus dem Forschungsschwerpunkt Telekommunikation und Rechnernetze, ShakerVerlag, Vol. 4, Aachen, 2006
[2] FIOLKA K., HEIDTMANN K., WOLFINGER B., Ein eLearning -Werkzeug zur Videokommunikation über simulierte verlustbehaftete Netze, GI-Workshop "Elektronische Unterstützung der Präsenzlehre", GI-Jahrestagung INFORMATIK 2004, Ulm, Sept. 2004
[3] HEIDTMANN K, Evaluation of Video Communication over Packet Switching Networks, 3rd
European Dependable Computing Conference EDCC-3, Prague, 1999
[4] HEIDTMANN K., Fault-Tolerant Real-Time Videocommunication via Best-Effort Networks, 4th
CaberNet Workshop, Pisa, Oct. 2001
[5] HEIDTMANN K., KOHLHAAS C., ZADDACH M., Messung der Netzlast und Bewertung der
Bildqualität bei Videokommunikation über Paketvermittlungsnetze, Proc. ARCS’99, Jena, Oct. 1999
[6] HEIDTMANN K., WOLFINGER B., Analytische Leistungsbewertung von Videokommunikation
gemäss H.261 über verlustbehaftete Paketvermittlungsnetze, 10. GI/ITG-Fachtagung über Messung,
Modellierung und Bewertung von Rechen- und Kommunikationssystemen MMB'99, Trier, Sept.
1999, 87-104
[7] ITU-T Recommendation H.264: Advanced video coding for generic audiovisual services, 2005
[8] KIRITZ M., Untersuchung von Fehlerrobustheitstechniken für H.264/AVC-kodierte Videoströme,
diploma thesis, Dept. Comp. Science, Univ. Hamburg, 2006
[9] KIRITZ M., Untersuchung der Fehlerrobustheit H.264/AVC-kodierter Videoströme bei simulierten
Übertragungsverlusten, MMBnet 2007,4. GI/ITG-Workshop, Hamburg, 13./14. Sept. 2007, in [14]
[10] NORGALL J., Lastanalyse zur Modellierung H.264/AVC-kodierter Videoströme, MMBnet 2005, 3
GI/ITG-Workshop, Hamburg, 8./9. Sept. 2007in [12]
[11] NORGALL J., Verkehrslast und Fehlerverhalten H.264/AVC-kodierter Videoströme in paketbasierten Kommunikationssystemen, diploma thesis, Dept. Comp. Science, Univ. Hamburg, 2006
[12] SCHERPE C., Emulation gekoppelter Rechnernetze mit lastabhängigem Verzögerungs- und Verlustverhalte - Architekturkonzepte, prototypische Realisierung und Fallstudien, PhD thesis, Dept. Comp.
Science, Univ. Hamburg, 2006, published as: WOLFINGER B. (Ed.), Berichte aus dem Forschungsschwerpunkt Telekommunikation und Rechnernetze, Vol. 4, Shaker-Verlag, Aachen, 2006
[13] WOLFINGER B., HEIDTMANN K. (Ed.), Leistungs-, Zuverlässigkeits- und Verlässlichkeitsbewertung von Kommunikationsnetzen und verteilten Systemen, 3. GI/ITG-Workshop MMBnet
2005, 8./9. Sept. Hamburg, Report No. 263, Dept. Comp. Science, Univ. Hamburg, 2005
[14] WOLFINGER B., HEIDTMANN K. (Ed.), Leistungs-, Zuverlässigkeits- und Verlässlichkeitsbewertung von Kommunikationsnetzen und verteilten Systemen, 4. GI/ITG-Workshop MMBnet
2007, 13./14. Sept. Hamburg, Report No. 281, Dept. Comp. Science, Univ. Hamburg, 2007
[15] WOLFINGER B., ZADDACH M., BAI G., HEIDTMANN K., Analytical Modeling of Primary and
Secondary Load as Induced by Video Applications Using UDP/IP, Intern. Symp. on Performance
Evaluation of Computer and Telecommunications Systems, SPECTS 2001, Orlando. Also in:
Computer Communications Journal, 2002
[16] ZADDACH M., Modellierung, Charakterisierung und Transformation von Videoverkehrslasten, PhD
thesis, Dept. Comp. Science, Univ. Hamburg, 2001, published as: WOLFINGER B. (Ed.) Berichte
aus dem Forschungsschwerpunkt Telekommunikation und Rechnernetze, Vol. 2, Shaker-Verlag,
Aachen, 2001
8