Intelligent Gap Filling

Description

Intelligent Gap Filling (IGF), specified in 3GPP TS 26.253, is an advanced media processing feature designed for streaming services, particularly Dynamic Adaptive Streaming over HTTP (DASH). Its primary function is to mitigate the negative impact of playback stalls (rebuffering events) on the user's Quality of Experience (QoE). When a player's buffer runs dry due to network throughput issues, instead of displaying a frozen screen or a spinner, IGF can generate and insert synthetic media content to fill the gap until the regular media stream can resume. This content is not pre-encoded and stored; it is intelligently created in real-time based on the surrounding actual media.

The architecture of IGF involves components both in the network and potentially at the client. A key entity is a media processing function, which could be located in an application server or as part of a Media Resource Function (MRF) in an IMS context. This function monitors the streaming session and, upon detecting or being informed of an impending buffer depletion, activates the gap filling procedure. It analyzes the last successfully decoded frames of the video and/or audio preceding the gap. Using techniques such as frame freezing, frame extrapolation, or generating comfort noise, it produces a seamless continuation of the media. This generated media is then packetized and delivered to the client as if it were part of the original stream, often using lower bitrate representations to conserve resources.

How IGF works is a multi-step process. First, the client or a network proxy detects buffer underrun and signals the condition (explicitly or implicitly) to the IGF function. The IGF function then takes the most recent video frame and audio samples. For video, it may repeatedly send that last frame (a "freeze frame") with appropriate timestamps, or it may create simple motion extrapolation. For audio, it may generate comfort noise or loop a short audio segment. Crucially, it manages the media timeline to ensure seamless splicing when the original high-quality stream becomes available again. The client decodes and renders this filler media, providing the user with a continuous, albeit potentially lower-quality, viewing experience instead of a complete interruption. IGF is closely related to other QoE features like bandwidth adaptation and switching logic in adaptive streaming.

Purpose & Motivation

IGF was developed to address a persistent and highly noticeable problem in internet video streaming: rebuffering. Even with advanced adaptive bitrate algorithms, temporary network congestion can cause the player's buffer to empty, forcing playback to halt. This "spinner" or frozen screen significantly degrades user satisfaction. The purpose of IGF is to intelligently mask these gaps, creating an illusion of continuous playback and thereby improving the perceived smoothness and quality of the service.

The motivation for its creation in 3GPP Release 18 stems from the increasing consumption of high-quality video (including 4K/8K, VR) over mobile networks, where bandwidth variability is inherent. Previous approaches relied solely on the client-side player to manage buffering, with no standardized way for the network to assist in mitigating the visual impact of stalls. IGF introduces a network-assisted, standardized mechanism to fill these gaps. It solves the problem of passive waiting during rebuffering by actively maintaining media output. This is particularly valuable for live streaming and real-time communications where pauses are especially disruptive, and it allows service providers to differentiate their offerings with a superior, more resilient QoE.

Key Features

• Dynamically generates filler media (video/audio) to cover rebuffering gaps
• Utilizes techniques like frame freezing, extrapolation, and comfort noise generation
• Operates within adaptive streaming (DASH) architectures
• Can be implemented as a network-based media processing function
• Maintains media timeline continuity for seamless transition post-gap
• Aims to improve perceived Quality of Experience (QoE) during network impairments

Evolution Across Releases

Defining Specifications