ControLling mUltiple streams for tElepresence

Description

CLUE stands for "ControLling mUltiple streams for tElepresence" and refers to the IETF working group that develops requirements and protocols for telepresence systems. In 3GPP, CLUE is referenced in specifications such as TS 23.333 (Multimedia Resource Function Controller), TS 24.103 (Telepresence using IP multimedia core network subsystem), TS 24.803 (IMS telepresence), TS 26.223 (Telepresence using IMS), and TS 26.923 (IMS telepresence interoperability). The term encompasses the CLUE protocol, which is used to negotiate and control the transmission of multiple media streams between telepresence endpoints, and the CLUE data channel, which carries the protocol messages.

Telepresence systems aim to provide a realistic, immersive meeting experience by transmitting multiple audio and video streams representing different participants or views. Unlike traditional video conferencing, which often uses a single mixed stream, telepresence requires the exchange of several streams that can be dynamically selected and arranged based on the room layout, speaker positions, and user preferences. The CLUE protocol enables endpoints to advertise their capabilities, such as the number of screens, cameras, and microphones, and to request specific streams from remote endpoints. This allows the receiving endpoint to compose a scene that best matches its local display and audio setup.

The CLUE protocol operates over a reliable data channel, typically established using the WebRTC data channel or a similar mechanism. It defines an XML-based messaging format for describing media captures (e.g., camera views, microphone signals) and media streams. The protocol supports advanced features such as simultaneous interpretation, multiple presentation streams, and spatial audio. It also handles dynamic changes, such as when a new participant joins or a camera is moved, by allowing endpoints to update their advertisements and requests in real time.

In 3GPP, CLUE is integrated into the IP Multimedia Subsystem (IMS) to support telepresence services. The IMS provides the session control, authentication, and QoS mechanisms, while CLUE handles the media stream negotiation. This integration is specified in TS 24.103 and TS 24.803, which define the IMS telepresence service and its interoperability with CLUE. The 3GPP specifications leverage the IETF CLUE protocol to enable multi-stream telepresence over mobile networks, ensuring that the service can be delivered with appropriate quality of service and security.

CLUE is important because it addresses the limitations of traditional conferencing protocols, which are not designed to handle the complexity of multiple, independent media streams. By providing a standardized way to control these streams, CLUE enables interoperable telepresence systems from different vendors. It also supports advanced use cases such as immersive telepresence, where the audio and video are rendered to create a sense of presence. As telepresence becomes more prevalent in enterprise and remote collaboration, CLUE plays a critical role in ensuring a high-quality, interoperable experience.

Purpose & Motivation

CLUE was created to solve the problem of controlling multiple media streams in telepresence systems, which traditional conferencing protocols could not handle effectively. In conventional video conferencing, a multipoint control unit (MCU) typically mixes audio and video into a single stream for each participant, which limits the ability to customize the view and audio for each endpoint. Telepresence systems, however, require the transmission of multiple independent streams to recreate a realistic meeting environment. Without a standardized control protocol, each vendor would implement proprietary mechanisms, leading to interoperability issues and limiting the adoption of telepresence.

The IETF CLUE working group was established to develop a protocol that allows endpoints to negotiate the exchange of multiple media streams dynamically. The protocol enables endpoints to describe their capabilities and preferences, such as the number of screens and speakers, and to request specific streams from remote endpoints. This allows the receiving endpoint to compose a scene that matches its local setup, providing a more immersive experience. The CLUE protocol also supports advanced features like simultaneous interpretation and spatial audio, which are essential for high-end telepresence.

In 3GPP, the adoption of CLUE ensures that telepresence services can be delivered over mobile networks with standardized interfaces. The 3GPP specifications reference CLUE to define how IMS-based telepresence services should use the protocol for media negotiation. This enables mobile operators to offer interoperable telepresence services that work across different devices and vendors. The motivation for 3GPP to include CLUE is to leverage existing IETF work and to ensure that telepresence can be integrated into the IMS framework, which provides QoS, security, and session control. By standardizing on CLUE, 3GPP facilitates the deployment of advanced telepresence services in mobile networks.