Description
The System for the Evaluation of Audio Quality (SEAQ) is a comprehensive methodology defined in 3GPP TS 26.936 for the subjective and objective assessment of speech and audio quality in telecommunications systems. It establishes standardized procedures for conducting listening tests, which involve human subjects evaluating processed speech samples under controlled conditions. These tests generate Mean Opinion Scores (MOS) or other perceptual metrics that quantify audio quality as experienced by end-users. The system encompasses detailed guidelines for test environment setup, including acoustic requirements for listening rooms, calibration of playback equipment, and selection of representative speech material and background noise conditions.
SEAQ operates by defining specific test methodologies such as Absolute Category Rating (ACR), Degradation Category Rating (DCR), and Comparison Category Rating (CCR) to measure different aspects of audio quality. The process involves playing processed speech samples—which may have undergone codec compression, packet loss concealment, or other network impairments—to a panel of listeners who rate the quality on standardized scales. Statistical analysis is then applied to the collected ratings to produce reliable quality scores with confidence intervals. The system also addresses the training of listeners, screening for hearing impairments, and the use of reference conditions to ensure test consistency and reproducibility across different laboratories and testing organizations.
Key components of SEAQ include the test supervisor who designs and oversees the evaluation, the listening panel comprising trained or untrained subjects depending on the test method, the playback system with precisely calibrated audio equipment, and the speech material database containing standardized recordings in multiple languages. The system defines specific impairment conditions that simulate real network scenarios, such as varying bit rates, frame erasures, delay, and tandem encodings. SEAQ's role in the network ecosystem is critical for validating that speech codecs (like AMR, EVS, or future codecs) and voice services (VoLTE, VoNR) meet quality requirements before deployment, ensuring interoperability and consistent user experience across different network implementations and device manufacturers.
Purpose & Motivation
SEAQ was created to address the need for a standardized, reliable, and repeatable method to evaluate the perceptual audio quality of speech codecs and voice services in mobile networks. Prior to its standardization, different equipment vendors and network operators often used proprietary testing methodologies, making it difficult to compare codec performance objectively or guarantee consistent voice quality across interoperable systems. This lack of standardization could lead to discrepancies in quality assessments, hindering the selection of optimal codecs for network deployment and potentially degrading the end-user experience in multi-vendor environments.
The technology solves the problem of subjective quality evaluation by providing a unified framework that ensures tests are conducted under scientifically controlled conditions, producing results that are statistically valid and comparable across different testing occasions and laboratories. It addresses the challenge of assessing how codecs perform under realistic network impairments—like packet loss, jitter, and bandwidth variations—which significantly affect perceived quality but are difficult to model accurately without standardized test conditions. SEAQ enables the 3GPP to make informed decisions during codec standardization by providing empirical evidence of quality under various operational scenarios.
Historically, as mobile networks evolved from 2G to 3G and beyond, introducing new wideband and fullband codecs, the complexity of quality evaluation increased. SEAQ provided the necessary methodology to validate that these advanced codecs (e.g., AMR-WB, EVS) delivered their promised quality improvements over legacy narrowband codecs. It also supports the evaluation of quality for emerging services like Voice over LTE (VoLTE) and Voice over NR (VoNR), where packet-switched transmission introduces new quality impairment factors that circuit-switched voice did not encounter.
Key Features
- Standardized methodologies for subjective listening tests (ACR, DCR, CCR)
- Detailed specifications for test environment acoustics and equipment calibration
- Guidelines for selection and training of listening panels
- Procedures for statistical analysis of results with confidence intervals
- Definition of representative speech material and background noise conditions
- Support for testing under various network impairment scenarios
Evolution Across Releases
Initially introduced in TS 26.936, providing the foundational framework for subjective audio quality evaluation. Established core methodologies including Absolute Category Rating (ACR) tests, specifications for listening room acoustics, equipment calibration requirements, and basic statistical analysis procedures for processing listener scores.
Defining Specifications
| Specification | Title |
|---|---|
| TS 26.936 | 3GPP TS 26.936 |