Neural Processing Unit

Description

The Neural Processing Unit (NPU) is a dedicated hardware accelerator designed to execute artificial intelligence (AI) and machine learning (ML) workloads with high efficiency, low latency, and reduced power consumption compared to general-purpose CPUs or GPUs. In the context of 3GPP Release 19, the NPU concept is studied and specified in technical reports such as 26.847 and 26.927, which address AI/ML media services and related codec and processing enhancements. The NPU enables neural-network-based functions to run directly within user equipment (UE), media gateways, or edge computing nodes, supporting use cases such as AI-driven video encoding, super-resolution, noise suppression, and speech enhancement.

Architecturally, an NPU typically consists of a matrix multiplication engine, vector processing units, on-chip memory (SRAM) for weights and activations, and a compiler/runtime stack that maps neural network graphs onto the hardware. In 3GPP systems, the NPU may be integrated into the UE modem or application processor, or deployed in the 5G core/edge as part of an AI/ML application function (AF). The 3GPP work in Rel-19 focuses on how media services can exploit NPUs for AI-based codecs (e.g., neural network-based video coding, NN-VC) and for quality-of-experience optimization, while maintaining interoperability and signaling through existing 5G media streaming and IMS frameworks.

From a protocol perspective, the NPU itself is not a 3GPP-defined network element with its own interface; rather, it is a hardware capability that supports AI/ML enablers specified in 3GPP. The relevant specifications define how AI/ML model management, inference offloading, and performance metrics are exchanged between UE and network, allowing the network to select appropriate AI/ML models and to negotiate capabilities. For example, an application function may request AI-based media processing, and the UE may indicate its NPU capabilities so that the network can decide whether to perform processing locally or in the cloud.

The introduction of NPU-related work in Rel-19 reflects the broader 3GPP trend of integrating AI/ML into 5G-Advanced. It complements other AI/ML studies in RAN (e.g., CSI feedback enhancement, beam management) and SA2 (e.g., network data analytics). By defining how NPUs can be leveraged for media services, 3GPP ensures that AI-enhanced applications can be delivered consistently across devices and networks, with standardized signaling for capability exchange, model delivery, and performance monitoring.

In summary, the NPU is a key hardware enabler for AI-native 5G-Advanced services. Its 3GPP relevance lies in the specifications that describe AI/ML media processing, where the NPU provides the computational substrate for neural-network-based codecs and enhancements, ultimately improving user experience and network efficiency.

Purpose & Motivation

The NPU exists to meet the growing demand for AI/ML processing in mobile networks and devices, where traditional CPUs and GPUs are often too power-hungry or inefficient for real-time neural network inference. As 5G-Advanced evolves toward AI-native operation, functions such as neural-network-based video coding, AI-enhanced audio processing, and intelligent media adaptation require dedicated hardware that can execute complex models within strict latency and power budgets. The NPU solves this by providing a specialized architecture optimized for tensor operations, enabling on-device AI without excessive battery drain or thermal impact.

Before NPUs became a focus in 3GPP Rel-19, AI/ML media processing was largely implementation-specific or reliant on cloud offloading, which introduced latency and dependency on network coverage. The lack of standardized capability signaling and model management meant that devices and networks could not reliably negotiate AI-based media services. The Rel-19 work in specifications such as 26.847 and 26.927 addresses this gap by defining how AI/ML media services can be supported, including the role of NPUs in UE and network nodes. This enables interoperable, efficient AI-enhanced communication and paves the way for future 6G AI-native designs.