An Overview of Quantization in AV2
Abstract
This paper presents an overview of the quantization techniques used in the AV2 video coding standard. For scalar quantization, an improved design consolidates six lookup tables into a simplified function where the step size doubles with every 24 quant index increments. Support for user-defined quantization matrices can be signaled in the sequence header, enabling segment-level matrix selection. Moreover, advanced quantization methods are described, including a parity-hiding method which can implicitly signal the parity of the DC (index 0,0) coefficient, based on the sum of the AC coefficients, thereby reducing the number of explicitly coded symbols. Additionally, an eight-state trellis-coded quantization method is introduced as a high-performance alternative to scalar quantization. This paper covers the rationale, implementation details, and the resulting coding gain benefits of these methods.
Research Areas
