Fast Transform Type Selection Using Conditional Laplace Distribution Based Rate Estimation

Selecting among multiple transform kernels to code prediction residuals are widely used for better compression efficiency. Conventionally, the encoder performs trials of each transform to estimate the rate-distortion (R-D) cost. However such an exhaustive approach suffers from a significant increase of complexity due to the excessive trials. In this paper, a novel rate estimation approach is proposed to by-pass the entropy coding process for each transform type using the conditional Laplace distribution model. The proposed method estimates the Laplace distribution parameter by the context inferred by the quantization level and finds the expected rate of the coefficient for transform type selection. Furthermore, a greedy search algorithm for separable transforms is also presented to further accelerate the process. Experiment results show that transform type selection using the proposed rate estimation method achieves high accuracy at lower complexity.

• Machine Perception

• Algorithms and Theory