Flexible-Region Based Adaptive In-Loop Filter for Video Coding

Adaptive loop filter (ALF) for video coding, which is designed to minimize the mean square error between original and reconstructed samples by using Wiener-based filter, has attracted increasing attention for its significant capability in improving coding efficiency. In the second and third Audio Video Coding Standard, i.e., AVS2 and AVS3, ALF is adopted as one of the in-loop filters. In current design, each frame is divided into 16 regions at most and corresponding filter coefficients are then derived and utilized to reconstruct each region. In this paper, a flexible-region based ALF (FRALF) scheme is proposed to improve the adaptability of existing ALF in AVS3, which introduces multiple region partition templates, such as 2×4, 4×4, 4×8 and 8×8. We subsequently propose the filter coefficients merging algorithm to further improve coding efficiency by estimating the distortion level of different partition regions. The proposed FRALF can fully consider the local texture characteristics as well as non-local similarities synthetically. The experimental results show that FRALF outperforms the existing region-based ALF in AVS3 with relatively low complexity increasing.