Implementation of the 4D prediction mode in the JPEG Pleno part 2 reference software

Abstract

A Light Field (LF) is an image modality captured from multiple viewpoints and combined into a single representation, providing visual capabilities that are not available in traditional pho- tography and video modalities. As with conventional two-dimensional images, storing raw LFs is impractical due to the enormous amount of data involved, making the adoption of compres- sion techniques necessary. Part 2 of the JPEG Pleno standard defines two coding modes for LFs: the 4DTM and the 4DPM. While the 4DTM exploits redundancies through the use of a four-dimensional Discrete Cosine Transform (DCT) and block partitioning, the 4DPM employs depth information and camera parameters to predict intermediate views from a subset of ref- erence views. Currently, two reference software implementations are available for the JPEG Pleno standard: the JPLM, which is publicly available but only supports the 4DTM, and the BVBP, which is available only as part of the materials accompanying Part 4 of the JPEG Pleno standard. In this context, the objective of this work was to implement the 4DPM coding mode in the JPLM software, extending the only publicly available reference implementation of JPEG Pleno to support both LF coding modes defined by the standard. This work implemented the complete mandatory 4DPM encoding pipeline in the JPLM software, including configuration handling, codestream generation, and intermediate view prediction. Experimental validation demonstrated that the proposed encoder produces standard-compliant codestreams that can be successfully decoded by the BVBP reference decoder. The generated reconstructions achieved satisfactory visual quality when accurate configuration files and camera geometry parameters were available. However, the experiments also revealed that coding performance is highly sen- sitive to configuration generation and bitrate allocation strategies, and reconstruction artifacts may still occur in some decoded views. Furthermore, the optional sparse-filtering stage defined by the standard was not implemented and is therefore left for future work.