Postdoc position - Learning and graph-based signal processing for video light field compression

The goal of the postdoc will be to develop methods for efficient compression of static and video light fields. Light fields are densely sampled high-dimensional signals containing information about the light rays interacting with the physical objects in the scene. They yield a very rich description of a 3D scene which enables advanced creation of novel images from a single capture [1][2]. However, Light fields constitute very large volumes of highly redundant data, hence the need to design efficient compression algorithms to enable practical use of this new imaging modality.

Although the ultimate goal is to develop novel compression schemes for dynamic light fields (light fields videos), the work will naturally start by developing methods for static light fields and then be extended taking into account the temporal dimension. The work will be at the frontier between signal processing, computer vision, and source coding theory. The candidate will explore methods allowing us to learn local models enabling to best capture the correlation present in the data. In order to do so, concepts of super-pixels and super-rays [3] will be considered as possible supports of local transforms adapted to the data characteristics via learning and/or using graphs connecting correlated pixels in the different spatio-angular dimensions [4]. These methods will then be extended considering scene flow estimation [5] to capture correlation in the 3 dimensions, i.e. temporal in addition to spatial and angular.

Graph-based transforms will similarly considered to best de-correlate the signal along super-rays and motion trajectories. The goal will also be to explore ways to encode the designed representations using coding tools tailored to its statistical properties.

The position is funded by the ERC advanced grant project CLIM: Computational Light Fields Imaging led by Dr Christine Guillemot at INRIA in Rennes, France

References

[1] M. Levoy and P. Hanrahan, Light Field Rendering, ACM Siggraph, pp. 31-42,1996.

[2] R. Ng, “Digital light field photography,” Ph.D. dissertation, Stanford university, 2006.

[3] M. Hog, N. Sabater, C. Guillemot, Super-rays for efficient light field processing, IEEE J. on Selected Topics on Signal Processing, special issue on light fields image processing,IEEEJ-STSP, vol. 11, No. 7, pp.1187–1199, Oct. 2017.

[4] M. Rizkallah, X. Su, T. Maugey, C. Guillemot, Graph-based Transforms for Predictive Light Field Compression based on Super-Pixels, IEEE International Conf. on Acoustics, Speech and Signal Processing, 2018 (accepted).

[5] P. Srinivasan, M. Tao, R. ng, R. Ramamoorthi, Oriented Light-Field Windows for Scene Flow, IEEE-ICCV, Dec. 2015