Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction

Liu, D; Mao, Y; Huang, Y; Cao, L; Wang, Y; Fang, Y

Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction

Liu, D Mao, Y Huang, Y

Cao, L Wang, Y Fang, Y

Permalink

Publisher:: Elsevier
Publication Type:: Journal Article
Citation:: Signal Processing: Image Communication, 2023, 119, pp. 117031
Issue Date:: 2023-11-01

Closed Access

	Filename	Description	Size
	1-s2.0-S0923596523001133-main.pdf	Published version	5.45 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Liu, D
dc.contributor.author	Mao, Y
dc.contributor.author	Huang, Y https://orcid.org/0000-0002-1363-5318
dc.contributor.author	Cao, L
dc.contributor.author	Wang, Y
dc.contributor.author	Fang, Y
dc.date.accessioned	2024-06-21T01:19:47Z
dc.date.available	2024-06-21T01:19:47Z
dc.date.issued	2023-11-01
dc.identifier.citation	Signal Processing: Image Communication, 2023, 119, pp. 117031
dc.identifier.issn	0923-5965
dc.identifier.uri	http://hdl.handle.net/10453/179604
dc.description.abstract	Light Field (LF) imaging can record both the intensities and directions of light rays in a single exposure, which has received extensive attentions. However, the limited angular resolution becomes the primary bottleneck for the wide-spread applications of LF imaging. To this end, this paper proposes a novel optical flow-assisted multi-level fusion network for LF angular reconstruction. In our method, we propose to infer the multi-angular optical flows to explore long-range dependency of LF sub-aperture images (SAIs) for high-quality angular reconstruction. By aligning the SAIs in multi-angular directions, the geometric consistency of reconstructed dense LF can be preserved. Moreover, a multi-level fusion framework for LF angular reconstruction is introduced, which consists of two stages, namely texture-optical flow feature fusion and parallax structure-information fusion. The former firstly extracts the texture and optical flow features from the reconstructed coarse LF and then fuses these two features by using the proposed texture-optical flow fusion-block. The latter further blends the LF parallax structure information with the fused texture and optical flow features using the proposed parallax structure-information fusion network. Comprehensive experiments on both real-world and synthetic LF scenes demonstrate the superiority of the proposed method for reconstructing high-quality dense LF. Moreover, practical application on depth estimation also validates that our method can recover more texture details, particularly for some occlusion regions.
dc.language	en
dc.publisher	Elsevier
dc.relation.ispartof	Signal Processing: Image Communication
dc.relation.isbasedon	10.1016/j.image.2023.117031
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0906 Electrical and Electronic Engineering
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	4006 Communications engineering
dc.subject.classification	4603 Computer vision and multimedia computation
dc.title	Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction
dc.type	Journal Article
utslib.citation.volume	119
utslib.for	0906 Electrical and Electronic Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	closed_access	*
dc.date.updated	2024-06-21T01:19:45Z
pubs.publication-status	Published
pubs.volume	119

Abstract:

Light Field (LF) imaging can record both the intensities and directions of light rays in a single exposure, which has received extensive attentions. However, the limited angular resolution becomes the primary bottleneck for the wide-spread applications of LF imaging. To this end, this paper proposes a novel optical flow-assisted multi-level fusion network for LF angular reconstruction. In our method, we propose to infer the multi-angular optical flows to explore long-range dependency of LF sub-aperture images (SAIs) for high-quality angular reconstruction. By aligning the SAIs in multi-angular directions, the geometric consistency of reconstructed dense LF can be preserved. Moreover, a multi-level fusion framework for LF angular reconstruction is introduced, which consists of two stages, namely texture-optical flow feature fusion and parallax structure-information fusion. The former firstly extracts the texture and optical flow features from the reconstructed coarse LF and then fuses these two features by using the proposed texture-optical flow fusion-block. The latter further blends the LF parallax structure information with the fused texture and optical flow features using the proposed parallax structure-information fusion network. Comprehensive experiments on both real-world and synthetic LF scenes demonstrate the superiority of the proposed method for reconstructing high-quality dense LF. Moreover, practical application on depth estimation also validates that our method can recover more texture details, particularly for some occlusion regions.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/179604