BDIS: Bayesian Dense Inverse Searching Method for Real-Time Stereo Surgical Image Matching

Song, J; Zhu, Q; Lin, J; Ghaffari, M

BDIS: Bayesian Dense Inverse Searching Method for Real-Time Stereo Surgical Image Matching

Song, J Zhu, Q Lin, J Ghaffari, M

Permalink

Publisher:: Institute of Electrical and Electronics Engineers (IEEE)
Publication Type:: Journal Article
Citation:: IEEE Transactions on Robotics, 2023, 39, (2), pp. 1388-1406
Issue Date:: 2023-04-01

Closed Access

	Filename	Description	Size
	BDIS_Bayesian_Dense_Inverse_Searching_Method_for_Real-Time_Stereo_Surgical_Image_Matching.pdf	Published version	10.14 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Closed Access

This item is closed access and not available.

Full metadata record

Field	Value	Language
dc.contributor.author	Song, J
dc.contributor.author	Zhu, Q
dc.contributor.author	Lin, J
dc.contributor.author	Ghaffari, M
dc.date.accessioned	2024-04-29T10:35:30Z
dc.date.available	2024-04-29T10:35:30Z
dc.date.issued	2023-04-01
dc.identifier.citation	IEEE Transactions on Robotics, 2023, 39, (2), pp. 1388-1406
dc.identifier.issn	1552-3098
dc.identifier.issn	1941-0468
dc.identifier.uri	http://hdl.handle.net/10453/178485
dc.description.abstract	In stereoscope-based minimally invasive surgeries (MISs), dense stereo matching plays an indispensable role in 3-D shape recovery, AR, VR, and navigation tasks. Although numerous deep neural network (DNN) approaches are proposed, the conventional prior-free approaches are still popular in the industry because of the lack of open-source annotated dataset and the limitation of the task-specific pretrained DNNs. Among the prior-free stereo matching algorithms, there is no successful real-time algorithm in none GPU environment for MIS. This article proposes the first CPU-level real-time prior-free stereo matching algorithm for general MIS tasks. We achieve an average 14-17 Hz on 640 × 480 images with a single-core CPU (i5-9400) for surgical images. Meanwhile, it achieves slightly better accuracy than the popular efficient large-scale stereo matching (ELAS) method. The patch-based fast disparity searching algorithm is adopted for the rectified stereo images. A coarse-to-fine Bayesian probability and a spatial Gaussian mixed model were proposed to evaluate the patch probability at different scales. An optional probability density function estimation algorithm was adopted to quantify the prediction variance. Extensive experiments demonstrated the proposed method's capability to handle ambiguities introduced by the textureless surfaces and the photometric inconsistency from the non-Lambertian reflectance and dark illumination. The estimated probability managed to balance the confidences of the patches for stereo images at different scales. It has similar or higher accuracy and fewer outliers than the baseline ELAS in MIS, while it is 4-5 times faster.
dc.language	en
dc.publisher	Institute of Electrical and Electronics Engineers (IEEE)
dc.relation.ispartof	IEEE Transactions on Robotics
dc.relation.isbasedon	10.1109/TRO.2022.3215018
dc.rights	info:eu-repo/semantics/closedAccess
dc.subject	0801 Artificial Intelligence and Image Processing, 0906 Electrical and Electronic Engineering, 0913 Mechanical Engineering
dc.subject.classification	Industrial Engineering & Automation
dc.subject.classification	4007 Control engineering, mechatronics and robotics
dc.title	BDIS: Bayesian Dense Inverse Searching Method for Real-Time Stereo Surgical Image Matching
dc.type	Journal Article
utslib.citation.volume	39
utslib.for	0801 Artificial Intelligence and Image Processing
utslib.for	0906 Electrical and Electronic Engineering
utslib.for	0913 Mechanical Engineering
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	closed_access	*
dc.date.updated	2024-04-29T10:35:24Z
pubs.issue	2
pubs.publication-status	Published
pubs.volume	39
utslib.citation.issue	2

Abstract:

In stereoscope-based minimally invasive surgeries (MISs), dense stereo matching plays an indispensable role in 3-D shape recovery, AR, VR, and navigation tasks. Although numerous deep neural network (DNN) approaches are proposed, the conventional prior-free approaches are still popular in the industry because of the lack of open-source annotated dataset and the limitation of the task-specific pretrained DNNs. Among the prior-free stereo matching algorithms, there is no successful real-time algorithm in none GPU environment for MIS. This article proposes the first CPU-level real-time prior-free stereo matching algorithm for general MIS tasks. We achieve an average 14-17 Hz on 640 × 480 images with a single-core CPU (i5-9400) for surgical images. Meanwhile, it achieves slightly better accuracy than the popular efficient large-scale stereo matching (ELAS) method. The patch-based fast disparity searching algorithm is adopted for the rectified stereo images. A coarse-to-fine Bayesian probability and a spatial Gaussian mixed model were proposed to evaluate the patch probability at different scales. An optional probability density function estimation algorithm was adopted to quantify the prediction variance. Extensive experiments demonstrated the proposed method's capability to handle ambiguities introduced by the textureless surfaces and the photometric inconsistency from the non-Lambertian reflectance and dark illumination. The estimated probability managed to balance the confidences of the patches for stereo images at different scales. It has similar or higher accuracy and fewer outliers than the baseline ELAS in MIS, while it is 4-5 times faster.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/178485