3D non-rigid SLAM in minimally invasive surgery
- Publication Type:
- Thesis
- Issue Date:
- 2020
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
Aiming at reducing trauma and morbidity associated with large incisions in open surgery, minimally invasive surgery (MIS) has been widely acquired in clinical practice as a powerful tool enabling patients with less pain, shorter hospital stay, and fewer complications. However, MIS narrows the surgeon's field of view which confines visual information when implementing MIS. Therefore, a stereoscope or monocular scope is an essential tool for capturing and transmitting 2D images during the procedure.
Although numbers of special sensors including laser, structured light, time-of-flight cameras have been applied or investigated in MIS, RGB scope is still widely applied in the intro-operative system because it is non-invasive and cheap to be installed. Thus it is an important topic to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages from stereo or monocular scopes. This research aims at proposing innovative robocentric simultaneous localization and mapping (SLAM) algorithm for deformable dense reconstruction of soft-tissue surfaces using a sequence of images obtained from a stereoscope or monocular camera. In this paper, we try to solve the problem by introducing a warping field based on the embedded deformation (ED) nodes which makes full use of the 3D shapes recovered from consecutive pairs of stereo images by deforming the last updated model to the current live model. Our robocentric SLAM system (off-line and tested on stereo videos) can: (1) Incrementally build a live model by progressively fusing new observations with vivid accurate texture. (2) Estimate the deformed shape of the unobserved region with the principle As-Rigid-As-Possible. (3) Perform the dynamic model shape deformation. (4) Estimate the current relative pose between the soft-tissue and the scope.
We further improve and optimize the proposed robocentric deformable SLAM algorithm to MIS-SLAM: a complete real-time large scale robocentric dense deformable SLAM system with stereoscope in MIS based on heterogeneous computing by making full use of CPU and GPU. Idled CPU is used to perform ORB-SLAM for providing robust global pose. Strategies are taken to integrate modules from CPU and GPU. We solve the key problem raised in previous work, that is, fast movement of scope and blurry images make the scope tracking fail. Benefiting from improved localization, MIS-SLAM can achieve large scale scope localizing and dense mapping in real-time. It transforms and deforms the current model and incrementally fuses new observation while keeping the vivid texture. In-vivo experiments conducted on publicly available datasets presented in the form of videos demonstrate the feasibility and practicality of MIS-SLAM for potential clinical purpose.
In MIS-SLAM, however, it remains challenging to keep constant speed in deformation nodes parameter estimation when the model grows larger. In practice, the processing time grows rapidly in accordance with the expansion of the maps. Therefore, we propose an approach to decouple nodes of deformation graph in large scale robocentric dense deformable SLAM and keep the estimation time to be constant. We discover that only partial deformable nodes in the graph are connected to visible points. Based on this principle, the sparsity of the original Hessian matrix is utilized to split parameter estimation into two independent steps. With this new formulation, we achieve faster parameter estimation with amortized computation complexity reduced from 𝑂(𝑛²) to closing 𝑂(1). As a result, the computation cost barely increases as the map keeps growing. By our strategy, the bottleneck of limited computation in estimating deformation field in large scale environment has been overcome. The effectiveness is validated by experiments, featuring large scale deformation scenarios.
In addition to robocentric SLAM, this thesis also aims at developing a general SLAM which estimates the scope poses correctly. An elaborate observability analysis is conducted on the ED graph. We demonstrate and prove that the ED graph widely used in such scenarios is unobservable and leads to multiple solutions unless suitable priors are provided. Example, as well as theoretical prove, are provided to show the ambiguity of ED graph and scope pose. Different from robocentric SLAM, in modelling non-rigid scenario with ED graph, motion priors of the deforming environment is essential to separate robot pose and deforming environment. The conclusion can be extrapolated to any free form deformation formulation. In guaranteeing the observability, this research proposes a preliminary deformable SLAM approach to estimate robot pose in complex environments that exhibits regular motion. A strategy that approximates deformed shape using a linear combination of several previous shapes is proposed to avoid the ambiguity in robot movement and rigid and non-rigid motions of the environment. Fisher information matrix rank analysis is performed to prove the effectiveness. Moreover, the proposed algorithm is validated using Monte Carlo simulations and real experiments. It is demonstrated that the new algorithm significantly outperforms conventional SLAM and ED based SLAM especially in scenarios where there is large deformation.
Please use this identifier to cite or link to this item: