3D Reconstruction of Colon Structures and Textures from Colonoscopic Videos

Zhang, Shuai

3D Reconstruction of Colon Structures and Textures from Colonoscopic Videos

Zhang, Shuai

Permalink

Publication Type:: Thesis
Issue Date:: 2023

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download thesisAdobe PDF (19.64 MB)

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Zhang, Shuai
dc.date.accessioned	2023-11-27T01:29:48Z
dc.date.available	2023-11-27T01:29:48Z
dc.date.issued	2023
dc.identifier.uri	http://hdl.handle.net/10453/173574
dc.description	University of Technology Sydney. Faculty of Engineering and Information Technology.	en_US.UTF-8
dc.description.abstract	Colonoscopy is considered the most effective method for detecting and removing precancerous polyps in the human colon. This procedure uses an endoscope to examine the internal surface of the entire colon. However, during a standard colonoscopy, it can be challenging for the endoscopist to ensure that the entire colon internal surface is inspected from the colon screening video, which can result in missed polyps and adenomas in uninspected regions. If a 3D map of the colon internal surface with detailed textures can be reconstructed during the colonoscopy procedure, the following two main potential benefits can be achieved: i) uninspected regions can be shown on this map and the endoscopist can navigate the endoscope to these missing regions to ensure more colon surfaces are inspected; ii) the detailed textures on the reconstructed map can help the endoscopist to inspect abnormalities offline. In this dissertation, we present three works for reconstructing 3D colon maps from colonoscopic videos. Meanwhile, we introduce a colonoscopy simulator developed in Unity that can simulate the procedures of colonoscopy, different levels of colonic surface deformation, and generate synthetic colonoscopy datasets in different scenarios for the development and validation of colon reconstruction algorithms. Furthermore, to foster research in this field, the colonoscopy simulator and source code are made publicly available. The first work presents a framework for 3D reconstruction of the colonic surface using stereo colonoscopic images. To improve the practicability of the first proposed framework, in the second work, we present a framework that can recover the 3D shape of deformable colon structures with textures from monocular colonoscopic images and a corresponding pre-operative CT-segmented colon mesh model. The third work is significantly differs from the previous two works, which require pair-wise photometric correspondences and dense geometric correspondences, posing a great challenge for low-textured colonoscopic images. In the third work, we formulate the textured colon reconstruction problem as a bundle adjustment problem where all the camera poses and the intensities of mesh model vertices are jointly optimized by maximizing the photometric consistency between the pre-operative CT-segmented colon mesh model and multiple views of colonoscopic images. Overall, the three frameworks proposed in this thesis represent a notable advancement in the field of 3D colonic surface reconstruction, using colonoscopic images and a pre-operative CT-segmented colon mesh model. These frameworks undergo validation through rigorous testing with simulation, phantom, and in-vivo datasets, demonstrating their feasibility, accuracy, and practicality.	en_US.UTF-8
dc.format	Thesis (PhD)
dc.language.iso	en_US	en_US.UTF-8
dc.relation	https://opus.lib.uts.edu.au/bitstream/10453/173574/1/thesis.pdf
dc.rights	info:eu-repo/semantics/openAccess
dc.rights	The author owns the copyright in this thesis including all reproduction and reuse rights for the work. The work may not be altered without the permission of the copyright owner. Attribution is essential when quoting or paraphrasing from this thesis.
dc.rights	© 2023 Shuai Zhang
dc.rights	au.edu.uts.lib/cph
dc.title	3D Reconstruction of Colon Structures and Textures from Colonoscopic Videos	en_US.UTF-8
dc.type	Thesis
utslib.copyright.status	open_access	*

Abstract:

Colonoscopy is considered the most effective method for detecting and removing precancerous polyps in the human colon. This procedure uses an endoscope to examine the internal surface of the entire colon. However, during a standard colonoscopy, it can be challenging for the endoscopist to ensure that the entire colon internal surface is inspected from the colon screening video, which can result in missed polyps and adenomas in uninspected regions. If a 3D map of the colon internal surface with detailed textures can be reconstructed during the colonoscopy procedure, the following two main potential benefits can be achieved: i) uninspected regions can be shown on this map and the endoscopist can navigate the endoscope to these missing regions to ensure more colon surfaces are inspected; ii) the detailed textures on the reconstructed map can help the endoscopist to inspect abnormalities offline. In this dissertation, we present three works for reconstructing 3D colon maps from colonoscopic videos. Meanwhile, we introduce a colonoscopy simulator developed in Unity that can simulate the procedures of colonoscopy, different levels of colonic surface deformation, and generate synthetic colonoscopy datasets in different scenarios for the development and validation of colon reconstruction algorithms. Furthermore, to foster research in this field, the colonoscopy simulator and source code are made publicly available. The first work presents a framework for 3D reconstruction of the colonic surface using stereo colonoscopic images. To improve the practicability of the first proposed framework, in the second work, we present a framework that can recover the 3D shape of deformable colon structures with textures from monocular colonoscopic images and a corresponding pre-operative CT-segmented colon mesh model. The third work is significantly differs from the previous two works, which require pair-wise photometric correspondences and dense geometric correspondences, posing a great challenge for low-textured colonoscopic images. In the third work, we formulate the textured colon reconstruction problem as a bundle adjustment problem where all the camera poses and the intensities of mesh model vertices are jointly optimized by maximizing the photometric consistency between the pre-operative CT-segmented colon mesh model and multiple views of colonoscopic images. Overall, the three frameworks proposed in this thesis represent a notable advancement in the field of 3D colonic surface reconstruction, using colonoscopic images and a pre-operative CT-segmented colon mesh model. These frameworks undergo validation through rigorous testing with simulation, phantom, and in-vivo datasets, demonstrating their feasibility, accuracy, and practicality.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/173574