Navigated endoscopy is generally agreed to be the next generation of interventional or surgical endoscopy. It usually combines pre- and intra-operative imaging information to guide physicians during endoscopic procedures. However, endoscope three-dimensional motion tracking that spatially and temporathrelly synchronizes various sensory information still remains challenging for developing different endoscopic navigation systems. To navigate or track the surgical endoscope, three modalities of sensory information are utilized in endoscopic procedures: (1) preoperative images, i.e., three-dimensional CT images, (2) two-dimensional video sequences from the endoscopic camera, and (3) location measurements, attaching an electromagnetic sensor at the endoscope distal tip for measuring the temporal endoscope movement. In this respect, endoscope tracking and navigation aims to fuse these various modalities information to accurately and robustly locate or fly through the endoscope at any interest of regions. Unfortunately, fusing the multimodal information is still an open issue due to the information incompleteness, e.g., image artifacts, tissue deformation, and sensor output inaccuracy in computer assisted endoscopic interventions.
This thesis work focuses on fusing the multimodal information for accurate and robust endoscope tracking and navigation. A novel framework of multimodal information fusion is proposed to use evolutionary computing for endoscopic navigation systems. Several main contributions of this dissertation are clarified as follows. First, the concept of evolutionary computing was initially introduced to assist minimally invasive endoscopic surgery. Next, this work modified two evolutionary algorithms of particle swarm optimizer and differential evolution and proposed an enhanced particle swarm optimizer (EPSO) and observation-driven adaptive differential evolution (OADE). EPSO can adaptively update evolutionary parameters in accordance with spatial constraints and the current observation. OADE performs a new mutation operation for DE methods by integrating the current observation of sensor measurements and camera images, which can control the perturbation velocity and the direction of each individual during evolution, to enhance the DE performance. Additionally, the improved evolutionary computing algorithms are applicable to computer vision tasks, e.g., object tracking, motion estimation, and stochastic optimization.
The experimental results demonstrate that the proposed evolutionarily computed endoscopic tracking and navigation approaches in this dissertation provide a more accurate and robust endoscopic guidance framework than state-of-the-art methods. Based static phantom data validation, the average guidance accuracy of the EPSO framework was about 3.0 mm, its average position smoothness was 1.0 mm, and its average visual quality was improved to 0.29. By evaluating on a dynamic phantom, the OADE approach reduces the tracking error from 3.96 to 2.89 mm, improves the tracking smoothness from 4.08 to 1.62 mm, and increases the visual quality from 0.707 to 0.741.
In conclusion, the concept of evolutionary computation is a promising strategy to improve endoscopic tracking and navigation for minimally invasive surgery. The validation demonstrated its effectiveness to improve the guidance accuracy, visual quality, and tracking smoothness during endoscopic surgery. Future work includes surgical data validation, real-time processing, and translation to clinical applications.