An autonomous area coverage method for endoscopic capsule robots

Abstract

Endoscopic examination with actively steerable capsule robots is an emerging technology for the diagnosis of various cancer types and many other gastrointestinal diseases such as Crohn's disease, ulcerative colitis, and hemorrhage. Ensuring a comprehensive screening with these robotic capsules is of paramount importance in the detection of such critical diseases. In this work, we propose a novel autonomous area coverage method for active capsule endoscopes, which aims to maximize the amount of monitored area in the human gastrointestinal tract in a time-efficient manner. We introduce a simulation environment and train a deep reinforcement learning model that learns to autonomously navigate magnetically-driven capsule robots for the area coverage task based on visual feedback provided by an on-board monocular camera. Under both physically and visually realistic capsule endoscopy circumstances our method performs successful reasoning and outperforms baseline coverage path planning approaches on human stomach organ models. Besides, we develop both localization and depth estimation methods for capsule robots by jointly training multiple convolutional neural networks in a self-supervised fashion through utilizing large-scale synthetic endoscopy data recorded in the simulation environment. Numerical assessments of the pose estimation network are presented in comparison with similar studies and the depth estimation method is qualitatively assessed on real endoscopy images and dense surface reconstruction task. Results demonstrate that the proposed learning-based coverage path planning approach in company with the monocular localization and surface reconstruction methods have the potential to become key software elements of the current and next-generation capsule robots and to pave the way for the development of future endoscopic systems.