Pixel-accurate Segmentation of Surgical Tools based on Bounding Box Annotations

Detection and segmentation of surgical instruments is an important problem for the laparoscopic surgery. Accurate pixel-wise instrument segmentation used as an intermediate task for the development of computer-assisted surgery systems, such as pose estimation, surgical phase estimation, enhanced image fusion, video retrieval and others. In this paper we describe our deep learning-based approach for instrument segmentation, which addresses the binary segmentation problem, where every pixel in an image is labeled as an instrument or background. Our approach relies on weak annotations provided as bounding boxes of the instruments, which is much faster and cheaper to obtain than a dense pixel-level annotation. To improve the accuracy even further we propose a novel approach to generate synthetic training images. Our approach achieves state-of-the-art results, outperforming previously proposed methods for automatic instrument segmentation, based on weak annotations only.