Size-based Adaptive Instance Pruning for Refined Segmentation of Cell Nuclei in Histology Images

Abstract

In this paper, a size-based instance pruning approach, which can be used for more accurate segmentation of cell nuclei in histology images and can be adapted to cell types, is presented. In the proposed study, different cell nuclei types in colon histology images were initially segmented semantically using the U-Net medical image segmentation method and the output segmentation maps were produced for each nuclei type. Then, the Watershed algorithm was employed on the segmentation maps and cell nuclei instances were obtained. Finally, instance segments that were below the acceptable nuclei sizes for each nuclei type and were not classified as nuclei instances were eliminated with the adaptive size-based instance pruning approach. In the tests performed on the CoNIC 2022 dataset, it was observed that the adaptive size-based instance pruning approach outperformed the normal segmentation methodology without this approach, and an average value of 0.5090 mPQ was measured.