Vision Transformer for the segmentation of 3D microscopy images

Gonçalo Da Silva Dias Moura de Mesquita

Key information

Authors:

Gonçalo Da Silva Dias Moura de Mesquita (Gonçalo Da Silva Dias Moura de Mesquita)

Supervisors:

Hemaxi Narotamo (Hemaxi Narotamo); Maria Margarida Campos da Silveira (Maria Margarida Campos da Silveira)

Published in

July 26, 2024

Abstract

The proper organization and functioning of blood vessels are vital for keeping human tissues healthy. When blood vessels are disorganized or malfunction, it can lead to various diseases, highlighting the importance of studying vascular formation and remodeling. To effectively study and address these vascular issues, segmentation becomes an essential tool. Segmentation helps in precisely identifying and analyzing the different components and patterns within blood vessels, allowing for better diagnosis and treatment of vascular related diseases. In recent years, Deep Learning methods have been extensively studied for the segmentation of microscopy images to efficiently and accurately quantify and characterize cells, nuclei, and other biological structures, This thesis explores the use of Vision Transformers for the segmentation of three dimensional microscopy images, focusing on the nuclei and Golgi structures in mouse retinas. While Convolutional Neural Networks have been widely used for image analysis, they often struggle to capture the full context of 3D images. ViTs, with their ability to capture both local and global features, offer a promising alternative. This study proposes a Convolutions to Vision Transformer based model, specifically the Fully Convolutional Transformer, and compares its performance with two other models: the 3D U-Net and UNet Transformers. Our dataset consists of 3D fluorescence microscopy images of mouse retinas. Our results demonstrate that the proposed FCT model outperforms the comparative models in terms of segmentation accuracy and generalization. By leveraging the strengths of both convolutional operations and attention mechanisms, the FCT model provides a robust solution for complex 3D image segmentation tasks.