Algavision: object recognition tool for phytoplankton cells in microscopy images

July 24, 2026

Deep learning has become an increasingly powerful tool for automated object recognition and image analysis across diverse imaging domains, including microscopy, while also helping to reduce human error and bias. This project explores the use of deep learning for multi-class phytoplankton cell segmentation in light microscopy images used in ecological research. A novel dataset comprising 11 phytoplankton strains was created to train instance segmentation models. The best-performing model achieved mean AP of 0.782 and was integrated into an image-processing pipeline. To address challenges posed by edge cases, the model’s confidence score calculation was readjusted, resulting in a slight performance improvement (+1.4% F1 score). Compared with an established thresholding-based approach, the developed pipeline demonstrated superior performance (Avg. F1 score: 86.2% vs. 34.3%). To extend applicability beyond the training data, the pipeline was coupled with a zero-shot segmentation model to segment previously unseen cells. By leveraging the region proposal network of the trained model, high-quality segmentation masks were generated for unseen phytoplankton strains and other cells across different microscopy settings (94.4% and 78.9% of ground-truth instances received predicted masks with IoU > 0.8 in two evaluation datasets, respectively). Classification of unknown cells into candidate strains yielded more modest results (F1 scores of 79.8% and 35.1% for three- and five-strain communities, respectively), highlighting the need for improved feature tailoring and clustering methods. A browser-based application was also developed to facilitate practical use of the pipeline and extraction of phytoplankton cell measurement data for further analysis.

Authors in the community:

• 

  S

  Simas Jasiunas

  ist1112099

Supervisors of this institution:

• 

  A

  Andreas Miroslaus Wichert

  ist24622

industrial-biotechnology - Industrial Biotechnology

eng - English

Instituto Superior Técnico