Search for and detect objects with foveal vision systems using Visual-Language Models

December 3, 2025

Deep learning architectures have advanced object detection, but many real-world applications still de- mand a better balance between computational efficiency and detection performance. Visual-Language Models (VLMs) have emerged to enhance generalization and disambiguation capabilities of standard object detection models, through text-image alignment, yet they remain computationally expensive. In- stead of processing the whole field of view with the same resolution, humans select fixation points where the closer region around that point (fovea) is processed with a higher acuity than the periphery. There- fore, this thesis aims to design and evaluate an architecture that integrates foveal vision mechanisms into VLMs. To this end, we adopt three different foveal transformations and integrate them into GLIP, result- ing in the variants GLIP+Laplacian, GLIP+FOVEA, and GLIP+Crop. These models sequentially attend to regions of interest guided by text–image alignment, with an inhibition-of-return mechanism to avoid redundant fixations. The models’ performance was evaluated on the COCO benchmark in comparison with baseline models. Additionally, we assessed cumulative search performance and fixation-sequence similarity against human scanpaths. Results show that foveal processing remains challenging, espe- cially for peripheral detection, and overall efficiency, although GLIP+Crop provides the best trade-off among the developed variants. The proposed methods achieved near-human searching performance, with GLIP+FOVEA and GLIP+Crop detecting over 84% of the target objects while processing only 63% of the total image pixels. Overall, this work demonstrates the feasibility of semantically driven foveal architectures and provides insights toward more efficient and human-like visual perception systems.

Authors in the community:

• 

  S

  Sofia Alexandre Nobre dos Santos

  ist1100088

Supervisors of this institution:

• 

  P

  Plinio Moreno Lopez

  ist31838

• 

  A

  Alexandre José Malheiro Bernardino

  ist13761

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico