Semantic-Based Active Perception for Humanoid Visual Tasks: Scene Exploration and Visual Search in Foveal Scenes using Deep Object Detection Models

11/30/2023

In this work, it is studied how well a semantic-based foveal active perception model is able to complete visual tasks that are regularly performed by humans, namely, scene exploration and visual search. To this end, the accuracy of the semantic-based model is compared with with the accuracy of a traditional saliency-based model, derived from past developments in the fields of neurology and psychology, that attempt to reproduce aspects of humanoid visual cognition. Regarding the scene exploration task, the semantic-based approach convincingly outperforms the traditional saliency-based model, when it comes to accurately mapping the semantic content contained by the visual field. In light of the visual search experiments, it was concluded that the semantic-based predictive approach significantly outperforms the saliency-based model, as well as a random gaze selection algorithm, both in accuracy and relative computational cost. The latter results were obtained while searching for instances of a target class in a visual field containing multiple distractors.

Authors in the community:

• 

  J

  João Miguel Barradas Luzio

  ist193096

Supervisors of this institution:

• 

  P

  Plinio Moreno Lopez

  ist31838

• 

  A

  Alexandre José Malheiro Bernardino

  ist13761

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

Keywords

• Active Perception
• Foveal Vision
• Object Detection
• Scene Exploration
• Visual Search

eng - English

Instituto Superior Técnico