Neural-Guided Projective Factorization using Monocular Depth Estimators

June 29, 2026

The recovery of three-dimensional structure from visual observations remains a fundamental challenge in computer vision. Classical Structure from Motion (SfM) pipelines offer mathematically rigorous reconstructions but are brittle in the presence of textureless surfaces, limited parallax, and long sequences. Conversely, modern end-to-end neural methods provide dense scene understanding but operate as black boxes that discard explicit multi-view geometric constraints. To bridge this gap, this thesis proposes a novel pipeline that integrates dense monocular depth estimators (MDE) directly into a classical projective factorization framework. To address the inherent scale and offset ambiguities of MDE outputs, a joint optimization problem is formulated and solved via a coordinate-descent scheme. This approach simultaneously calibrates per-frame affine depth parameters, recovers the missing entries of highly sparse feature tracking matrices using Alternating Least Squares (ALS) completion, and enforces the strict low-rank constraints of the projective camera model. Evaluations on synthetic datasets and the real-world ETH3D benchmark demonstrate that the proposed method effectively recovers globally coherent 3D structures and camera poses. Notably, the pipeline provides robust reconstructions in challenging configurations where classical incremental methods like COLMAP typically struggle—such as near-planar scenes and degenerate pure-rotation trajectories—while maintaining competitive runtime performance.

Authors in the community:

• 

  M

  Manuel Luís da Fonseca Ferreira

  ist196429

Supervisors of this institution:

• 

  M

  Manuel Marques

  ist149415

• 

  A

  Alexandra Moutinho

  ist13959

mechanical-engineering - Mechanical engineering

eng - English

Instituto Superior Técnico