Master's Thesis

UAV Detection and Tracking with Deep Learning: Visible-Infrared Fusion and Frame Rate Analysis

João Manuel Brás Lopes Ferreira — 2025

Key information

Authors:

João Manuel Brás Lopes Ferreira (João Manuel Brás Lopes Ferreira)

Supervisors:

Afzal Suleman (Afzal Suleman); Alexandra Bento Moutinho (Alexandra Moutinho)

Published in

June 30, 2025

Abstract

This thesis presents the development of a framework for the detection and tracking of unmanned aerial vehicles. For the detection task, a dataset comprising spatial and temporally aligned visible and infrared images was used to train and evaluate YOLO detectors independently and through three fusion strategies: pixel-level fusion, decision-level fusion, and a hybrid approach. Pixel-level fusion methods included classical techniques, principal component analysis, and convolutional neural network approaches. For the decision-level fusion, a comprehensive threshold study was conducted to understand how they affect performance metrics and find their optimal values. The hybrid approach combines both strategies by feeding pixel-level fused images into the decision-level fusion framework. Among these, the pixel-level fusion approach offered the best trade-off between accuracy and computational efficiency. It achieved a precision of 89.8\%, recall of 95.5\%, F2-score of 94.3\%, mAP@0.5 of 95.8\%, and mAP@0.5:0.95 of 72.1\%. For the tracking component, ten videos were processed at different frame rates. The YOLO detector was integrated with the ByteTrack tracker to measure metrics such as ID switches and Multi-Object Tracking Accuracy, offering valuable insights into the trade-offs between frame rate and tracking performance.

Publication details

Authors in the community:

Supervisors of this institution:

Fields of Science and Technology (FOS)

mechanical-engineering - Mechanical engineering

Publication language (ISO code)

eng - English

Rights type:

Embargo lifted

Date available:

April 27, 2026

Institution name

Instituto Superior Técnico