Leveraging Multi-Object Tracking in Vision-based Target Following for Unmanned Aerial Vehicles

Diogo Costa Ferreira

Key information

Authors:

Diogo Costa Ferreira (Diogo Costa Ferreira)

Supervisors:

Meysam Basiri (Meysam Basiri)

Published in

07/04/2024

Abstract

This thesis presents an autonomous vision-based mobile target tracking and following system designed for Unmanned Aerial Vehicles (UAVs) leveraging multi-target information. It explores the research gap in applying the most recent Multi-Object Tracking (MOT) methods in target following scenarios over traditional Single-Object Tracking (SOT) algorithms. The system integrates the real-time object detection model, You Only Look Once (YOLO)v8, with the MOT algorithms BoT-SORT and ByteTrack, extracting multi-target information. The system leverages this multi-target information to improve redetection capabilities, addressing key challenges such as target miss-identifications (ID changes), and partial and full occlusions in dynamic environments. A depth sensing module is incorporated to enhance distance estimation when feasible. When depth information is not available, the target size in the image is used instead. A 3D flight control system is proposed for target following, capable of reacting to changes in target speed and direction while maintaining line-of-sight. The system is initially tested in simulation and then deployed in real-world scenarios within a UAV platform assembled from scratch for this thesis. Results show precise target tracking and following, resilient to partial and full occlusions in dynamic environments, effectively distinguishing the followed target from bystanders. A comparison between the BoT-SORT and ByteTrack trackers reveals a trade-off between computational efficiency and tracking precision, with a preference for BoT-SORT due to its substantial decrease in ID changes in target tracking. In overcoming the presented challenges, this thesis enables new practical applications in the field of vision-based target following from UAVs leveraging multi-target information.