Task Offloading Optimization for UAV based Edge Computing using Reinforcement Learning

Vítor Daniel de Rebelo Maçãs

Key information

Authors:

Vítor Daniel de Rebelo Maçãs (Vítor Daniel de Rebelo Maçãs)

Supervisors:

António Manuel Raminhos Cordeiro Grilo (António Manuel Raminhos Cordeiro Grilo)

Published in

November 17, 2025

Abstract

The integration of Unmanned Aerial Vehicle (UAV)s as mobile Mobile Edge Computing (MEC) servers presents a promising solution to the latency and bandwidth challenges of modern Internet of Things (IoT) applications. However, optimizing their dynamic trajectories, limited resources, and connectivity in real- world scenarios remains a significant challenge. This research addresses these gaps by proposing a Deep Reinforcement Learning (DRL) solution within a custom, partially observable simulation environ- ment. We develop two distinct Proximal Policy Optimization (PPO) agents: a Heuristic-Assisted agent, which receives a state vector presorted by task urgency, and a Permutation-Invariant agent, which must learn to identify Ground Devices (GD) features from an unordered state. The simulation environment in- troduces a realistic wireless communication model and a novel staleness penalty to enforce exploration. Our results demonstrate that the Heuristic-Assisted agent successfully learns a sophisticated and robust ”patrol” policy, balancing exploration with exploitation to achieve a high Task Completion Ratio (TCR). Conversely, the more ambitious Permutation-Invariant agent fails to converge, adopting a suboptimal policy that prioritizes staleness minimization above all else. This failure highlights the critical difficulty of autonomous feature extraction and reward balancing in such a complex state space. We conclude that while the partially observable model and reward components are effective, future work should focus on recurrent architectures, such as Long Short-Term Memory (LSTM)s, and integrating connection choice into the agent’s action space to overcome these limitations.