Comparative Analysis of Linear Quadratic Regulator and Reinforcement Learning for UAV Trajectory Tracking

July 4, 2024

The performance of two different controllers is assessed for generalization in the trajectory tracking task of autonomously following a reference trajectory by a fixed wing UAV. The closed form calculation of the gains for the LQR rely heavily on having an accurate linearized model of the plant, and on working in a region closing to linearization operating point. However, real-world applications are usually modelled with non-linear dynamical systems, and might also suffer with unmodeled disturbances, which can degrade the controllers' performance. In that sense this thesis aims to compare the LQR with a more robust and generalizable controller based on RL. The RL agent was found to sacrifice performance in situations where it would spawn close to the reference, being slightly worse than LQR, but managed to excel in situations far away from the reference, being able to eliminate odd behavior presented by the LQR. The agent was also adapted to a more realistic scenario where limitations in actuation and velocity were implemented. Transfer Learning technique was also explored, being the LQR policy transferred to an agent through weights initialization before training began. The results were not encouraging because the inherent tendency of the LQR to decelerate beyond negative velocities had to be unlearned in a first instance, to then achieve a successful policy, which in terms of speed to convergence was equivalent to starting a training session from a randomly initialized agent, with no specific initial policy incorporated whatsoever.

Authors in the community:

• 

  F

  Francisco Serra

  ist198873

Supervisors of this institution:

• 

  R

  Rita Maria Mendes de Almeida Correia da Cunha

  ist31057

mechanical-engineering - Mechanical engineering

eng - English

Instituto Superior Técnico