Segmentation of Maritime Vehicles from Video Sequences

Maximilian Vieweg

Key information

Authors:

Supervisors:

Maria Margarida Campos da Silveira (Maria Margarida Campos da Silveira); Alexandre José Malheiro Bernardino (Alexandre José Malheiro Bernardino)

Published in

July 24, 2024

Abstract

The monitoring of maritime vessels is of great importance due to the significance of shipping in global trade. While shore-based systems, e.g. coastal radar or imaging, are effective for static scenarios, such as for ports, sensor-carrying vehicles, e.g. UAVs or aerial vehicles, are of great value in off-shore scenarios. The use of video sequences for segmentation on these mobile sensor-carrying platforms is promising given the advantages in power, weight, and space of RGB cameras. The variation in imaging conditions, due to weather, sun, glare, waves, and foreshortening, makes this a nontrivial problem. Some of these conditions depend on the motion of the camera. This suggests that the inclusion of camera pose information could aid in improving the performance of computer vision methods. In this master thesis, three methods for including camera pose information to enhance video segmentation quality are investigated. First, a transformer-based model is extended to estimate camera motion parameters in addition to video segmentation masks. Next, two 3D U-Net networks were modified to predict either specific camera pose parameters or the motion field. After extensive experimentation and statistical analysis, we were able to demonstrate that knowledge of camera pose and motion does not improve the segmentation quality of the model when including them as additional tasks during training. Experimentation with different representations of camera movement, different loss functions, and network types showed that there were no statistically significant improvements during segmentation in a multitask learning framework.