Vision based collaborative task planning and execution with a UR3 robotic manipulator

December 6, 2021

This thesis aimed at developing a collaborative interface between an automation bench and an operator. It is intended for the system to be able to collect and deliver objects from the hands of the operator. An Intel D435 depth camera an a UR3 collaborative robot were used. The algorithm developed employs the colour images and depth information from the camera to generate a path for the robot to follow. The algorithm was able to correctly segment the hands of the operator and the objects to be carried in normal operation conditions. The robot followed correctly the paths generated, except when sharp changes in direction occur or when transitioning from control on joint space to control in tool space. In these situations, the blend radii selected was not respected. The discrepancy between the positions reported by the camera and the robot were under 15 mm in the XY plane and 40 mm in terms of depth. These errors are due to the camera and the transformation between camera coordinates and robot coordinates. Collaboration was achieved since the algorithm reacted in real time to demands from the operator and the tool deviation observed was acceptable for the task at hand.

Authors in the community:

• 

  T

  Tomás Moreira Furtado Carvalho Fernandes

  ist180911

Supervisors of this institution:

• 

  M

  Mário António Da Silva Neves Ramalho

  ist12229

mechanical-engineering - Mechanical engineering

por - Portuguese

Instituto Superior Técnico