Learning the Sequence of Packing Irregular Objects from Human Demonstrations

André José Freitas Santos

Key information

Authors:

André José Freitas Santos (André José Freitas Santos)

Supervisors:

Atabak Dehban (Atabak Dehban); José Alberto Rosado dos Santos Victor (José Santos Victor)

Published in

11/16/2022

Abstract

Exclusively online supermarkets have been expanding in recent years. Their operation involves packing and shipping millions of orders as efficiently as possible. The automation of the former task, bin packing, has seen slow progress mainly due to the inherent complexity of safely packing irregular objects such as groceries. This task's underlying constraints on object placement and manipulation, and the diverse objects' physical properties make preprogrammed strategies unfeasible. Our approach is to learn directly from expert demonstrations in order to extract implicit task knowledge and strategies. As such, we collect and make available a novel and diverse dataset of bin packing demonstrations by humans in virtual reality. In total, 263 boxes were packed with supermarket-like objects by 43 participants, yielding 4644 object manipulations. This collection is annotated with multiple task parameters and is the most diverse public dataset involving irregular objects. It has the potential to train models that achieve efficient space usage, safe object positioning and to generate human-like behaviors that enhance human-robot trust in a collaborative scenario. We leverage the data in this new dataset to learn a Markov chain to predict the object packing sequence for a given set of objects. The proposed model makes predictions in real-time, only requires a simple and fast training scheme, and accurately captures the strategies that humans use during the bin packing task. Our experimental results show that the model generates sequence predictions that are indistinguishable from human-generated sequences when classified by individuals.