It is very glad to announce that our paper of “Visually Grounding Language Instruction for History-Dependent Manipulation” is accepted to ICRA 2022!
I was almost crying since this paper was rejected for 4 times. But after several revisions, this work could become better and finally find its place for resting :)
Visually Grounding Language Instruction for History-Dependent Manipulation
Hyemin Ahn*, Obin Kwon*, Kyungdo Kim, Jaeyeon Jeong, Howoong Jun, Hongjung Lee, Dongheui Lee, Songhwai Oh. (*These authors are equally contributed to this work.)
Abstract : This paper emphasizes the importance of a robot’s ability to refer to its task history, especially when it executes a series of pick-and-place manipulations by following language instructions given one by one. The advantage of referring to the manipulation history can be categorized into two folds: (1) the language instructions omitting details or using expressions referring to the past can be interpreted, and (2) the visual information of objects occluded by previous manipulations can be inferred. For this, we introduce a history-dependent manipulation task whose objective is to visually ground a series of language instructions for proper pick-and-place manipulations by referring to the past. We also suggest a relevant dataset and model which can be a baseline, and show that our network trained with the proposed dataset can also be applied to the real world based on the CycleGAN.
We will release the code and dataset before the conference with a fine github project page 
