Grasp2Vec: Learning Object Representations from Self-Supervised Grasping

Robotic learning algorithms based on reinforcement, self-supervision,
and imitation can acquire end-to-end controllers from raw sensory
inputs such as images. These end-to-end controllers acquire perception
systems that are tailored to the task, picking up on the cues that are
most useful for the task at hand. However, to learn generalizable
robotic skills, we might prefer more structured image representations,
such as ones encoding the persistence of objects and their identities.
In this paper, we study a specific instance of this problem: acquiring
object representations through autonomous robotic interaction with its
environment.

Our representation learning method is based on object persistence:
when a robot picks up an object and ``subtracts'' it from the scene,
its representation of the scene should change in a predictable way. We
can use this observation to formulate a simple condition that an
object-centric representation should satisfy: the features
corresponding to a scene should be approximately equal to the feature
values for the same scene after an object has been removed, minus the
feature value for that object.

• Machine Perception

• Machine Intelligence