A model needs
a sense of contact.
Embodied foundation models for manipulation. We combine vision, joint state and contact history to study tool use with unfamiliar objects.
Open question Can a shared representation distinguish a secure grasp from a convincing image of one?