Identifying Objects from Hand Configurations during In-hand Exploration

In this work we use hand configuration and contact points during in-hand object exploration to identify the manipulated objects. Different contact points associated to an object shape can be represented in a latent space and lie on a lower dimensional non-linear manifold in the contact points space which is suitable for modelling and recognition. Associating and learning hand configurations to specific objects by means of Gaussian mixture models, later by identifying the hand configuration during the in-hand object exploration we can generate hypotheses of candidate objects to be identified. This process selects a set of the most probable objects from a database. The accumulated set of contact points (partial volume of the object shape) during the object in-hand exploration is matched to the set selected from the database (most probable candidate objects). Results are presented for human manipulation of objects, but this can also be applied to artificial hands, although we have not addressed the hand control, only the object identification.