Abstract
An important step in the construction of novel objects is the ability to recognise combinations of raw materials which are likely to be useful. We aim to exploit the intuition that the visual characteristics of candidate raw materials provide useful cues to their potential combinations. Toward this end, we present a Siamese neural network based model that is able to recognise unseen raw materials present in objects given a list of candidate material images. We demonstrate the utility and efficacy of our model within two domains. The first being a proof-of-concept within Minecraft where we predict the combinations of objects that will result in a target object. The second, more realistic domain, uses the ShapeNet 3D model dataset where we attempt to recover the materials present in a model. We empirically demonstrate that our model is able to learn from a subset of object material pairs and generalise to unseen objects, materials, texture packs. Under these conditions of high visual variation, we show that our model outperforms chance and baseline methods.
Benjamin Rosman
Lab Director
I am a Professor in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand in Johannesburg. I work in robotics, artificial intelligence, decision theory and machine learning.
Pravesh Ranchod
Lecturer
I am a Lecturer in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand