Nonparametric Bayesian Reward Segmentation for Skill Discovery Using Inverse Reinforcement Learning

Pravesh Ranchod, Benjamin Rosman, George Konidaris
September 2015
PDF Cite Project

Abstract

We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed to be optimizing. The skill boundaries and the number of skills making up each demonstration are unknown. We use a Bayesian nonparametric approach to propose skill segmentations and maximum entropy inverse reinforcement learning to infer reward functions from the segments. This method produces a set of Markov Decision Processes (MDPs) that best describe the input trajectories. We evaluate this approach in a car driving domain and a simulated quadcopter obstacle course, showing that it is able to recover demonstrated skills more effectively than existing methods.

Type
Conference paper
Publication
IEEE/RSJ International Conference on Intelligent Robots and Systems
Reinforcement Learning Inverse Reinforcement Learning Learning from Demonstration
Pravesh Ranchod
Pravesh Ranchod
Lecturer

I am a Lecturer in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand

Benjamin Rosman
Benjamin Rosman
Lab Director

I am a Professor in the School of Computer Science and Applied Mathematics at the University of the Witwatersrand in Johannesburg. I work in robotics, artificial intelligence, decision theory and machine learning.