Composition

Last updated on 6 Jun, 2023

In this line of work, we are interested in techniques that allow an agent to leverage past knowledge to solve new tasks quickly. In particular, we focus on how agents can acquire behaviours that can be combined to generate interesting, novel abilities. One particular focus is on applying Boolean operators to learned behaviours to generate provably optimal solutions to new tasks. Not only are these approaches human-understandable, but they result in a combinatorial explosion in an agent’s abilities, which is key to tackling the multitask or lifelong learning setting.

Reinforcement Learning

Publications

Optimal Task Generalisation in Cooperative Multi-Agent Reinforcement Learning
While task generalisation is widely studied in the context of single-agent reinforcement learning (RL), little research exists in the …
PDF Cite Project
Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning
It is desirable for an agent to be able to solve a rich variety of problems that can be specified through language in the same …
PDF Cite Project
Counting Reward Automata: Sample Efficient Reinforcement Learning Through the Exploitation of Reward Function Structure
We present counting reward automata—a finite state machine variant capable of modelling any reward function expressible as a …
PDF Cite Project
End-to-End Learning to Follow Language Instructions with Compositional Policies
We develop an end-to-end model for learning to follow language instructions with compositional policies. Our model combines large …
PDF Cite Project
Skill Machines: Temporal Logic Composition in Reinforcement Learning
A major challenge in reinforcement learning is specifying tasks in a manner that is both interpretable and verifiable. One common …
PDF Cite Project
Facilitating Safe Sim-to-Real through Simulator Abstraction and Zero-shot Task Composition
Simulators are a fundamental part of training robots to solve complex control and navigation tasks. This is due to the speed and safety …
PDF Cite Project Video
Skill Machines: Temporal Logic Composition in Reinforcement Learning
A major challenge in reinforcement learning is specifying tasks in a manner that is both interpretable and verifiable. One common …
PDF Cite Project Video
World Value Functions: Knowledge Representation for Learning and Planning
We propose world value functions (WVFs), a type of goaloriented general value function that represents how to solve not just a given …
PDF Cite Project
World Value Functions: Knowledge Representation for Multitask Reinforcement Learning
An open problem in artificial intelligence is how to learn and represent knowledge that is sufficient for a general agent that needs to …
PDF Cite Project
Generalisation in Lifelong Reinforcement Learning through Logical Composition
We leverage logical composition in reinforcement learning to create a framework that enables an agent to autonomously determine whether …
PDF Cite Project
Generalisation in Lifelong Reinforcement Learning through Logical Composition
We leverage logical composition in reinforcement learning to create a framework that enables an agent to autonomously determine whether …
PDF Cite Project
Learning to Follow Language Instructions with Compositional Policies
We propose a framework that learns to execute natural language instructions in an environment consisting of goal-reaching tasks that …
PDF Cite Project
A Boolean Task Algebra for Reinforcement Learning
The ability to compose learned skills to solve new tasks is an important property for lifelong-learning agents. In this work we …
PDF Cite Project
Logical Composition for Lifelong Reinforcement Learning
The ability to produce novel behaviours from existing skills is an important property of lifelong-learning agents. We build on recent …
PDF Cite Project
A Boolean Task Algebra for Reinforcement Learning
We propose a framework for defining a Boolean algebra over the space of tasks. This allows us to formulate new tasks in terms of the …
PDF Cite Project
Composing Value Functions in Reinforcement Learning
An important property for lifelong-learning agents is the ability to combine existing skills to solve new unseen tasks. In general, …
PDF Cite Project Supplementary Material
Will it Blend? Composing Value Functions in Reinforcement Learning
An important property for lifelong-learning agents is the ability to combine existing skills to solve unseen tasks. In general, …
PDF Cite Project
Hierarchical Subtask Discovery with Non-negative Matrix Factorization
Hierarchical reinforcement learning methods offer a powerful means of planning flexible behavior in complicated domains. However, …
PDF Cite Project
Hierarchical Subtask Discovery with Non-negative Matrix Factorization
Hierarchical reinforcement learning methods offer a powerful means of planning flexible behavior in complicated domains. However, …
PDF Cite Project
Hierarchy Through Composition with Multitask LMDPs
Hierarchical architectures are critical to the scalability of reinforcement learning methods. Most current hierarchical frameworks …
PDF Cite Project Supplementary Material