Here we focus on learning abstract representations, which we believe are an important component if we are ever to apply reinforcement learning to the real world. In particular, we focus on skill- and symbol-discovery, as well as the interplay between the two.