Automata guided hierarchical reinforcement learning for zero-shot skill composition

Files
1711.00129v1.pdf(912.52 KB)
Published version
Date
2017
DOI
Authors
Li, Xiao
Ma, Yao
Belta, Calin
Version
OA Version
Citation
Xiao Li, Yao Ma, Calin Belta. 2017. "Automata Guided Hierarchical Reinforcement Learning for Zero-shot Skill Composition.." CoRR, Volume abs/1711.00129,
Abstract
An obstacle that prevents the wide adoption of (deep) reinforcement learning (RL) in control systems is its need for a large amount of interactions with the environment in order to master a skill. The learned skill usually generalizes poorly across domains and re-training is often necessary when presented with a new task. We present a framework that combines methods in formal methods with hierarchical reinforcement learning (HRL). The set of techniques we provide allows for convenient specification of tasks with complex logic, learn hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards using any RL methods and is able to construct new skills from existing ones without additional learning. We evaluate the proposed methods in a simple grid world simulation as well as simulation on a Baxter robot.
Description
License