Paper on Deep RL for Valkyrie accepted to Humanoids 2018

Chuanyu Yang, Kai Yuan, Wolfgang Merkt, Taku Komura, Sethu Vijayakumar, Zhibin Li. “Learning Whole-Body Motor Skills for Humanoids”, Proc. IEEE Intl. Conf. on Humanoid Robots (Humanoids 2018), Beijing (2018).

Publisher’s link – DOI: 10.1109/HUMANOIDS.2018.8625045

Abstract

This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots. The advantage over traditional methods is the integration of high-level planner and feedback control all in one single coherent policy network, which is generic for learning versatile balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned quickly by many state-of-the-art learning algorithms. By comparing our learned results to studies of preprogrammed, special-purpose controllers in the literature, self-learned skills are comparable in terms of disturbance rejection but with additional advantages of producing a wide range of adaptive, versatile and robust behaviors.

BibTex

@INPROCEEDINGS{yang2018learning,
author={C. {Yang} and K. {Yuan} and W. {Merkt} and T. {Komura} and S. {Vijayakumar} and Z. {Li}},
booktitle={2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)},
title={Learning Whole-Body Motor Skills for Humanoids},
year={2018},
volume={},
number={},
pages={270-276},
keywords={Humanoid robots;Aerospace electronics;Collision avoidance;PD control;Reinforcement learning},
doi={10.1109/HUMANOIDS.2018.8625045},
ISSN={2164-0580},
month={Nov},}