Guillermo Castillo Wins the Best Paper Award at the 2023 ICRA RAP4Robots Workshop

May 29, 2023 2 min read

Our paper titled, “Effective State Representation for Learning Hierarchical Robust Bipedal Locomotion”, won the Best Paper Award at the 2023 ICRA Workshop on Effective Representations, Abstractions, and Priors for Robot Learning (RAP4Robots), held in London, UK.

Big congratulations to Guillermo A Castillo! We also greatly appreciate the acknowledgement to our work from the workshop programming committee.

Overall structure of the proposed learning-based framework. The HL policy maps a reduced-order latent state to task space trajectories that are tracked by the LL policy.

Abstract

This paper introduces an innovative hierarchical strategy for bipedal walking that merges a data-driven effective state representation of the system with a Reinforcement Learning (RL)-based high-level (HL) policy for real-time generation of task space commands and a model-based low-level (LL) controller for task space trajectory tracking. Distinct from conventional end-to-end learning techniques, our HL policy leverages insights from reduced-order template models to thoughtfully design the observation and action spaces of the Markov Decision Process (MDP). This efficient design establishes a meaningful connection between a low-dimensional latent state that effectively encapsulates the intricate dynamics of bipedal walking and a collection of task space outputs that shape the robot’s walking gait. The HL policy remains independent of the task space LL controller, enhancing the design’s flexibility and adaptability to other bipedal robots. This hierarchical design shows superior performance and robustness compared to a traditional template model-inspired learning-based approach for bipedal walking. The trained policy inherently learns human-like locomotion behaviors and effectively tracks an extensive range of walking speeds while maintaining the walking gait’s robustness and stability, even in challenging conditions.