End-to-End Policy Learning for Hip Exoskeleton via Reinforcement Learning and Reflex-Based Musculoskeletal Simulation

초록

Lower-limb exoskeletons can reduce human effort during walking; however, conventional controllers often require manual tuning and exhibit limited adaptability across users and environments. This study presents a lightweight sim-to-real reinforcement learning (RL) framework that learns continuous hip-assistive torque policies entirely in simulation and transfers them directly to a wearable hip exoskeleton. A sequence-aware long short-term memory proximal policy optimization (LSTM-PPO) controller is proposed to improve a standard multilayer perceptron PPO (MLP-PPO) baseline. A reflex-driven planar musculoskeletal model is used to generate physiologically realistic gait, while training incorporates domain randomization and sensor-noise injection to enhance robustness. The learned policies were deployed directly on a lightweight hip exoskeleton and evaluated with eight healthy participants on a treadmill, with outdoor walking assessed in a single-participant proof-of-concept. Both controllers reduced net metabolic cost relative to unassisted walking (15.2% for LSTM-PPO and 11.0% for MLP-PPO). The LSTM-based controller produced smoother assistance and demonstrated improved adaptation to variations in speed and terrain. These results indicate that reflex-based musculoskeletal simulation, combined with sequence-aware RL, can yield efficient and transferable assistance, supporting scalable and user-adaptive hip-exoskeleton control. © 2016 IEEE.

키워드