Bipedal locomotion control is a challenging problem in the field of control and robotics. Conventional model-based methods involve designing a gait cycle and path planning for each robot leg followed by designing a controller. However, classical model-based methods force the robot to exactly repeat a designed pattern, it usually requires simplifying assumptions and accurate modelling. Reinforcement learning (RL) methods offer an adaptable, robust approach inspired by the human brain. This study explores recent advances in deep reinforcement learning algorithms and their application to intelligent robot locomotion control. The use of artificial neural networks (ANNs) makes deep RL methods powerful in complex control tasks. We employed the Proximal Policy Optimization (PPO), a model-free deep RL algorithm to train a bipedal walking robot model using the MuJoCo simulator and Walker-2d environment. We attempt an end-to-end approach where the controller performs the whole procedure of getting sensory inputs to producing torque commands by itself without needing any other system. In this research a bipedal robot is trained to walk on different terrains with different slopes. Necessarily the reward function of the environment is modified in order to make incentive for the robot to go along the direction parallel to the ramp direction. In addition, the environments episode termination condition has been changed, allowing the bipedal robot to continue the episode while its height increases or decreases due to walking on sloped terrain. The other reason for this modification is the dependency of the reward function to the episode termination, as robot gets a positive reward if the termination condition is not violated and robot stays alive in each timestep. This research demonstrates the potential of deep reinforcement learning algorithms for robot locomotion control and their ability to adapt to terrains with different slopes. Another experiment is done on a trained robot on a ramp with a certain slope to eva‎luate its robustness for walking on terrains with other slopes. This experiment shows that surprisingly, robot still was able to walk to some extent. Furthermore, we perform some experiment to report torque commands and states such as velocity and position of the robot during the tests.

محمد دانش

سعيد بهبهاني , مهدي كاروان