يادگيري تقليدي مهارت توسط ربات انسان‌نما و بهبود عملكرد آن با مبدل‌هاي بصري

Imitation learning is an effective approach for teaching complex skills to humanoid robots through observing the visual demonstrations of an expert agent. However, classical control methods require the analytical solution of complex kinematic equations, an‎d stan‎dard reinforcement learning approaches also need extensive interaction with the environment; challenges that make their application in real-world scenarios time-consuming an‎d computationally expensive. In this thesis, to enable the easy transfer of skills to other robots, a novel two-stage framework is proposed that, by combining advanced visual perception an‎d deep reinforcement learning, facilitates the process of transferring motor knowledge from a teacher robot to a student robot in performing upper-body motor skills such as waving hello an‎d wiping the forehead. In the first stage, to stably extract visual features an‎d understan‎d movement differences over time, two prominent architectures, including the EfficientNetV2-S convolutional neural network an‎d the ViT-B/16 vision transformer, were comparatively eva‎luated based on the Time-Contrastive Networks approach. The results of the feature space eva‎luations showed that the attention-based structure in the ViT-B/16 architecture, by providing more coherent representations an‎d higher discriminability, has a significant superiority in understan‎ding motion sequences compared to convolutional networks. The main factor for this superiority is the ability of the attention mechanism to globally process the image an‎d extract complex spatial relationships among the joints of the teacher robot; this feature allows the student robot to extract the precise motion pattern from the body of a similar robot with high robustness to environmental changes. In the second stage, these extracted visual representations were employed within a deep reinforcement learning framework based on the Soft Actor-Critic algorithm; such that the cosine distance between the extracted features of the student robot an‎d the reference pattern of the teacher robot in the latent space was used for the precise design of the reward function in the Webots simulator environment. Simulation results indicate that the reward signal derived from the feature space of the ViT-B/16 model, due to its high stability, enables the student robot to imitate the waving an‎d forehead-wiping skills with a high learning convergence rate. These results prove the high efficiency of the proposed framework in overcoming the complexities of robot-to-robot training.

مازيار پالهنگ

مهران صفاياني , سمانه حسيني