This thesis explores the application of reinforcement learning in controlling Cable-driven parallel robots. These robots are characterized by their intricate dynamics and system nonlinearity, creating an ideal setting for the implementation of reinforcement learning algorithms. However, these algorithms typically demand vast amounts of data to learn the optimal strategy, which can be challenging to obtain in real-world scenarios. To address this limitation, a sim-to-real approach is proposed.
The study initially utilizes the Newton-Euler equation to establish the robot’s dynamic model. By adjusting the parameters to reflect the actual robot values, the model’s accuracy is validated by comparing simulation outcomes with empirical data. The implementation of the model in Matlab/Simulink, followed by conversion to a C++ library, facilitates seamless integration with the Python environment for the gym platform, thereby ensuring precise simulation data and reduced execution time.
To train the reinforcement learning-based controllers effectively, defining the controller objective is crucial. Since most applications of cable-driven parallel robots involve trajectory tracking, a reward function aligned with this goal is devised. Furthermore, a mechanism for generating target trajectories is developed, and constraints on the action space are introduced to maintain cable tension within permissible limits during training.
By incorporating renowned reinforcement learning algorithms for continuous spaces such as DDPG, PPO, and SAC, a comprehensive training framework is established for generating controllers for cable-driven parallel robots. Comparative analysis among these algorithms is conducted during training, and the performance of the trained controllers is evaluated. A comparison is made between the reinforcement learning controller and a PID-based controller developed for insect tracking, considering factors like tracking error, energy consumption, and controller robustness.
A significant challenge addressed in the study pertains to transitioning to different robot configurations. Since the trained policy is configuration-specific, a new training process is necessitated for each distinct configuration. To mitigate this challenge, a novel method for learning an actuator-level policy is proposed, with a comparative assessment against the conventional policy.
Finally, the trained controller is tested on the actual robot to confirm the transferability of the policy from simulation to the real world. The research provides valuable insights into the application of reinforcement learning in controlling cable-driven parallel robots, offering a significant contribution to the field of robotics.
About the Author
