Ball Launcher Reinforcement Learning

A robot that learns to play basketball by using Reinforcement Learning

In this project, we are implementing a tossing ball robot that learns by means of Reinforcement Learning. The robot is able to determine its optimal actions by using Computer Vision techniques, combined with different solutions, such as Q-learning and DDPG+HER, which have been broadly researched to have a higher accuracy. Simulations have also been done with OpenAI Gym and ROS (Gazebo), prior to the real implementation, to deeply study the best approach and prepare the model for the real scenario(Stevedan et al., 2022).

The following vide shows the reesult of the project:

Ball-Launcher-Reinforcement-Learning

Some of the results of the experiments performed can be found below:

Training result using Q-learning Simulation
Training results using DDPG Simulation
Training results using DDPG + HER + Sparse Reward Simulation
Training results from the real scenario

Conclusions

In this report, a robot is taught to toss a ball into a bucket using reinforcement different reinforcement learning techniques. In order to do so, a prototype has to be built from scratch both in the real world and in simulation. Despite the limited robustness of the robot, the results presented before are quite promising, both in simulation and in the real-scenario. Using DDPG with HER is a good choice for this kind of application where you need trial and error experiments, and its success rate was good and fast. Adding HER is very useful when the states are varying and the model learns faster. There are some variations that ought to be made to enhance the perfor- mance, however, the designed first prototype has a huge potential.

Full implementation can be found here

References

2022

  1. robot_learning.gif
    Implementation of a tossing ball robot using reinforcement learning
    Omodolor Stevedan, Cornella Guillem, and Sardà David
    Oct 2022
    Abs Bib PDF