Flock Navigation by Coordinated Shepherds via Reinforcement Learning

Nov 7, 2022

TapiaLab at University of New Mexico. Special thanks to Lydia Tapia, Yazied Hasan, John Baxter, and Elena Delgado for their warm welcome to the lab and the great work done together!

This work produced a paper published at the Workshop on the Algorithmic Foundations of Robotics (WAFR) 2022, and a further paper published at the Learning for Agile Robotics workshop at the Conference on Robot Learning (CoRL) 2022. Likewise, the project was presented at the Computer Science Student Conference (CSSC) 2022 at the University of New Mexico (UNM).

Proposed Deep RL solution on Strombom flock dynamics.

Proposed Deep RL solution on Reynolds flock dynamics.

The goal of this set of research projects was to improve the state-of-the-art for the multi-agent shepherding task, a control problem where a set of active agents must guide a flock of passive agents to a target location. The project focused on the use of Reinforcement Learning (RL) to learn shepherding policies, with the ultimate goal of improving the scalability and robustness of the shepherding task compared to heuristic methods. Our proposal consists of Proximal Policy Optimization (PPO) algorithm for centralized training and descentralized execution. The resulting policies were able to outperform the state-of-the-art in terms of convergence speed, scalability, and energy consumption, as shown in the video below.

References

Flock navigation by Coordinated Shepherds via Reinforcement Learning Perception-Based Rewards for Robotic Shepherd Teams Maneuvering Split Flocks