Robust Reinforcement Learning for Real-World Autonomous Multitask Agents

Overview

In the last decade, Reinforcement Learning (RL) has shown exceptional results on a wide range of tasks, in particular when applied to video games. Most strikingly, these agents have been able to solve both games that require fast, nimble actions, and deep, well-planned decisions. When applied to robotics, these agents have shown similar aptitudes, often outperforming their classical control counterparts. The recent progress in robotic manipulation and legged robot locomotion is a representative example of their capacities on low-dimension continuous control tasks.

The advent of RL, Neural Networks (NN), and ever-more-efficient edge devices are offering the ability to deploy high-performance visuomotor policies in real time on robots. However, key challenges remain: simulators capable of recreating high-fidelity environments are slow, and their visual appearance remains largely different from the real world — particularly for outdoor applications where environmental complexity is much higher. This leads to a gap between what can be simulated and the deployment environment.

The main developments in the R3AMA project are focused on RL solutions applicable to real robotic systems operating in unstructured environments with real sensors. RL is applied to sensorimotor tasks, coordination of tasks on a single agent, and coordination between agents in a shared environment. The core tools are model-based reinforcement learning and high-performance, high-fidelity simulation. The project targets space applications, including the automation of space debris capture and de-orbiting with robotic satellites operating in orbit.

Research Activities

Multi-Task Reinforcement Learning for Space Robotics

A core challenge in deploying autonomous agents in space is the need to perform multiple distinct tasks — navigation, inspection, proximity operations, and manipulation — within a single unified policy. R3AMA develops RL frameworks that learn shared representations across tasks, enabling agents to transfer knowledge between objectives and adapt to novel mission requirements without retraining from scratch.

Sim-to-Real Transfer

Orbital environments present extreme sim-to-real challenges: zero-gravity dynamics, harsh lighting conditions, and the absence of GPS or standard sensing modalities. To bridge this gap, R3AMA employs domain randomisation, physics-informed simulation, and high-fidelity rendering to train policies that generalise robustly to real hardware.

Autonomous Navigation

Autonomous agents operating in proximity to other spacecraft must navigate safely while avoiding collisions and respecting fuel constraints. R3AMA applies deep RL to develop robust navigation policies that account for disturbances, partial observability, and the unique dynamics of orbital mechanics.

Publications

• El-Hariry, M., Richard, A., Marsal, R., Batista, L. F. W., Geist, M., Pradalier, C., & Olivares-Mendez, M. (2025). RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation. Transactions on Machine Learning Research. Read

Partners

CNRS IRL2958 GT-CNRS — France
Joint research lab of CNRS and the Georgia Institute of Technology. Expertise: field robotics, AI, robotics for natural environments.
Cédric Pradalier, Stéphanie Aravecchia

IADYS — France
SME commercialising Unmanned Surface Vessels (USV) for waterway cleaning. Expertise: USVs, system integration, computer vision.
Nicolas Carlesi, Ronald Loschmann

University of Luxembourg — SpaceR — Luxembourg
Space Robotics laboratory. Expertise: space robotics, AI, reinforcement learning, Zero-G environments.
Miguel Olivares-Mendez, Antoine Richard, Ricard Marsal i Castan