Illinois aerospace engineers combined reinforcement learning with contraction theory, placing mathematical bounds on how far a robot can drift off course, and improved tracking by up to 15% on three real machines.

Reinforcement learning has produced remarkable results in simulation: machines that teach themselves to walk, fly and manipulate objects by trial and error. Letting such a system drive a real robot is another matter, because nothing in the method promises that the robot will not do something ruinous.

Aerospace engineers at the University of Illinois Urbana-Champaign report a way to supply that promise. Writing in IEEE Transactions on Robotics, they describe CARL, for Contraction-Aware Reinforcement Learning, which pairs a learning controller with a guarantee from control theory, and which improved tracking performance by up to 15 per cent over existing methods.

“Reinforcement learning offers a promising way for robots to learn effective behavior, but it often lacks stability guarantees,” said Minjae Cho, a PhD student at the Grainger College of Engineering and the paper’s first author. “Without confidence that a robot will remain stable, deploying it in practice is difficult.”

“In a worst-case scenario, it could break and require costly repairs,” he said. “So, guaranteeing stability of the robot as well as ensuring it can learn effective behavior is the key idea of this project.”

Two traditions that rarely meet

The work sits at a boundary between two fields that have historically answered different questions.

Classical control theory, the discipline behind autopilots and industrial machinery, is built on proofs. An engineer designs a controller and demonstrates mathematically that the system will converge to its target and stay there. The guarantees are strong, and the cost is that the designer must model the system in advance.

Reinforcement learning inverts that. A controller discovers a policy from experience, which can handle situations nobody anticipated, but what it has learned is a function approximated by a neural network, with no accompanying proof about what it will do in a case it has not seen.

CARL’s contribution is to constrain the learning with a result from control theory. “CARL combines reinforcement learning with contraction theory to provide stability guarantees for its performance,” Cho said.

Contraction theory concerns whether nearby trajectories of a system converge toward one another over time. If they do, the system is contracting, and small disturbances shrink rather than compound. It is a useful framing because it bounds behaviour without requiring the designer to know every disturbance in advance.

The paper’s title adds “statistical robustness” to the description, which points to the kind of guarantee on offer: not an absolute worst-case bound covering every conceivable disturbance, but one that holds with quantified confidence given the uncertainty the system is likely to meet.

Bounded error, not zero error

The guarantee is not perfection, and the paper does not claim it.

“Small tracking errors may occur, but the method provides mathematical limits on how large they can become,” Cho said. “This allows reinforcement learning to serve as the robot’s main controller with stability guarantees.”

That is the significant phrase: as the robot’s main controller. Much practical work in this area treats a learned policy as a suggestion, wrapped in a conventional safety controller that overrides it when things look dangerous. Such architectures work but sacrifice much of what the learning offered. A bounded guarantee on the learner itself lets it stay in charge.

A bound on error is also what engineers need in order to specify a system. A drone that may deviate by an unknown amount cannot be flown near obstacles. One that will not deviate by more than a stated margin can be given clearances based on that margin.

The gap between simulation and reality

Cho tested CARL in simulation first, then on three physical robots: a wheeled TurboBot 3 Burger, a flapping aerial drone and a six-legged Jethexa.

The three were chosen to span different dynamics. A wheeled robot rolls on the ground; a flapping-wing aircraft generates lift through an unsteady, hard-to-model process; a hexapod coordinates many legs and changes which of them are touching the ground. A method that holds across all three is being tested on more than one kind of problem.

The obstacle common to all of them is that a simulator is never the world. “There is a difference between the model we train in the computer world to the physical robot in the real world,” Cho said. “For example, a model in the simulator has a mass of 1 kilogram. In the real world, it may be 1.2 kilograms. This is called the sim-to-real gap, but our algorithm was shown to be better at handling those uncertainties.”

The sim-to-real gap is the standing difficulty of learned robot control. A policy trained in simulation can exploit the simulator’s inaccuracies, performing beautifully on a model and poorly on hardware, and a 20 per cent error in mass is a mild example compared with unmodelled friction, flex and sensor noise.

Better on hardware than expected

The result Cho found most striking was that the improvement survived the move to hardware.

“There was no guarantee that our method would work better than the existing methods, particularly with the increasing complexity of the robots we tested,” he said. “But we found that, not only was integrating those two methods novel, but the performance achieved in the computer world is consistently better in the real-world testing as well, by up to 15%.”

A method whose advantage holds up on real machines is doing something more than fitting a simulator, which is the usual suspicion about results of this kind.

Where it might be used

Cho suggests wildfire management as an application, with robots identifying hotspots or delivering supplies. It is a reasonable example of the case for the method: an environment that cannot be modelled in advance, where a machine must adapt, and where a failure is both expensive and difficult to reach.

The paper is open access and the code is on GitHub, which matters for a result of this kind. Claims about learned controllers are notoriously hard to reproduce, since performance depends on details of implementation and tuning that papers often omit, and published code is what allows other groups to check.

Cho’s co-authors were Hiroyasu Tsukamoto and Huy T. Tran, both of whom advise him. The release does not state the project’s funding sources.

---

Notes

1. Cho, M., Tsukamoto, H., Tran, H. T. “Contraction-Aware Reinforcement Learning for Nonlinear Control with Statistical Robustness.” IEEE Transactions on Robotics, 2026. DOI: 10.1109/TRO.2026.3727138.
2. University of Illinois Grainger College of Engineering. “New method for guaranteed stability and improved performance for robots on the move.” EurekAlert!, 20 October 2026. https://www.eurekalert.org/news-releases/1148581.