The mission
Land a rocket on a pad, in weather it has never seen, with no human pilot. The AI above taught itself how, by trial and error. Every flight on this page is flown live, in your browser.
-
Learn by crashing
It started with no idea how to fly. Every attempt earned points for landing gently and lost points for crashing, tilting or wasting fuel. Over two million practice steps it adjusted itself to earn more points. That is reinforcement learning.
-
Practise in a thousand worlds
An AI that practises in one world learns tricks that only work there. So every practice flight rolled new gravity, wind, turbulence and engine power. Robotics teams use the same idea, domain randomization, to move robots from simulation into the real world.
-
Feel the wind, then tune
Wind is invisible, so the final AI remembers its last four moments and can feel which way it is being pushed. An automatic search then tuned its training settings, judged on practice landings only, never on the final test.
Results
Tested once, on 300 landings per condition that no AI had seen before. The test plan was committed to git before the test ran, and nothing was changed after.
The data
Landing safely, by test condition
Share of 300 unseen test landings that came to rest on the pad. The two columns on the right are the limits: beyond what any agent lands reliably.
Learning curves
Official game score during practice, checked every 33,000 steps in the normal game. Line: average of 3 training runs. Band: best to worst run. 200 is the "solved" mark. The axis stops at −650; a few of the very first check-ups scored lower.
More practice in one world made it more fragile
The same agent trained in the normal game only, for 1 million and then 2 million steps. Twice the practice, worse in conditions it never saw.
Memory in wind
Randomized practice, the same 2 million steps, with and without memory of the last 4 moments. With 300 landings per bar, the gap in plain strong wind could still be chance.
Flight records
Real test landings, recorded from the simulator at 1920 by 1280.