Rocket Landing

A rocket landing game I made in pygame, and an AI that taught itself to fly it.

GitHub

A rocket, a target on the ground, and gravity. The goal was an AI that lands on the target like a SpaceX booster. It never learnt to land. It learnt to turn towards the target and fly straight into it, like a missile, because that’s what scored best. The full story is in the post.

Play it

This is the game as I made it, running in your browser with the same physics. The faint rocket is the best AI, flying from the same spot, and you’re both scored by the rules it was trained on.

Land it yourself

Space to fire the engine, ← → to turn. The faint rocket is the AI.

Click the game first, then space or ↑ for the engine and ← → to turn. On a phone, hold the buttons.

How it learns

Every generation, 250 rockets spawn at random spots in the sky, each flown by its own small neural network, and get 20 seconds. The best scorers are bred into the next generation.

  • What a rocket sees (7): how far across it is, how high, its angle, its speed sideways and up or down, and how far the target is and at what angle.
  • What it can do (3): thrust, turn left or turn right, one at a time, whichever it wants most.
  • How it’s scored: points for getting closer to the target, for landing within 100 pixels of it and for landing upright, and penalties for flying off the screen, never moving sideways and running out of time.
The average score of each generation's 250 rockets. It peaked around generation 300, and carrying on to 1,000 only made it worse.

What it ended up as

The best network is tiny: seven inputs, three outputs and a single hidden node. Its strongest link goes from vertical speed to thrust.

The best rocket's saved network, with all 17 of its links, flying from a random spot at a random target, like in training. Whichever control wants it most lights up, and the rocket uses it.

The scoring was most likely the problem. A rocket that starts right above the target scores well by luck, and with no penalty for coming in fast, hitting the target scores about as well as landing on it.

It came straight after an AI that learnt to play Flappy Bird, which I made by following a tutorial to learn how NEAT works. This one has no tutorial behind it. The code, with a full write-up of every reward and network shape I tried, is on GitHub.