Autonomous ATCWith taxiing and queueing working, one big gap was left: the runway. Only one aircraft can use it at a time, and a runway incursion, an aircraft entering it when it shouldn’t, is one of the most dangerous mistakes in air traffic control. In the US alone, more than 26,000 were recorded between 2002 and 2020.
Collision avoidance didn’t cover this. Queueing is closer to what pilots do themselves, and giving way at intersections is a controller’s call, but neither decides who gets the runway. My original plan was a reinforcement learning model that made these higher-level decisions, but it had just failed at a simpler task, so I dropped it.
Instead, I copied what a controller actually does: look. Out of the tower window, or at the radar screen, to check the runway is clear before clearing anyone onto it.
What it looks at
The detector doesn’t see the whole airport. Each time, it gets one strip of the simulation’s picture, about 3 kilometres long and 225 metres across, over the runway and the taxiways up to its holding points: the area where only one aircraft can be at a time.
It reports whether an aircraft is in that strip, and where. The simulation only needs yes or no, but I chose object detection over a simple classifier because knowing where could matter later. And I chose YOLO over slower detectors like Faster R-CNN because it has to keep up with the simulation.
Making the data
There’s no dataset of radar pictures of Birmingham, so I made one. A script drew 5,000 strips from the simulation’s own airport image, with aircraft placed at random in and around the runway, including some only partly in the strip, like a wingtip. Half had an aircraft in them and half didn’t. Each image was varied a little, with changes to brightness, contrast, rotation and noise, and every aircraft’s box was written to a file for training.
Training it
YOLOv8 trained on those strips for 50 epochs, which took about an hour. On the validation set, it was perfect:
Those scores say as much about the task as the model. Aircraft stand out clearly on a radar picture, and the validation images were made exactly like the training ones, so this doesn’t mean it would cope with real-world conditions.
Live in the simulation, it was a little less perfect. Each check took under 100 milliseconds, still too slow for every frame, so it runs twice a second. Sometimes it drew two overlapping boxes on one aircraft, which didn’t change any decisions. Early on, it also mistook runway markings for aircraft, probably because the training images weren’t varied enough. Applying the same changes to the live pictures before checking stopped the false alarms.
Who gets the runway
Detecting aircraft is only half of it. The simulation’s holding points became real stop lines: an aircraft reaching one stops, unless it has been given permission to go on. Once it’s past, the permission resets.
To decide who gets permission, every aircraft has a priority that grows while it waits: 0.01 a frame on the ground, and 0.04 a frame in the air, since an aircraft in the air can’t wait long. When the runway is clear, the aircraft with the highest priority goes. So arrivals usually go first, but a departure that has waited long enough (by which point a queue has likely formed behind it) can go before them.
Real controllers weigh far more than waiting time: fuel, delays, weather and their workload. But this simple version covers the basics, and it’s easy to extend. An emergency, for example, could just add a lot of priority at once.
There are eight basic situations it has to handle: four with the runway clear, where priorities decide, and four with it occupied, where aircraft on the ground hold and aircraft in the air go around.
Putting it together
The detector is also a safety net. If an aircraft enters the runway when it shouldn’t, the detector sees it and the landing aircraft is told to go around immediately. It doesn’t have to be an aircraft either: retrained on other images, it could watch for vehicles or anything else on the runway, which is something a controller can miss when their view is blocked.