Changes in the aviation industry
COVID-19 has had a significant impact on the aviation industry. Being exposed to the possible resurgence of the pandemic has forced the industry to shift even more focus on safety, efficient workflows, sustainability, and flexibility. Though current air traffic is low, experts believe the overall trend of flights to be increasing. To survive future fluctuations, airports need to consider further digitalization and automation of the workplaces. The current situation makes upcoming years the right time to improve infrastructures and operational efficiency. The use of artificial intelligence (AI) has the potential to bring further benefits for all involved parties. One major part of the industry is air traffic management (ATM). The aim of ATM is to control the safety risks and to facilitate the flow of air traffic in a seamless and efficient way.
Air traffic controllers (ATC) are the eyes and ears on the ground, their area of responsibility falls into three general operational disciplines: local control or air control, ground control, and flight data/clearance delivery. Currently, the primary method of controlling the immediate airport environment is visual observation from the airport control tower. During the periods following the times of intense activity, controllers tend to relax and overlook the presence of traffic and conditions. This may cause errors and is considered a risk for safety. Using neural networks to contribute will improve the quality of ATC services giving them more time to concentrate on tasks that require their immediate attention and to make quick decisions.
The Remote Tower
With the Remote Tower software, it is possible to provide a flight control service remotely. This gives the aviation industry the aids to optimize the workforce and increase the quality of operating the towers.
Display for the air traffic controllers is a live video provided by two video screens connected to analytics servers. Cameras are set up around an airport providing a seamless panoramic view of the airport environment. The feeling is the same as looking out of the windows in the actual tower.
For current project, we have set up 16 panorama cameras and 2 PTZ cameras, each delivering 30 FPS in HD with data processing speed 3,4GB per second.
Added value for air traffic controllers
Instead of looking out of the window, air traffic controllers (ATC) spend a lot of time searching for data on displays and analyzing it. The Remote Tower project helps to gather and analyze data for ATC, helping to receive additional information that would otherwise go unnoticed or would take a lot of time to gather and analyze. Provided data will help to make better plans and decisions in the future.
Part of the solution is an object tracking software that detects a set of object classes in less than 200ms. The software marks objects with an overlaid graphical elements and follows them throughout the video.
By using data augmentation in the learning process, we enhanced the neural network’s capability to help the software to peer through low visibility, darkness, and extreme weather conditions. Adding noise to the training process, helped us to simulate various weather conditions and increase the overall safety of the airport environment.
Data analysis
Provided data will help to make better plans and decisions in the future. Helping ATC to receive additional information that would otherwise go unnoticed or would take a lot of time to gather and analyze.
Object detection
Dedicated cameras detectobjects. Whether it is anaircraft, a vehicle on therunway, people, a flock ofbirds, or another sort ofintrusion that is causing thealert on the apron, taxiway or runway.
Data augementation
Night vision and thermal cameras help to eliminate existing blind spots. The neural network is trained with the data augmentation method to detect objects in low light and low visibility conditions.
Path prediction
Path prediction of objects’ trajectories helps to ensure better informed ATC and increase the safety of the airport environment.
Product features and benefits
- Data augmentation and intelligent data collection
- Improvements in computational efficiency
- Custom dataset with more than 4.5 mio. images
- 16 panorama cameras and 2 PTZ cameras, each delivering 30 FPS in HD with data processing speed 3,4 GB per second
We provided the customer with an environment to perform re-training of the algorithm, I.e. to add object classes or videos from other airports. The software can be used by all airports, especially smaller ones with small to medium traffic.
The next step
The product, as it is now implemented, is designed for a specialized task and will be used by a small group of people. However, the principle behind the project very generic and can easily be generalized to fit a very broad spectrum of use cases. The ability to detect and track objects automatically in real-time is unique and opens new possibilities for various industries where the location, type and movement of objects is important.
