Hardware-in-the-loop validation of a sense and avoid system leveraging data fusion between radar and optical sensors for a mini UAV

Hardware-in-the-loop validation of a sense and avoid system leveraging data fusion between radar and optical sensors for a mini UAV

Marco Fiorio, Roberto Galatolo, Gianpietro Di Rito

Abstract. The present work illustrates the results obtained at the conclusion of the three-year project TERSA (Tecnologie Elettriche e Radar per Sapr Autonomi), involving the aerospace section of the Dept. of Civil and Industrial Engineering (DICI) of the University of Pisa and its industrial partners. The project aimed at the design and development of a fully autonomous Sense and Avoid (SAA) prototype system, based on data fusion between optical and radar sensors data, for a tactical lightweight surveillance UAV (MTOW<25Kg). The problem of non-cooperative collision avoidance is well known in literature and is currently a central theme of investigation within the aeronautical industry, considering the growing UAV traffic and the consequent need to employ autonomous self-separation technologies in the market. Several past works have investigated the most varied solutions for the Sense problem utilizing optical, acoustic, electro-magnetic signals or a combination of the previous. Likewise, the Avoidance problem has been successfully tackled in literature by means of a wide variety of different approaches ranging from rule-based methods, strategies based on game theory, force field methods, optimization frameworks leveraging genetic algorithms and nonlinear programming techniques and geometric methods. Yet, to the knowledge of the authors, no previous work found in literature has successfully demonstrated and validated the real-time simultaneous interaction of both sense and avoid functionalities within a highly integrated simulation environment. The present work describes the implementation of a complex, nonlinear, simulation environment conceived in order to perform real-time, Hardware-in-the-Loop (HIL), testing of the effective cooperation between sense and the avoid algorithms constituting the core of the SAA system developed within the context of the project. The system effectiveness has been validated by means of complex dynamic simulations, comprising an accurate, fully nonlinear, flight mechanic model of the aircraft, a graphic rendering engine of the scene, proper video capture and transmission pipelines, computer vision algorithms and collision avoidance logics running on the target hardware (Nvidia Jetson Nano) and tailored noise resilient data fusion algorithms. Results show the effectiveness of the system in detecting impending collisions and performing last-resort resolution manoeuvres with high computational efficiency and update frequencies compatible with real world applications in Unmanned Aircraft Systems (UAS). Keywords
Sense and Avoid, Unmanned Aerial Vehicles, Hardware in The Loop, Data Fusion

Published online 11/1/2023, 6 pages
Copyright © 2023 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA

Citation: Marco Fiorio, Roberto Galatolo, Gianpietro Di Rito, Hardware-in-the-loop validation of a sense and avoid system leveraging data fusion between radar and optical sensors for a mini UAV, Materials Research Proceedings, Vol. 37, pp 70-75, 2023

DOI: https://doi.org/10.21741/9781644902813-16

The article was published as article 16 of the book Aeronautics and Astronautics

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

References
[1] G. Fasano, D. Accardo and A. Moccia, “Sense and Avoid for Unmanned Aircraft Systems,” IEEE A&E SYSTEM MAGAZINE, pp. 82-110, November 2016. https://doi.org/10.1109/MAES.2016.160116
[2] G. Fasano, D. Accardo, A. E. Tirri and A. Moccia, “Radar/electro-optical data fusion for non-cooperative UAS sense and avoid,” Aerospace Science and Technology, vol. 46, pp. 436-450, 2015. https://doi.org/10.1016/j.ast.2015.08.010
[3] S. Ramasmy and R. sabatini, “A Unified Approach to Cooperative and Non-Cooperative Sense-and-Avoid,” in International Conference on Intelligent robots and Systems (IROS), Vancouver, BC, Canada, 2017., 2017.
[4] M. Fiorio, R. Galatolo and G. D. Rito, “Sense and Avoid system for a mini UAV based on data fusion between electro-optical and radar sensors,” in AIDAA XVI International Congress, Pisa, 2021.
[5] M. Fiorio, R. Galatolo and G. D. Rito, “Object Detection and Tracking Algorithms Based on KLT Feature Tracker for a Sense and Avoid System,” in AIDAA XXVI International Congress, Pisa, 2021.
[6] “Echoes Tech,” 2022. [Online]. Available: https://www.echoes-tech.it/
[7] “Sky Eye Systems,” [Online]. Available: https://www.skyeyesystems.it/
[8] C. Carbone, U. Ciniglio, F. Corraro and S. Luongo, “A Novel 3D Geometric Algorithm for Aircraft Autonomous Collision Avoidance,” in Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, CA, USA, 2006. https://doi.org/10.1109/CDC.2006.376742
[9] K. D. Bilimora, “A GEometric Optimization Approach to Aircraft Conflict Resolution,” in AIAA Guidance, Navigation, and Control Conference and Exhibit , Denver, CO, USA, 2000.