Insights on state of the art and perspectives of XR for human machine interfaces in advanced air mobility and urban air mobility

Insights on state of the art and perspectives of XR for human machine interfaces in advanced air mobility and urban air mobility

Sandhya Santhosh, Francesca DeCrescenzio, Millene Gomes Araujo, Marzia Corsi, Sara Bagassi, Fabrizio Lamberti, Filippo Gabriele Pratticò, Domenico Accardo, Claudia Conte, Francesco De Nola, Marco Bazzani, Joyce Adriano Losi

Abstract. With technological innovation and advancements, especially in autonomy, battery and digitization, the future of air transport and mobility is transiting towards a broader spectrum of Advanced Air Mobility (AAM) and Urban Air Mobility (UAM). UAM envisions safer, faster, and more sustainable air mobility for smarter cities and urban environments including passenger transport and goods delivery. Nevertheless, this concept is still considered extremely breakthrough and several technological and operational aspects are mostly undefined. In this context, a comprehensive approach to AAM/UAM may be to adapt cutting-edge technologies in developing sustainable framework and Human-Machine Interfaces (HMIs) in order to realize the challenges, benefits, and conditions of such transport system in advance for future safer, more reliable and globally approved operations. One of the technologies that can contribute to accelerate advancements through human centred simulating UAM processes and operations is XR (eXtended Reality). This paper presents the early steps of a multidisciplinary study performed under the framework of PNRR (Piano Nazionale di Ripresa e Resilienza) and MOST (Centro Nazionale Mobilità Sostenibile) project in analyzing the perspectives of XR based HMIs for UAM paradigm and potential AAM/UAM use case scenarios that can be simulated with XR in view of attaining efficient and effective future solutions. Furthermore, the work introduces the state-of-the-art overview on XR facilitated UAM applications and considers prospective potential use cases that can be developed through PNRR research study in demonstrating XR as an enabling technology in promising areas of the UAM framework.

Keywords
Urban Air Mobility, Advanced Air Mobility, Unmanned Aerial Systems, Immersive Technologies, Extended Reality, Human Machine Interfaces, U-Space

Published online 11/1/2023, 5 pages
Copyright © 2023 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA

Citation: Sandhya Santhosh, Francesca DeCrescenzio, Millene Gomes Araujo, Marzia Corsi, Sara Bagassi, Fabrizio Lamberti, Filippo Gabriele Pratticò, Domenico Accardo, Claudia Conte, Francesco De Nola, Marco Bazzani, Joyce Adriano Losi, Insights on state of the art and perspectives of XR for human machine interfaces in advanced air mobility and urban air mobility, Materials Research Proceedings, Vol. 37, pp 426-430, 2023

DOI: https://doi.org/10.21741/9781644902813-94

The article was published as article 94 of the book Aeronautics and Astronautics

Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

References
[1] AAM National Strategic Plan (2021-2030) for the development of Advanced Air Mobility in Italy, www.enac.gov.it
[2] White Paper on Urban Air Mobility and Sustainable development, https://www.asd-europe.org (2023).
[3] M. Tojal, H. Hesselink, A. Fransoy, E. Ventas, V. Gordo, Y. Xu, Analysis of the definition of Urban Air Mobility –how its attributes impact on the development of the concept, Transportation Research Procedia, Volume 59, 2021, Pages 3-13, ISSN 2352-1465. https://doi.org/10.1016/j.trpro.2021.11.091
[4] EASA Urban Air Mobility https://www.easa.europa.eu/en/domains/urban-air-mobility-uam
[5] Bauranov, A., & Rakas, J. (2021). Designing airspace for urban air mobility: A review of concepts and approaches. Progress in Aerospace Sciences, 125, p.100726. https://doi.org/10.1016/j.paerosci.2021.100726
[6] Schweiger, K. and Preis, L., 2022. Urban Air Mobility: Systematic Review of Scientific Publications and Regulations for Vertiport Design and Operations. Drones, 6(7), p.179. https://doi.org/10.3390/drones6070179
[7] Full report https://www.easa.europa.eu/sites/default/files/dfu/uam-full-report.pdf
[8] Santhosh, S., De Crescenzio, F. and Vitolo, B., 2022. Defining the potential of extended reality tools for implementing co-creation of user oriented products and systems. In Design Tools and Methods in Industrial Engineering II: ADM 2021, September 9–10, 2021, Rome, Italy (pp. 165-174). Springer International Publishing. https://doi.org/10.1007/978-3-030-91234-5_17
[9] Bagassi, S., De Crescenzio, F., Piastra, S., Persiani, C. A., Ellejmi, M., Groskreutz, A. R., & Higuera, J. (2020). Human-in-the-loop evaluation of an augmented reality based interface for the airport control tower. Computers in Industry, 123, 103291. https://doi.org/10.1016/j.compind.2020.103291
[10] Pratticò, F. G., & Lamberti, F. (2021). Towards the adoption of virtual reality training systems for the self-tuition of industrial robot operators: A case study at KUKA. Computers in Industry, 129, 103446. https://doi.org/10.1016/j.compind.2021.103446
[11] Sikorski, B., Leoncini, P., & Luongo, C. (2020). A glasses-based holographic tabletop for collaborative monitoring of aerial missions. In Augmented Reality, Virtual Reality, and Computer Graphics: 7th Int.Conf., AVR 2020, Lecce, Italy, September 7–10, 2020, Proceedings, Part I 7 (pp. 343-360). Springer International Publishing. https://doi.org/10.1007/978-3-030-58465-8_26
[12] https://www.easa.Europa.eu/en/light/topics/vertiports-urban-environment
[13] https://www.agendadigitale.eu/industry-4-0/hmi-cose-ladvanced-human-machine-interfaces-e-perche-e-utile-per-lindustria-4-0/, Accessed on 22/05/2023.
[14] https://www.statista.com/
[15] Malich T., Hanakova L., Socha V., Van den Bergh S., Serlova M., Socha L., Stojic S., Kraus J.Use of virtual and Augmented Reality in design of software for airspace (2019) ICMT 2019 – 7th International Conference on Military Technologies, Proceedings, art. no. 8870030. https://doi.org/10.1109/MILTECHS.2019.8870030
[16] Marayong, P., Shankar, P., Wei, J., Nguyen, H., Strybel, T. Z., & Battiste, V. (2020, March). Urban Air Mobility System Testbed using CAVE Virtual Reality Environment. In 2020 IEEE Aerospace Conference (pp. 1-7). IEEE. https://doi.org/10.1109/AERO47225.2020.9172534
[17] Dao, Q. V., Homola, J., Cencetti, M., Mercer, J., & Martin, L. (2019, August). A Research Platform for Urban Air Mobility (UAM) and UAS Traffic Management (UTM) Concepts and Application. In International Conference on Human Interaction & Emerging Technologies (IHIET 2019) (No. ARC-E-DAA-TN68588).
[18] Aalmoes, R., & Sieben, N. (2021, March). Noise and visual perception of Urban Air Mobility vehicles. In Delft International Conference on Urban Air Mobility (DICUAM), virtual.
[19] Stolz, Maria and Tim Laudien. “Assessing Social Acceptance of Urban Air Mobility using Virtual Reality.” 2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC) (2022): 1-9. https://doi.org/10.1109/DASC55683.2022.9925775
[20] Janotta, F. & Hogreve, J (2021). Acceptance of AirTaxis – Empirical insights following a flight in virtual reality. https://doi.org/10.31219/osf.io/m62yd
[21] Stewart Birrell, William Payre, Katie Zdanowicz, Paul Herriotts, Urban air mobility infrastructure design: Using virtual reality to capture user experience within the world’s first urban airport, Applied Ergonomics, Volume 105, 2022, 103843, ISSN 0003-6870. https://doi.org/10.1016/j.apergo.2022.103843
[22] T. Laudien, J. M. Ernst and B. Isabella Schuchardt, “Implementing a Customizable Air Taxi Simulator with a Video-See-Through Head-Mounted Display – A Comparison of Different Mixed reality Approaches,” 2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC), Portsmouth, VA, USA, 2022, pp. 1-10. https://doi.org/10.1109/DASC55683.2022.9925870
[23] https://aurora-uam.eu/#summary
[24] https://360.world/clairity/#:~:text=The%20CLAIRITY%20system%20takes%20the,unique%20HMD%20and%20camera%20solutions.
[25] Konstantoudakis K, Christaki K, Tsiakmakis D, Sainidis D, Albanis G, Dimou A, Daras P. Drone Control in AR: An Intuitive System for Single-Handed Gesture Control, Drone Tracking, and Contextualized Camera Feed Visualization in Augmented Reality. Drones. 2022 Feb 10;6(2):43. https://doi.org/10.3390/drones6020043
[26] https://www.volocopter.com/newsroom/cae-and-volocopter-partner-to-create-global-air-taxi-pilot-workforce/
[27] https://flyelite.com/evtol/
[28] https://www.geospatialworld.net/news/worlds-first-augmented-reality-drone-flight-simulator-app-launched-epson/
[29] https://www.dji.com/it/simulator/info