Safe Reinforcement Learning for Autonomous and Constraint-Aware Earth Observation Satellite Scheduling
Fabio FAVARETTO, Claudio RONCHETTI, Guido VOLPI
Abstract. Satellite scheduling requires balancing dynamic tasking demands, limited resources, and real-time adaptability, conditions under which traditional rule-based and heuristic methods often fall short. This study explores Safe Reinforcement Learning (SRL) as a framework for autonomous, adaptive, and constraint-aware mission planning. We propose a frugal fine-tuning strategy that adapts a Proximal Policy Optimization (PPO) baseline to new operational domains within the BSK-RL simulation environment, leveraging soft constraints modeled through cost signals. Two SRL approaches are evaluated: (i) reward reshaping, which integrates constraint costs into the reward function, yielding stability improvements and an 8% reduction in actions per episode without sacrificing cumulative rewards; and (ii) Lagrangian-based Constrained Policy Optimization (CPO), which separately optimizes rewards and constraint costs, prioritizing constraint satisfaction and reducing actions by 20–28% at the expense of overall reward. Results demonstrate that SRL enhances both efficiency and constraint adherence in scheduling, while the proposed fine-tuning approach enables rapid transfer to new mission scenarios. These findings highlight SRL as a promising pathway toward scalable, safe, and adaptive satellite operations.
Keywords
Satellite Scheduling, Safe Reinforcement Learning, Autonomous Mission Planning, Constraint-Aware Optimization, Proximal Policy Optimization, Reward Reshaping, Lagrangian Relaxation, BSK-RL Framework, Space Operations, Intelligent Systems for Aerospace
Published online 7/20/2026, 6 pages
Copyright © 2026 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA
Citation: Fabio FAVARETTO, Claudio RONCHETTI, Guido VOLPI, Safe Reinforcement Learning for Autonomous and Constraint-Aware Earth Observation Satellite Scheduling, Materials Research Proceedings, Vol. 69, pp 1030-1035, 2026
DOI: https://doi.org/10.21741/9781644904251-178
The article was published as article 178 of the book CEAS – AIDAA Conference 2025
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
References
[1] Xhafa, F., Herrero, X., Barolli, A., Barolli, L., & Takizawa, M. Evaluation of struggle strategy in genetic algorithms for ground stations scheduling problem. Journal of Computer and System Sciences, 2013, 79(7), 1086-1100. https://doi.org/10.1016/j.jcss.2013.01.023
[2] Xhafa, F., Sun, J., Barolli, A., Takizawa, M., & Uchida, K. Evaluation of genetic algorithms for single ground station scheduling problem. In 2012 IEEE 26th International Conference on Advanced Information Networking and Applications, 2012, pp. 299-306. https://doi.org/10.1109/AINA.2012.59
[3] Thangavel, K., Sabatini, R., Gardi, A., Ranasinghe, K., Hilton, S., Servidia, P., & Spiller, D. Artificial intelligence for trusted autonomous satellite operations. Progress in Aerospace Sciences, 2024, 144, 100960. https://doi.org/10.1016/j.paerosci.2023.100960
[4] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. Proximal policy optimization algorithms (2017). arXiv preprint arXiv:1707.06347.
[5] Huang, Y. Deep Q-networks. In Deep reinforcement learning: fundamentals, research and applications. Singapore: Springer Singapore, 2020, pp. 135-160. https://doi.org/10.1007/978-981-15-4095-0_4
[6] Wang, H., Qin, J., & Kan, Z. Shielded planning guided data-efficient and safe reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems, 36(2), 2024, 3808-3819. https://doi.org/10.1109/TNNLS.2024.3359031
[7] Maisonobe, L., Pommier, V., & Parraud, P. Orekit: An open source library for operational flight dynamics applications. In 4th international conference on astrodynamics tools and techniques, Paris: European Space Agency, 2010 May, pp. 3-6.
[8] Kenneally, P. W., Piggott, S., & Schaub, H. Basilisk: A flexible, scalable and modular astrodynamics simulation framework. Journal of aerospace information systems, 2020, 17(9), pp. 496-507. https://doi.org/10.2514/1.I010762
[9] Chen, X., Tian, T., Dai, G., Wang, M., Song, Z., Zheng, W., & Zhou, Q. A conflict Rainbow DQN-based two-stage optimization framework for multiple agile satellites scheduling. IEEE Transactions on Aerospace and Electronic Systems, 2025, 61(3). https://doi.org/10.1109/TAES.2025.3538476
[10] Liu, Z., Xiong, W., Jia, Z., & Han, C. Two-stage deep reinforcement learning method for agile optical satellite scheduling problem. Complex & Intelligent Systems, 2025, 11, 35. https://doi.org/10.1007/s40747-024-01667-x
[11] Xing, X., Wang, S., Liu, W., & Liu, C. A multi-satellite multi-target observation task planning and replanning method based on DQN. Sensors, 2025, 25(6), 1856. https://doi.org/10.3390/s25061856
[12] Chun, J., Yang, W., Liu, X., Wu, G., He, L., & Xing, L. Deep reinforcement learning for the agile earth observation satellite scheduling problem. Mathematics, 2023, 11(19), 4059. https://doi.org/10.3390/math11194059
[13] Liu, D., & Zhou, G. Deep reinforcement learning-based attention decision network for agile Earth observation satellite scheduling. Remote Sensing, 2024, 16(23), 4436. https://doi.org/10.3390/rs16234436
[14] Garcıa, J., & Fernández, F. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 2015, 16(1), pp. 1437-1480
[15] Achiam, J., Held, D., Tamar, A., & Abbeel, P. Constrained policy optimization. In Proceedings of the 34th International Conference on Machine Learning, PMLR, 2017, pp. 22-31.
[16] Stephenson, M. A., & Schaub, H. BSK-RL: Modular, high-fidelity reinforcement learning environments for spacecraft tasking. 75th International Astronautical Congress, Milan, Italy, 2024. https://doi.org/10.52202/078372-0120
[17] Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., … & Stoica, I. Ray: A distributed framework for emerging {AI} applications. In 13th USENIX symposium on operating systems design and implementation (OSDI 18), 2018, pp. 561-577.
[18] Information on https://docs.ray.io/en/latest/ray-overview/getting-started.html

