Reinforcement learning-based adaptive control strategies for sustainable production systems
Apoorva VERMA, Leena BHATIA, Ajay Kumar KUSHWAHA, Roop RAJ, Syed Hassan Imam GARDEZI, Pralay GANGULY
Abstract. The need to achieve sustainable production has become an urgent necessity in the conditions of stricter environmental requirements and the rise in the cost of energy worldwide. The classical proportionalintegralderivative controllers and linear Model Predictive Controllers are conventional model-based control strategies that by nature rely on precise process models and fixed optimization horizons and hence are not well suited to the dynamic, non-linear, and complex nature of the modern production environment. The current paper suggests a new adaptive control system based on Reinforcement Learning (RL), where the overall production system is optimized and controlled to achieve Sustainable Production Systems, with the multi-objective rewarding function, which aims to minimize the Overall Equipment Effectiveness (OEE), defect rate, specific energy consumption, and carbon dioxide emissions, and a physics-informed Digital Twin safety filter that stops unsafe policy execution in training and deployment. The proposed framework is evaluated on a multi-machine flexible manufacturing cell benchmark, which includes CNC milling, turning, and robotic assembly, and yields an OEE of 93.6, a defect rate of 0.9, a decrease in specific energy consumption of 27.8, and a decrease in CO 2 emissions of 23.4 compared to PID baseline controllers. Experiments with ablation prove all the above-mentioned in the necessity of each of the architectural constituents, and the Pareto frontier analysis proves that the proposed SAC agent is the best in the OEE-versus-energy trade-off space compared to all of the competing methods.
Keywords
Reinforcement Learning, Soft Actor-Critic, Sustainable Manufacturing, Adaptive Process Control, Multi-Objective Optimization, Digital Twin, Energy Efficiency
Published online 8/15/2026, 8 pages
Copyright © 2026 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA
Citation: Apoorva VERMA, Leena BHATIA, Ajay Kumar KUSHWAHA, Roop RAJ, Syed Hassan Imam GARDEZI, Pralay GANGULY, Reinforcement learning-based adaptive control strategies for sustainable production systems, Materials Research Proceedings, Vol. 71, pp 183-190, 2026
DOI: https://doi.org/10.21741/97816449042312-25
The article was published as article 25 of the book AI-Driven Innovations in Mechanical, Manufacturing, and Energy Engineering
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
References
[1] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA: MIT Press, 2018.
[2] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017.
[3] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proc. ICML, Stockholm, Sweden, 2018, pp. 1861-1870.
[4] D. Silver et al., “Mastering the game of Go with deep neural networks and tree search,” Nature, vol. 529, pp. 484-489, Jan. 2016. https://doi.org/10.1038/nature16961
[5] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996. https://doi.org/10.1613/jair.301
[6] V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529-533, Feb. 2015. https://doi.org/10.1038/nature14236
[7] S. Gu, E. Holly, T. Lillicrap, and S. Levine, “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” in Proc. ICRA, Singapore, 2017, pp. 3389-3396. https://doi.org/10.1109/ICRA.2017.7989385
[8] Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling network architectures for deep reinforcement learning,” in Proc. ICML, New York, NY, USA, 2016, pp. 1995-2003.
[9] T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” in Proc. ICLR, San Juan, Puerto Rico, 2016.
[10] T. P. Lillicrap et al., “Continuous control with deep reinforcement learning,” in Proc. ICLR, San Juan, Puerto Rico, 2016.
[11] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in Proc. ICML, Lille, France, 2015, pp. 1889-1897.
[12] M. Riedmiller et al., “Learning by playing – solving sparse reward tasks from scratch,” in Proc. ICML, Stockholm, Sweden, 2018, pp. 4344-4353.
[13] S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in Proc. ICML, Stockholm, Sweden, 2018, pp. 1587-1596.
[14] K. Dröder, P. Bobka, T. Germann, F. Gabriel, and F. Dietrich, “A machine learning-enhanced digital twin approach for human-robot collaborative assembly,” Procedia CIRP, vol. 76, pp. 187-192, 2018. https://doi.org/10.1016/j.procir.2018.02.010
[15] A. Nair et al., “Overcoming exploration in reinforcement learning with demonstrations,” in Proc. ICRA, Brisbane, Australia, 2018, pp. 6292-6299. https://doi.org/10.1109/ICRA.2018.8463162
[16] Rout, P., Jha, A. K., Gupta, P., Singh, B. and Choudhury, S., “Failure analysis of composite plate under ballistic impact,” Mater. Today Proc., vol. 74, pp. 1008-1011, 2023. https://doi.org/10.1016/j.matpr.2022.11.357
[17] Ruhela, H. S., Bhardwaj, S., Agrawal, T. and Gupta, P., “Explicit dynamics analysis of shin pads using finite element analysis,” in Int. Conf. Industrial Problems on Machines and Mechanism, Springer, Singapore, 2022, pp. 683-690. https://doi.org/10.1007/978-981-99-4270-1_67
[18] Kumar, M., Rajiyan, J. and Gupta, P., “A computational approach for solving elasto-statics problems,” Mater. Today Proc., vol. 46, pp. 6876-6879, 2021. https://doi.org/10.1016/j.matpr.2021.04.462
