Reciprocal human–machine learning flow modelling for assisted assembly systems
Yuchen FAN, Alessandro SIMEONE, Dario ANTONELLI
Abstract. Industry 5.0 emphasizes human-centric manufacturing by integrating advanced assistive technologies for inclusive production environments. Large Language Models (LLMs) offer new possibilities for assembly error detection and correction. This study introduces a reciprocal human–machine learning framework utilizing LLMs vision capabilities to improve assembly accuracy through real-time image comparison and corrective instruction generation. Human and machine-in-the-loop mechanisms enable continuous refinement, minimizing labeled datasets while enhancing responsiveness. Experimental validation demonstrates the system’s ability to detect errors, diagnose causes, and provide corrective actions. Findings show LLM-driven reciprocal learning improves efficiency, supports diverse operator needs, and enables adaptive error detection in manufacturing.
Keywords
Human-Robot Collaboration, Real-Time Object Detection, Multi-Modal Instruction Interfaces, Assisted Assembly Systems, Generative AI
Published online 9/10/2025, 9 pages
Copyright © 2025 by the author(s)
Published under license by Materials Research Forum LLC., Millersville PA, USA
Citation: Yuchen FAN, Alessandro SIMEONE, Dario ANTONELLI, Reciprocal human–machine learning flow modelling for assisted assembly systems, Materials Research Proceedings, Vol. 57, pp 457-465, 2025
DOI: https://doi.org/10.21741/9781644903735-54
The article was published as article 54 of the book Italian Manufacturing Association Conference
Content from this work may be used under the terms of the Creative Commons Attribution 3.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
References
[1] Leng, J., Sha, W., Wang, B., Zheng, P., Zhuang, C., Liu, Q., Wuest, T., Mourtzis, D., Wang, L.: Industry 5.0: Prospect and retrospect. J Manuf Syst. 65, 279–295 (2022). https://doi.org/10.1016/j.jmsy.2022.09.017
[2] Simeone, A., Fan, Y., Antonelli, D., Catalano, A.R., Priarone, P.C., Settineri, L.: Inclusive manufacturing: A contribution to assembly processes with human-machine reciprocal learning. CIRP Ann Manuf Technol. 73, 5–8 (2024). https://doi.org/10.1016/j.cirp.2024.03.005
[3] Konstantinou, C., Antonarakos, D., Angelakis, P., Gkournelos, C., Michalos, G., Makris, S.: Leveraging Generative AI Prompt Programming for Human-Robot Collaborative Assembly. Procedia CIRP. 128, 621–626 (2024). https://doi.org/10.1016/j.procir.2024.03.040
[4] Liu, S., Zhang, J., Wang, L., Gao, R.X.: Vision AI-based human-robot collaborative assembly driven by autonomous robots. CIRP Annals. (2024). https://doi.org/10.1016/j.cirp.2024.03.004
[5] Yurong, G., Jian, M.: Optimization analysis of assembly system based on digital twin and deep learning. In: Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering. pp. 627–631. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3443467.3443825
[6] Fu, B., Hadid, A., Damer, N.: Generative AI in the context of assistive technologies: Trends, limitations and future directions. Image Vis Comput. 154, 105347 (2025). https://doi.org/10.1016/j.imavis.2024.105347
[7] Fan, H., Liu, X., Fuh, J.Y.H., Lu, W.F., Li, B.: Embodied intelligence in manufacturing: leveraging large language models for autonomous industrial robotics. J Intell Manuf. (2024). https://doi.org/10.1007/s10845-023-02294-y
[8] Shafiee, S.: Generative AI in manufacturing: a literature review of recent applications and future prospects. Procedia CIRP. 132, 1–6 (2025). https://doi.org/10.1016/j.procir.2025.01.001
[9] Zang, Y., Li, W., Han, J., Zhou, K., Loy, C.C.: Contextual Object Detection with Multimodal Large Language Models. Int J Comput Vis. 133, 825–843 (2025). https://doi.org/10.1007/s11263-024-02214-4
[10] Yu, Q., Shen, X., Chen, L.-C.: Towards Open-Ended Visual Recognition with Large Language Models. Presented at the (2025). https://doi.org/10.1007/978-3-031-72630-9_21
[11] Jiang, Y., Yan, X., Ji, G.-P., Fu, K., Sun, M., Xiong, H., Fan, D.-P., Khan, F.S.: Effectiveness assessment of recent large vision-language models. Visual Intelligence. 2, 17 (2024). https://doi.org/10.1007/s44267-024-00050-1
[12] Zhang, X., Wu, K., Ma, Q., Chen, Z.: Research on Object Detection Model Based on Feature Network Optimization. Processes. 9, 1654 (2021). https://doi.org/10.3390/pr9091654
[13] Wang, Z., Qin, H.: Intelligent industrial production process automatic regulation system based on LLM agents. In: 2024 5th International Conference on Artificial Intelligence and Electromechanical Automation (AIEA). pp. 133–137. IEEE (2024). https://doi.org/10.1109/AIEA62095.2024.10692701
[14] Lim, J., Patel, S., Evans, A., Pimley, J., Li, Y., Kovalenko, I.: Enhancing Human-Robot Collaborative Assembly in Manufacturing Systems Using Large Language Models. In: 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE). pp. 2581–2587. IEEE (2024). https://doi.org/10.1109/CASE59546.2024.10711843
[15] Kang, J.H., Dhanaraj, N., Wadaskar, S., Gupta, S.K.: Using Large Language Models to Generate and Apply Contingency Handling Procedures in Collaborative Assembly Applications. In: 2024 IEEE International Conference on Robotics and Automation (ICRA). pp. 15585–15592. IEEE (2024). https://doi.org/10.1109/ICRA57147.2024.10610875
[16] Salvagno, M., Taccone, F.S., Gerli, A.G.: Artificial intelligence hallucinations. Crit Care. 27, 180 (2023). https://doi.org/10.1186/s13054-023-04473-y
[17] Ashqar, H.I., Alhadidi, T.I., Elhenawy, M., Khanfar, N.O.: Leveraging Multimodal Large Language Models (MLLMs) for Enhanced Object Detection and Scene Understanding in Thermal Images for Autonomous Driving Systems. Automation. 5, 508–526 (2024). https://doi.org/10.3390/automation5040029
[18] Wang, Z., Kiyokawa, T., Sera, I., Yamanobe, N., Wan, W., Harada, K.: Error Correction in Robotic Assembly Planning From Graphical Instruction Manuals. IEEE Access. 11, 107276–107286 (2023). https://doi.org/10.1109/ACCESS.2023.3319822
[19] Langer, M., Baum, K., Schlicker, N.: Effective Human Oversight of AI-Based Systems: A Signal Detection Perspective on the Detection of Inaccurate and Unfair Outputs. Minds Mach (Dordr). 35, 1 (2024). https://doi.org/10.1007/s11023-024-09701-0
[20] Aleixo, E.L., Colonna, J.G., Cristo, M., Fernandes, E.: Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy. Journal of the Brazilian Computer Society. 30, (2024). https://doi.org/10.5753/jbcs.2024.3966
[21] Fan, Y., Simeone, A., Antonelli, D., Caggiano, A., Priarone, P.C., Settineri, L.: A framework for digital assembly instructions as a step towards manufacturing inclusiveness. Procedia CIRP. 132, 116–121 (2025). https://doi.org/10.1016/j.procir.2025.01.020
[22] Chen, H., Li, S., Fan, J., Duan, A., Yang, C., Navarro-Alarcon, D., Zheng, P.: Human-in-the-Loop Robot Learning for Smart Manufacturing: A Human-Centric Perspective. IEEE Transactions on Automation Science and Engineering. 1–1 (2025). https://doi.org/10.1109/TASE.2025.3528051
[23] Iyenghar, P.: Clever Hans in the Loop? A Critical Examination of ChatGPT in a Human-in-the-Loop Framework for Machinery Functional Safety Risk Analysis. Eng. 6, 31 (2025). https://doi.org/10.3390/eng6020031
[24] Hatia, A., Doldo, T., Parrini, S., Chisci, E., Cipriani, L., Montagna, L., Lagana, G., Guenza, G., Agosta, E., Vinjolli, F., Hoxha, M., D’Amelio, C., Favaretto, N., Chisci, G.: Accuracy and Completeness of ChatGPT-Generated Information on Interceptive Orthodontics: A Multicenter Collaborative Study. J Clin Med. 13, 735 (2024). https://doi.org/10.3390/jcm13030735
[25] Woo, K.C., Simon, G.W., Akindutire, O., Aphinyanaphongs, Y., Austrian, J.S., Kim, J.G., Genes, N., Goldenring, J.A., Major, V.J., Pariente, C.S., Pineda, E.G., Kang, S.K.: Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings. Journal of the American Medical Informatics Association. 31, 1983–1993 (2024). https://doi.org/10.1093/jamia/ocae117.
[26] Govender, R.G.: My AI students: Evaluating the proficiency of three AI chatbots in completeness and accuracy. Contemp Educ Technol. 16, ep509 (2024). https://doi.org/10.30935/cedtech/14564
[27] Fan, Y., Simeone, A., Antonelli, D., Catalano, A.R., Priarone, P.C., Settineri, L.: A logic-assisted assembly support system to promote the inclusion of neurodiverse operators in manufacturing. Procedia CIRP. (2025). Accepted for publication.
