HYBRID MODEL OF TEST SCENARIO GENERATION BASED ON LLM AND DEFECT HISTORY ANALYSIS
DOI:
https://doi.org/10.32689/maup.it.2025.4.8Keywords:
automated testing, large language models, defect history, hybrid model, risk-based scenarios, CI/CDAbstract
The article presents the concept of a hybrid model of automated software testing that combines the capabilities of large language models with defect history analysis. The model is focused on forming risk-based test scenarios that can adaptively respond to changes in software systems and improve the efficiency of testing processes. Particular attention is paid to integrating the generative capabilities of LLMs with log analytics mechanisms, which enables the creation of relevant test sets in the context of CI/CD environments. Purpose. The purpose of the study is to develop a concept of an information system that integrates artificial intelligence technologies and defect analysis methods to ensure more accurate and timely error detection. Methodology. The key feature of the approach is the construction of a universal hybrid model that can be scaled for various software development domains and is not limited to a specific class of systems or technologies. In the future, the model will support test prioritization mechanisms based on risk levels, taking into account defect frequency and criticality, as well as the ability to automatically restore scenarios when requirements change. Scientific novelty. The scientific novelty of the research lies in combining the generative capabilities of LLMs with riskbased defect history analysis, which enables the creation of adaptive and relevant testing scenarios. Conclusion. The practical value lies in the possibility of integrating the proposed model into modern CI/CD processes to reduce the cost of maintaining automated tests, increase the accuracy of detecting critical errors, and improve overall software reliability. The proposed model demonstrates the potential for developing intelligent mechanisms of automated testing and building a flexible quality assurance infrastructure that promotes the effective combination of engineering practices with modern AI technologies.
References
Alencar P., Cowan D., Lucena C., Lucena M. Log-based anomaly detection in software systems: A natural language processing approach. Journal of Systems and Software, 2020. 165, 110570. https://doi.org/10.1016/j.jss.2020.110570
Hellendoorn V. J. Large language models for software engineering: The next generation of tools. Communications of the ACM, 2023. 66(7), 34–36. https://doi.org/10.1145/3589300
Kiciński P., Dylong T. Applying ChatGPT in Software QA: A comparative study of manual and AI-generated test cases. Proceedings of the 2024 IEEE International Conference on Software Quality, Reliability and Security (QRS). 2024.
Li W., Zhao Y., Sun Q. Defect data-driven testing strategy optimization. Software Quality Journal, 2023. 31(2), 487–509.
Liu C., Wu J., Lu P. A hybrid framework for AI-assisted test case generation using bug reports. IEEE Access, 2023. 11, 90345–90358.
Panichella A., Kifetew F. M., Tonella P. Automated test case generation as a learning task. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18) 2018. (pp. 1070–1081). New York, NY: ACM. https://doi.org/10.1145/3180155.3180204
Schäfer M., Nadi S., Eghbali A., Tip F. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 2023. 50(1), 85–105. https://doi.org/10.1109/TSE.2023.3334955
Wang S., Chen T. Y., Harman M. Test case prioritization using machine learning: A systematic mapping study. Information and Software Technology, 2019. 93, 41–57. https://doi.org/10.1016/j.infsof.2017.09.002
Wang Y., Guo S., Tan C. W. From code generation to software testing: AI Copilot with context-based RAG. IEEE Software. 2025. https://doi.org/10.1109/MS.2025.3549628
Zhang J., Harman M., Ma L. Machine learning testing: Survey, landscapes and horizons. IEEE Transactions on Software Engineering, 2020. 47(11), 2205–2231. https://doi.org/10.1109/TSE.2019.2962027






