HYBRID MODEL OF TEST SCENARIO GENERATION BASED ON LLM AND DEFECT HISTORY ANALYSIS

Authors

DOI:

https://doi.org/10.32689/maup.it.2025.4.8

Keywords:

automated testing, large language models, defect history, hybrid model, risk-based scenarios, CI/CD

Abstract

The article presents the concept of a hybrid model of automated software testing that combines the capabilities of large language models with defect history analysis. The model is focused on forming risk-based test scenarios that can adaptively respond to changes in software systems and improve the efficiency of testing processes. Particular attention is paid to integrating the generative capabilities of LLMs with log analytics mechanisms, which enables the creation of relevant test sets in the context of CI/CD environments. Purpose. The purpose of the study is to develop a concept of an information system that integrates artificial intelligence technologies and defect analysis methods to ensure more accurate and timely error detection. Methodology. The key feature of the approach is the construction of a universal hybrid model that can be scaled for various software development domains and is not limited to a specific class of systems or technologies. In the future, the model will support test prioritization mechanisms based on risk levels, taking into account defect frequency and criticality, as well as the ability to automatically restore scenarios when requirements change. Scientific novelty. The scientific novelty of the research lies in combining the generative capabilities of LLMs with riskbased defect history analysis, which enables the creation of adaptive and relevant testing scenarios. Conclusion. The practical value lies in the possibility of integrating the proposed model into modern CI/CD processes to reduce the cost of maintaining automated tests, increase the accuracy of detecting critical errors, and improve overall software reliability. The proposed model demonstrates the potential for developing intelligent mechanisms of automated testing and building a flexible quality assurance infrastructure that promotes the effective combination of engineering practices with modern AI technologies.

References

Alencar P., Cowan D., Lucena C., Lucena M. Log-based anomaly detection in software systems: A natural language processing approach. Journal of Systems and Software, 2020. 165, 110570. https://doi.org/10.1016/j.jss.2020.110570

Hellendoorn V. J. Large language models for software engineering: The next generation of tools. Communications of the ACM, 2023. 66(7), 34–36. https://doi.org/10.1145/3589300

Kiciński P., Dylong T. Applying ChatGPT in Software QA: A comparative study of manual and AI-generated test cases. Proceedings of the 2024 IEEE International Conference on Software Quality, Reliability and Security (QRS). 2024.

Li W., Zhao Y., Sun Q. Defect data-driven testing strategy optimization. Software Quality Journal, 2023. 31(2), 487–509.

Liu C., Wu J., Lu P. A hybrid framework for AI-assisted test case generation using bug reports. IEEE Access, 2023. 11, 90345–90358.

Panichella A., Kifetew F. M., Tonella P. Automated test case generation as a learning task. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18) 2018. (pp. 1070–1081). New York, NY: ACM. https://doi.org/10.1145/3180155.3180204

Schäfer M., Nadi S., Eghbali A., Tip F. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering, 2023. 50(1), 85–105. https://doi.org/10.1109/TSE.2023.3334955

Wang S., Chen T. Y., Harman M. Test case prioritization using machine learning: A systematic mapping study. Information and Software Technology, 2019. 93, 41–57. https://doi.org/10.1016/j.infsof.2017.09.002

Wang Y., Guo S., Tan C. W. From code generation to software testing: AI Copilot with context-based RAG. IEEE Software. 2025. https://doi.org/10.1109/MS.2025.3549628

Zhang J., Harman M., Ma L. Machine learning testing: Survey, landscapes and horizons. IEEE Transactions on Software Engineering, 2020. 47(11), 2205–2231. https://doi.org/10.1109/TSE.2019.2962027

Published

2025-12-30

How to Cite

ЖЕПЛІНСЬКИЙ, В., & ЛУЧКЕВИЧ, М. (2025). HYBRID MODEL OF TEST SCENARIO GENERATION BASED ON LLM AND DEFECT HISTORY ANALYSIS. Information Technology and Society, (4 (19), 50-54. https://doi.org/10.32689/maup.it.2025.4.8