MONITORING MACHINE LEARNING MODEL DRIFT IN PRODUCTION PIPELINES: METHODS, METRICS, AND DEPLOYMENT CONSIDERATIONS

Abstract

The relevance of the study is determined by the need to ensure the stability and effectiveness of machine learning models in the context of dynamic changes in data. The problem of model drift – changes in the statistical characteristics of input data or relationships between features and the target variable – leads to a decrease in prediction accuracy. Detecting and monitoring drift in real-time is crucial for maintaining the stability of models, particularly in fields such as finance, healthcare, and cybersecurity, where changes in input data or conditions can significantly affect model performance. The aim of the paper is to investigate methods for monitoring model drift, particularly within the integration of CI/CD pipelines, to ensure their stability in real-world conditions. Special attention is paid to types of drift (data drift, concept drift, label drift) and the metrics used for their detection. The research methods include analyzing existing tools for monitoring and detecting changes in model behavior through the example of financial risk forecasting, as well as evaluating the effectiveness of integrating monitoring into CI/CD. The scientific novelty lies in the proposed comprehensive approach to detecting drift and integrating monitoring into production pipelines using advanced tools such as Google Vertex AI, AWS SageMaker, and TensorFlow Extended, which allow automatic response to changes in data. The use of such technologies improves prediction accuracy and reduces errors in realworld environments. The study confirms the importance of integrating drift monitoring into the continuous process of updating and adapting models to maintain their effectiveness in the context of constantly changing data. The conclusions show that integrating drift monitoring systems into CI/CD pipelines significantly improves the stability and effectiveness of models. Timely detection of drift allows for prompt model adjustments, reducing the likelihood of model degradation. It has been found that for achieving model stability, the automation of monitoring is crucial, as it allows for a prompt response to changes without manual intervention. This enhances the system’s efficiency and reduces risks related to the deterioration of prediction quality.