AUTOMATED DATA MIGRATION METHOD ACROSS STORAGE VARIANTS IN SYSTEMS WITH MULTIVARIANT PERSISTENCE

Abstract

The purpose of the research is to develop a concept and architecture of an automated data migration system capable of ensuring consistency, integrity and equivalence of information during the transition between heterogeneous storage models in conditions of multivariate persistence. The work is devoted to the creation of a method of automated data migration in environments where the same unit of information can exist in several equivalent forms. Methodology. The method used is based on a formal model of a fact that preserves invariants of structure, types and semantics during the transition between different storage models. The principles of correct mappings between repositories and the conditions under which data transformations remain lossless and reproducible are determined. A system of invariants is proposed that fixes the boundaries of permissible transformations and ensures the logical integrity of facts regardless of the specific storage technology. The key element of the method is a subject-oriented language for describing transformations, which sets the rules for the transition between data models in a formalized form. Its constructions allow performing operations of projection, nesting, expansion, connection and induction of relations between structures; the language has a clear typing and provides correctness checking at the compilation and execution stages. Based on the developed language, a prototype of the system was created, which performs automated data transformation between relational, document, graph and keyvalue repositories without user intervention. The solution architecture includes modules for checking invariants, compiling transformations and performing migrations with transactional control, rollback and transaction log, as well as cost estimation and correctness checking services. Scientific novelty. For the first time, a holistic formal method for automated data migration between types of repositories in systems with multivariate persistence has been formed. A mathematical model of the fact, invariants and mappings has been constructed, which determine the boundaries of permissible transformations and guarantee semantic equivalence between different data materializations. A subject-oriented transformation language with formal verification of totality, typing, and preservation of invariants is proposed; the architecture of an automated system with unified adapters to heterogeneous repositories is implemented. Within the framework of the experiments, a comparison was made with manual transformations and the classical ETL approach, which allowed us to establish a zero number of invariant violations with only a slight increase in migration time. Conclusions. The proposed method is characterized by the ability to provide correct, fault-tolerant, and formally verified data migration without loss of integrity. The prototype confirmed the zero level of invariant violations and the stability of results in various scenarios, as well as the method’s suitability for scaling. The results obtained prove the effectiveness of the proposed approach and open up prospects for further adaptation of the system through the use of training agents and dynamic migration management policies.