THE USE OF ACTIVE LEARNING FOR REFINING LABELS IN WEAK SUPERVISION BASED TASKS

Abstract

An iterative method has been proposed that uses active learning to detect and correct mislabeled data in a sparsely annotated dataset. The starting point is a dataset in which examples have only noisy or partially provided annotations.The proposed approach gradually refines the training sample by identifying examples that are highly likely to contain incorrect labels and involves targeted verification, both by humans and using heuristic rules with a high level of confidence.The goal of this paper is to improve the quality of labels in a sparsely annotated dataset with minimal manual annotation costs. To do this, we propose an iterative approach based on active learning and selective human validation.Methodology. An iterative method has been developed in which a discriminative model (DistilBERT) is trained on the current set of labels, evaluates the uncertainty of its predictions based on the variability of outputs, and generates pseudo- labels for samples with high uncertainty. These labels are sent to annotators for quick confirmation or correction. Weak annotation is based on four heuristic λ-functions that form the initial labels, and the generative model takes into account the dependencies between these functions through a regularized connection matrix. The effectiveness of the approach was experimentally investigated on the IMDb Movie Reviews dataset, which contains 50,000 text reviews, with a clear division into training, validation, and test samples.Scientific novelty. Unlike traditional active learning methods, which involve manual re-annotation of the most uncertain samples, a hybrid strategy is proposed that combines weak learning, pseudo-annotation, and selective human verification. This allows the target model accuracy to be achieved 2–3 times faster than classical approaches without weak labels, due to the efficient use of limited human resources and initial information from heuristic rules.Conclusions. An iterative strategy that combines sample selection based on prediction divergence, automatic pseudo- annotation, and selective human validation allows for effective improvement of label quality in weakly annotated data without complete re-annotation. The proposed method demonstrated competitive results on a real-world IMDb review corpus, providing high classification model accuracy with reduced manual labor costs.