METHOD FOR QUANTITATIVE EVALUATION OF THE EMPIRICAL CONFIRMABILITY OF INVARIANT-ORIENTED SIGNALS IN AUTOMATIC SOFTWARE ERROR DETECTION

Abstract

This paper addresses the problem of improving the reliability of automatic error detection in software through the integration of formal invariant analysis and machine learning methods. The study focuses on the gap between invariant-oriented formal signals and empirically observed anomalies in program execution, which limits the effectiveness of both formal and data-driven approaches to log and metric analysis.

For the first time, a method for the quantitative evaluation of the empirical confirmability of invariant-oriented signals is substantiated, based on their systematic comparison with operational anomalies in program execution. The proposed method formalizes the limits of applicability of invariant analysis, introduces confirmability as an independent criterion for evaluating the quality of error detection models, and justifies the necessity of integrating formal and machine learning levels within a unified information technology framework.

The method is implemented through the construction of execution transitions as aggregated behavioral units, their multimodal representation based on logs and metrics, and the subsequent alignment of formal and empirical signals within a shared analytical space. Empirical verification was conducted on the LO2 dataset, which represents a microservice environment with execution logs, metrics, and labels of correct and erroneous states.

The proposed approach achieved a harmonic quality measure of 0.854 and a precision–recall area under the curve of 0.873, along with improvements in structural characteristics of the model, including an increase in the consistency coefficient to 0.702 and a reduction in entropy-based mixing to 0.398. It was established that 81.4% of invariant violations have empirical confirmation in execution logs, while 18.6% remain unconfirmed. This quantitatively defines the boundary of effectiveness of formal analysis.