THE METHOD OF IDENTIFYING KEY ELEMENTS OF A DIGITAL IMAGE IN THE DECISION-MAKING PROCESS OF CLASSIFICATION BY A NEURAL NETWORK

Abstract

This paper addresses the problem of evaluating the quality of data used for training neural networks by identifying significant elements of digital images, which the neural network algorithm relies on for classification. As is well known, image classification systems are widely used in computer vision, where entire images are assigned to a single class without distinguishing individual objects within them. This is a typical problem in computer vision. For classification tasks, pre-trained convolutional neural networks (CNNs) are often employed, trained on labelled datasets. However, the unresolved issue remains as to which specific elements the neural network relies on when making a particular decision. The paper presents a method based on a competitive gradient descent process to extract details (elements) that were significant during the classification process, i.e., key elements. This method involves the competition between the process of image detail degradation and the preservation of classification results. Using a self-trained neural network, the authors analyse the presence of details in a classified digital image by visually assessing the elements directly related to the classified object after applying the proposed method. Thanks to this approach, the degradation of digital image details while preserving classification quality is achieved, and the network architecture may be arbitrary. This allows for a comparison of attention areas in neural networks with different architectures: convolutional architecture and mixing architecture. Based on the research findings, a method for localizing neural network attention with arbitrary architecture is proposed. The preserved elements on the degraded image can provide additional information about the validity of the classification performed by a specific neural network. This can be assessed by localizing the preserved elements. For example, the presence of these elements in the classified object indicates a high probability of correct neural network performance. In other words, if the preserved elements do not belong to the classified object, it can be concluded that the training data is not representative (for example, one of the objects may more frequently appear against a characteristic background). Experimental studies demonstrated the advantages of the proposed method over existing alternatives: accuracy in localizing significant details (elements), the presence of information about global significant elements of the digital image and their shape, as well as its applicability for both convolutional and other types of neural networks.