METHOD FOR REDUCTIVE PRUNING OF NEURAL NETWORKS AND ITS APPLICATIONS

Abstract

Trained neural networks usually contain redundant neurons that do not affect or degrade the quality of target identification. The redundancy of models possesses an excessive load on computational resources, leading to high electricity consumption. Further, the deployment and operation of such models in resource-constrained environments such as mobile phones or smart devices are either complicated or impossible. Therefore, there is a need to simplify models while maintaining their effectiveness. This work presents a method for fast neural network reduction, allowing for automatic detection and removal of a large number of redundant neurons while simultaneously improving the efficiency of the models. The technique introduces perturbations to the target variable and then identifies and removes the weights with the most considerable relative deviations from the weights of the control model. The method removes up to 90% of active weights. At the same time, unlike classical pruning methods, the efficiency of models improves simultaneously with the reduction. The scientific novelty of the work consists of method development and new practical applications. The reduction method detects and removes large groups of redundant parameters of neural networks. The logic of automatically determining the optimal number of residual "significant" weights was implemented. The mentioned features speed up the discovery and elimination of redundant weights; reduce required time and resources for computations; and automate the identification of the essential neurons. The method's effectiveness was demonstrated on two applied tasks: predicting the yield of chemical reactions and the molecular affinity. The implementation and applications of the method are available via the link: https://github.com/ogurbych/ann-reduction.