WAYS TO DETERMINE THE RANGE OF KEYWORDS IN A FREQUENCY DICTIONARY FOR TEXT CLASSIFICATION

Abstract

The paper is devoted to the actual problem of classifying textual documents of the collection by characteristic features, which is used for classifying news, reviews, determining the emotional tone of the text, as well as for forming catalogs of scientific, academic and research works. The paper proposes an approach for determining the significant words of a document for their further use as a feature vector in the classification process. In the course of the work, the author's keywords were identified, a partial dictionary was built, and the correlation between the author's keywords and the list of ordered words of the frequency dictionary based on the TF method, which also includes the author’s keywords, was analyzed. The determination of the range and percentage of significant words allows for further classification of scientific and research papers when forming thematic catalogs even in the absence of a list of author's keywords that can be used for classification. The results show that the use of the entire input range of frequency dictionary words is redundant and leads to a longer classification time.