PRIMARY-BASED SPECTRAL BLOOM FILTER FOR THE ENSURING CONSISTENCY IN DISTRIBUTED DOCUMENT-BASED NoSQL DATABASES USING ACTIVE ANTI-ENTROPY MECHANISM

Abstract

The purpose of this work is to compare the existing methods of forming the Spectral Bloom filter using hash functions and the proposed method using prime numbers. The proposed method allows obtaining snapshots from documents that can be used to maintain data consistency in distributed document-oriented NoSQL databases as part of the Active Anti-Entropy mechanism. Data consistency is an important and challenging task due to the need for horizontal scaling of information systems. Neglecting this can lead to material or even human losses, since digitalization covers absolutely all spheres of human activity and there is a need for distributed processing and storage of information.

Consistency can be ensured in various ways, including an architectural approach and Active Anti-Entropy mechanisms. The architectural approach refers to centralized write operations that are distributed to secondary nodes. Accordingly, read operations take place from secondary nodes. This approach is not flexible, as it requires stable and fast communication with the central node, which is not always possible.

The Active Anti-Entropy mechanism is a background process that checks the consistency of data between nodes using special snapshots that can be obtained using hash functions or such a data structure as a Merkle Tree. Using the latter is ideal for checking the consistency of entire data sets, but for mission-critical data, this solution is not suitable. The probability of collisions or the computational cost can lead to inconsistency of the entire data set and this requires a special solution for critical data.

The proposed method makes it possible to obtain the Spectral Bloom filter from the original data set faster. In addition, it has higher collision resistance compared to the use of hash functions, which allows faster identification of inconsistencies in documents stored on different nodes.