FORMAL SPECIFICATION OF MULSEMEDIA OBJECT’S DIGITAL TWIN BASED ON DISCRETE INTERVALS TEMPORAL RELATIONS

Abstract

The article proposes a modification of the relations between discrete intervals, which makes it possible to formally determine the relationship between sets of temporal data of different modalities for the formal description of a mulsemedia object`s digital twin model. A mulsemedia object is a physical object, the state of which is recorded using a set of sensors to form a temporal multimodal digital description that comprehensively defines the object as a person perceives it through the senses. A digital twin of a mulsemedia object is a complex software model of this object, which is designed to predict the possible states and behaviour of the mulsemedia object. The formal description of a mulsemedia object is based on data obtained from a set of sensors, each of which captures information of a certain modality. To combine these data into a single object specification, a temporal relationship must be established between them, since data from different modalities can be registered and be meaningful for the research in different periods of the object`s observation. Qualitative determination of the temporal relationship between sets of data can be done using relations between discrete intervals ("Is Before", "Is After", "Coincides", etc.), but quantitative determination (for example, "How much before") using existing relations discrete intervals are impossible. Therefore, the article proposes to consider existing relations of discrete intervals as qualitative relations, at the same time, introducing their modification - quantitative relations of discrete intervals. The use of quantitative relations of discrete intervals will make it possible to simplify the development of digital twin technology software by improving the quality of the formal specification of data structures that comprehensively reflect interconnected sets of temporal multimodal data obtained in the process of monitoring mulsemedia objects.