Logo: to the web site of Uppsala University

Many business and mission-critical decisions of the Industrial Internet of Things (IIoT) depend on efficient data management of sensor streams. Contemporary distributed IIoT applications consist of large numbers of sensors, producing massive volumes of heterogeneous sensor streams at high rates. The combination of these features of IIoT applications pose substantial challenges for existing Database Management Systems (DBMSs) in providing scalable data analytics. For example, Relational-DBMSs (RDBMSs) exhibit scalability issues, single point of failure, and difficulty in managing heterogeneity due to it’s rigid schemas. In contrast to RDBMSs, distributed NoSQL datastores could provide scalability of heterogeneous data. However, the simple query processing capabilities of NoSQL datastores limit advanced analytics. In this paper, we first compare both approaches, having an RDBMS and NoSQL backend for providing data-management solutions for distributed IIoT applications. Then, we utilize query processing in an in-memory database to integrate edge computing with the NoSQL datastore. By utilizing high-volume streams from a real-world IIoT application of Bosch Rexroth - Hägglund, we show that the proposed approach can potentially overcome the limitations of both RDBMS and NoSQL databases for performing advanced analytics.

Multidimensional numeric arrays are often serialized to binary formats for efficient storage and processing. These representations can be stored as binary objects in existing relational database management systems. To minimize data transfer overhead when arrays are large and only parts of arrays are accessed, it is favorable to split these arrays into separately stored chunks. We process queries expressed in an extended graph query language SPARQL, treating arrays as node values and having syntax for specifying array projection, element and range selection operations as part of a query. When a query selects parts of one or more arrays, only the relevant chunks of each array should be retrieved from the relational database. The retrieval is made by automatically generated SQL queries. We evaluate different strategies for partitioning the array content, and for generating the SQL queries that retrieve it on demand. For this purpose, we present a mini-benchmark, featuring a number of typical array access patterns. We draw some actionable conclusions from the performance numbers.

With the advent of the Industrial Internet of Things (IIoT) and Industrial Analytics, numerous application scenarios emerge, where business and mission-critical decisions depend upon large scale analytics of sensor streams. However, very large volumes of data from data streams generated at a high rate pose substantial challenges in providing scalable analytics from existing Database Management Systems (DBMS). While scalability can be provided by high-performance distributed datastores, due to the simple query operations, access to high-level query-based data analytics is usually limited. This work combines high-level query-based data analytics capabilities with high-performance distributed scalability by applying a wrapper-mediator approach. The Amos II extensible main-memory DBMS provides online query processing data analytics engine in front of the MongoDB distributed NoSQL datastore to support large-scale distributed data analytics over persisted data streams. Thus, the implemented system enables query-based online data stream analytics over persisted data streams stored/logged in distributed NoSQL datastores.

With the advent of cyber-physical systems, industrial internet of things (IIoT) and industrial analytics numerous application scenarios have emerged where business and mission-critical decisions depend upon large scale analysis of data in form of sensor streams. However, large volumes of sensor stream data generated at high frequency pose substantial challenges for existing scalable data analysis techniques requiring the use of high-performance distributed datastores. This work covers in-depth performance comparison of three principal categories of distributed state-of-the-art NoSQL datastores by evaluating their applicability and efficiency for large-scale analysis of sensor logs from real-world hydraulic power systems. One central datastore is selected from each of the three principal categories of NoSQL datastores: MongoDB from the document store, Cassandra from the column store and Redis from the distributed main memory key-value store to be included in the performance evaluation. Understanding the differences and behavior of this type of systems are crucial for optimizing application performance. Key insights from this work can serve as a basis for an improved understanding of the applicability of NoSQL datastores in systems for large scale data stream analysis. This will be important for supporting data analytics in IIoT applications as found in monitoring and control of Power plants, Smart Cities, Transportation systems, Environmental and Health monitoring, etc.

A stream validation system called SVALI is developed in order to continuously validate correct behavior of industrial equipment. A functional data model allows the user to define meta-data, analyses, and queries about the monitored equipment in terms of types and functions. Two different approaches to validate that sensor readings in a data stream indicate correct equipment behavior are supported: with the model-and-validate approach anomalies are detected based on a physical model, while with learn-and-validate anomalies are detected by comparing streaming data with a model of normal behavior learnt during a training period. Both models are expressed on a high level using the functional data model and query language. The experiments show that parallel stream processing enables SVALI to scale very well with respect to system throughput and response time. The paper is based on a real world application for wheel loader slippage detection at Volvo Construction Equipment implemented in SVALI.

Visual Data stream Monitor (VisDM) is a new approach to integrate a visual programming language with a data stream management system (DSMS) to support the construction, configuration, and visualization of data stream applications through a set of building blocks, Visual Data Flow Components (VDFCs). This functionality is provided by extending the LabVIEW visual programming platform to support easy declarative specification of continuous visualizations of continuous query results. With actor-based data flows, visualization of data stream output becomes more accessible.

Main-memory database systems (MMDBs) are viable solutions for many scientific applications. Scientific and engineering data often require special indexing methods, and there is a large number of domain specific main memory indexing implementations developed. However, adding an index structure into a database system can be challenging. Mexima (Main memory External Index Manager) provides an MMDB where new main-memory index structures can be plugged-in without modifying the index implementations. This has allowed to plug into Mexima complex and highly optimized index structures implemented in C/C++ without code changes. To utilize new user defined indexes in queries transparently, Mexima automatically transforms query fragments into index operations based on index properly tables containing index meta-data. For scalable processing of complex numerical query expressions, Mexima includes an algebraic query transformation mechanism that reasons on numerical expressions to expose potential utilization of indexes. The index property tables furthermore enable validating the correctness of an index implementation by executing automatically generated test queries based on index meta-data. Experiments show that the performance penalty of using an index plugged into Mexima is low compared to using the corresponding stand-alone C/C++ implementation. Substantial performance gains are shown by the index exposing rewrite mechanisms.