Making IoT Data Processing Scalable: Architectures That Actually Work

Abstract

Scalable Data Processing Architectures for Internet of Things Applications (2022) investigates the trade-offs involved in different architectural paradigms for processing data generated by the Internet of Things, covering considerations of latency, scalability, reliability, and governance. The analysis demonstrates that no single architecture is suitable for all IoT data- processing workloads and identifies edge and fog computing approaches as the most realistic choice at present. A cloud- centric approach offers advantages in terms of scale and elasticity but suffers from higher latency and a lack of resilience to network failures. Recent developments in data ingestion and transport, storage strategies, and stream processing frameworks are also examined, highlighting protocols, standards, storage models, and fault-tolerance techniques conducive to IoT workloads.
The rapid growth of the Internet of Things presents immense opportunities and challenges for society. Layered architectures that help solve these challenges by distributing computing, storage, and control elements can be mapped to specific use cases; the choice of architecture depends on the workloads associated with the various subsystems. Recent work demonstrates the need for structured high-level data interchange during the data-ingestion and transport phase of IoT data- processing pipelines. However, the scalable nature of the cloud computing paradigm remains appealing for many IoT data- processing workloads running on cloud infrastructures, because of the ease of elasticity and resource management it provides.