Please use this identifier to cite or link to this item: http://dx.doi.org/10.14279/depositonce-9384
Main Title: Distributed Mega-Datasets: The Need for Novel Computing Primitives
Author(s): Semmler, Niklas
Smaragdakis, Georgios
Feldmann, Anja
Type: Conference Object
Language Code: en
Abstract: With the ongoing digitalization, an increasing number of sensors is becoming part of our digital infrastructure. These sensors produce highly, even globally, distributed data streams. The aggregate data rate of these streams far exceeds local storage and computing capabilities. Yet, for radical new services (e.g., predictive maintenance and autonomous driving), which depend on various control loops, this data needs to be analyzed in a timely fashion. In this position paper, we outline a system architecture that can effectively handle distributed mega-datasets using data aggregation. Hereby, we point out two research challenges: The need for (1) novel computing primitives that allow us to aggregate data at scale across multiple hierarchies (i.e., time and location) while answering a multitude of a priori unknown queries, and (2) transfer optimizations that enable rapid local and global decision making.
URI: https://depositonce.tu-berlin.de/handle/11303/10432
http://dx.doi.org/10.14279/depositonce-9384
Issue Date: 7-Jul-2019
Date Available: 4-Dec-2019
DDC Class: 006 Spezielle Computerverfahren
Subject(s): database
computer network
computer system
system architecture
Sponsor/Funder: EC/H2020/679158/EU/Resolving the Tussle in the Internet: Mapping, Architecture, and Policy Making/ResolutioNet
License: http://rightsstatements.org/vocab/InC/1.0/
Proceedings Title: 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Publisher Place: Piscataway, NJ
Publisher DOI: 10.1109/ICDCS.2019.00167
Page Start: 1684
Page End: 1692
EISSN: 2575-8411
ISBN: 978-1-7281-2519-0
978-1-7281-2520-6
ISSN: 1063-6927
Notes: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Appears in Collections:FG Internet Network Architectures (INET) » Publications

Files in This Item:
File Description SizeFormat 
semmler_etal_2019.pdfAccepted manuscript1.54 MBAdobe PDFThumbnail
View/Open


Items in DepositOnce are protected by copyright, with all rights reserved, unless otherwise indicated.