SNR: Network-aware geo-distributed stream analytics

dc.contributor.authorMostafaei, Habib
dc.contributor.authorAfridi, Shafi
dc.contributor.authorAbawajy, Jemal H.
dc.date.accessioned2021-08-18T07:20:59Z
dc.date.available2021-08-18T07:20:59Z
dc.date.issued2021-08-02
dc.description.abstractEmerging applications such as those running on the Internet of Things (IoT) devices produce constant data streams that need to be processed in real-time. Distributed stream processing systems (DSPs), with geographically distributed cluster networks interconnected via wide area network (WAN) links, have recently gained interest in handling these applications. However, these applications have stringent requirements such as low-latency and high bandwidth that must be guaranteed to ensure the quality of service (QoS). These application requirements raise fundamental DSPs resource management and scheduling challenge. In this paper, we formulate the problem of placement of worker nodes on a geo-distributed DSPs cluster network as a multi-criteria decision-making problem and propose an additive weighting-based approach to solve it. The proposed solution finds the trade-off among different network parameters and allows executing the tasks according to the desired performance metrics. We evaluated the proposed approach using the Yahoo! streaming benchmark on a testbed and compare it against mechanisms deployed in Apache Spark, Apache Storm, and Apache Flink. The results of the evaluation show that our approach improves the performance of Spark up to 2.2x-7.2x, Storm up to 1.2x-3.4x, and Flink up to 1.4x-3.3x compared to other approaches, which makes our approach useful for use in practical environments.en
dc.description.sponsorshipBMBF, 01IS18025A, Verbundprojekt BIFOLD-BBDC: Berlin Institute for the Foundations of Learning and Dataen
dc.description.sponsorshipBMBF, 01IS18037A, Verbundprojekt BIFOLD-BZML: Berlin Institute for the Foundations of Learning and Dataen
dc.identifier.isbn978-1-7281-9586-5
dc.identifier.isbn978-1-7281-9587-2
dc.identifier.urihttps://depositonce.tu-berlin.de/handle/11303/13496
dc.identifier.urihttp://dx.doi.org/10.14279/depositonce-12279
dc.language.isoenen
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subject.ddc000 Informatik, Informationswissenschaft, allgemeine Werkede
dc.subject.otherIoTen
dc.subject.otherworker node placementen
dc.subject.othergeo-distributed analyticsen
dc.subject.otherstream processingen
dc.subject.othersimple additive weightingen
dc.titleSNR: Network-aware geo-distributed stream analyticsen
dc.typeConference Objecten
dc.type.versionacceptedVersionen
dcterms.bibliographicCitation.doi10.1109/CCGrid51090.2021.00100en
dcterms.bibliographicCitation.originalpublishernameIEEEen
dcterms.bibliographicCitation.originalpublisherplaceNew York, NYen
dcterms.bibliographicCitation.proceedingstitleIEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID)en
dcterms.bibliographicCitation.volume2021en
tub.accessrights.dnbdomain*
tub.affiliationFak. 4 Elektrotechnik und Informatik>Inst. Telekommunikationssysteme>FG Internet Network Architectures (INET)de
tub.affiliation.facultyFak. 4 Elektrotechnik und Informatikde
tub.affiliation.groupFG Internet Network Architectures (INET)de
tub.affiliation.instituteInst. Telekommunikationssystemede
tub.publisher.universityorinstitutionTechnische Universität Berlinen
Files
Original bundle
Now showing 1 - 1 of 1
Loading…
Thumbnail Image
Name:
Mostafaei_etal_SNR_2021_Cover.pdf
Size:
517.07 KB
Format:
Adobe Portable Document Format
Description:
Collections