Please use this identifier to cite or link to this item:
For citation please use:
Main Title: Exploring Network-Wide Flow Data with Flowyager
Author(s): Saidi, Said Jawad
Maghsoudlou, Aniss
Foucard, Damien
Smaragdakis, Georgios
Poese, Ingmar
Feldmann, Anja
Type: Article
Language Code: en
Abstract: Many network operations, ranging from attack investigation and mitigation to traffic management, require answering network-wide flow queries in seconds. Although flow records are collected at each router, using available traffic capture utilities, querying the resulting datasets from hundreds of routers across sites and over time, remains a significant challenge due to the sheer traffic volume and distributed nature of flow records. In this paper, we investigate how to improve the response time for a priori unknown network-wide queries. We present Flowyager, a system that is built on top of existing traffic capture utilities. Flowyager generates and analyzes tree data structures, that we call Flowtrees, which are succinct summaries of the raw flow data available by capture utilities. Flowtrees are self-adjusted data structures that drastically reduce space and transfer requirements, by 75% to 95%, compared to raw flow records. Flowyager manages the storage and transfers of Flowtrees, supports Flowtree operators, and provides a structured query language for answering flow queries across sites and time periods. By deploying a Flowyager prototype at both a large Internet Exchange Point and a Tier-1 Internet Service Provider, we showcase its capabilities for networks with hundreds of router interfaces. Our results show that the query response time can be reduced by an order of magnitude when compared with alternative data analytics platforms. Thus, Flowyager enables interactive network-wide queries and offers unprecedented drill-down capabilities to, e.g., identify DDoS culprits, pinpoint the involved sites, and determine the length of the attack.
Issue Date: 27-Oct-2020
Date Available: 8-Jun-2021
DDC Class: 000 Informatik, Informationswissenschaft, allgemeine Werke
Subject(s): network data summarization
network monitoring
network-wide traffic analytics
Sponsor/Funder: EC/H2020/679158/EU/Resolving the Tussle in the Internet: Mapping, Architecture, and Policy Making/ResolutioNet
BMBF, 01IS18025A, Verbundprojekt BIFOLD-BBDC: Berlin Institute for the Foundations of Learning and Data
BMBF, 01IS18037A, Verbundprojekt BIFOLD-BZML: Berlin Institute for the Foundations of Learning and Data
Journal Title: IEEE Transactions on Network and Service Management
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Publisher Place: New York, NY
Volume: 17
Issue: 4
Publisher DOI: 10.1109/TNSM.2020.3034278
Page Start: 1988
Page End: 2006
EISSN: 1932-4537
Appears in Collections:FG Internet Measurement and Analysis (IMA) » Publications

Files in This Item:

Accepted manuscript

Format: Adobe PDF | Size: 7.36 MB
DownloadShow Preview

Item Export Bar

Items in DepositOnce are protected by copyright, with all rights reserved, unless otherwise indicated.