Skip to main content
Kent Academic Repository

NetAgg: Using Middleboxes for Application-specific On-path Aggregation in Data Centres

Mai, Luo, Rupprecht, Lukas, Abdul, Alim, Costa, Paolo, Migliavacca, Matteo, Pietzuch, Peter, Wolf, Alexander L. (2014) NetAgg: Using Middleboxes for Application-specific On-path Aggregation in Data Centres. In: CoNEXT '14 Proceedings of the 10th ACM International on Conference on emerging Networking Experiments and Technologies. CoNEXT International Conference on Emerging Networking Experiments and Technologies . pp. 249-262. ACM, New York, USA ISBN 978-1-4503-3279-8. (doi:10.1145/2674005.2674996) (The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided) (KAR id:49289)

The full text of this publication is not currently available from this repository. You may be able to access a copy if URLs are provided.
Official URL:
http://dx.doi.org/10.1145/2674005.2674996

Abstract

Data centre applications for batch processing (e.g. map/reduce frameworks) and online services (e.g. search engines) scale by distributing data and computation across many servers. They typically follow a partition/aggregation pattern: tasks are first partitioned across servers that process data locally, and then those partial results are aggregated. This data aggregation step, however, shifts the performance bottleneck to the network, which typically struggles to support many-to-few, high-bandwidth traffic between servers.

Instead of performing data aggregation at edge servers, we show that it can be done more efficiently along network paths. We describe NETAGG, a software platform that supports on-path aggregation for network-bound partition/aggregation applications. NETAGG exploits a middlebox-like design, in which dedicated servers (agg boxes) are connected by high-bandwidth links to network switches. Agg boxes execute aggregation functions provided by applications, which alleviates network hotspots because only a fraction of the incoming traffic is forwarded at each hop. NETAGG requires only minimal application changes: it uses shim layers on edge servers to redirect application traffic transparently to the agg boxes. Our experimental results show that NETAGG improves substantially the throughput of two sample applications, the Solr distributed search engine and the Hadoop batch processing framework. Its design allows for incremental deployment in existing data centres and incurs only a modest investment cost.

Item Type: Conference or workshop item (Paper)
DOI/Identification number: 10.1145/2674005.2674996
Subjects: Q Science > QA Mathematics (inc Computing science) > QA 76 Software, computer programming,
Divisions: Divisions > Division of Computing, Engineering and Mathematical Sciences > School of Computing
Depositing User: Matteo Migliavacca
Date Deposited: 09 Jul 2015 11:38 UTC
Last Modified: 17 Aug 2022 10:59 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/49289 (The current URI for this page, for reference purposes)

University of Kent Author Information

  • Depositors only (login required):

Total unique views for this document in KAR since July 2020. For more details click on the image.