Skip to main content

Spark on Entropy: A Reliable & Efficient Scheduler for Low-latency Parallel Jobs in Heterogeneous Cloud

Chen, Huankai, Wang, Frank Z. (2015) Spark on Entropy: A Reliable & Efficient Scheduler for Low-latency Parallel Jobs in Heterogeneous Cloud. In: Local Computer Networks Conference Workshops (LCN Workshops), 2015 IEEE 40th. . IEEE E-ISBN 978-1-4673-6773-8. (doi:10.1109/LCNW.2015.7365918) (KAR id:51357)

PDF
Language: English
Download (423kB) Preview
[img]
Preview
Official URL
https://doi.org/10.1109/LCNW.2015.7365918

Abstract

In heterogeneous cloud, the provision of quality of

more challenging than off-line ones, mainly due to the many

pattern and dynamic query workload. In this paper we propose

parallel analysis as a service more reliable and efficient, and

Entropy, as a measure of the degree of disorder in a system,

and into a chaotic condition, and it can thus serve to measure a

our Entropy Scheduler is to construct the new resource entropy

the help of the new metric so as to provide QoS guarantees for

approach significantly reduces the average query response time

with the native Fair Scheduler in Spark.

Item Type: Conference or workshop item (Paper)
DOI/Identification number: 10.1109/LCNW.2015.7365918
Uncontrolled keywords: Entropy Scheduler, Reliable, Low-latency, Spark, Jobs Scheduling, Heterogeneous Cloud, Cloud Computing
Subjects: Q Science > QA Mathematics (inc Computing science)
Divisions: Faculties > Sciences > School of Computing
Faculties > Sciences > School of Computing > Data Science
Depositing User: Frank Wang
Date Deposited: 02 Nov 2015 19:40 UTC
Last Modified: 06 Feb 2020 04:13 UTC
Resource URI: https://kar.kent.ac.uk/id/eprint/51357 (The current URI for this page, for reference purposes)
Wang, Frank Z.: https://orcid.org/0000-0003-4378-2172
  • Depositors only (login required):

Downloads

Downloads per month over past year