Caerus: Nimble Task Scheduling for Serverless Analytics

Abstract

Serverless analytics requires optimizing job-completion time and execution cost rather than the utilization and isolation objectives of server-centric frameworks. Caerus addresses this new scheduling problem with the fine-grained NIMBLE algorithm, which pipelines tasks within a job and models pipelineable and non-pipelineable dependencies together with data generation, consumption, and processing rates. Across a broad range of analytics workloads, Caerus achieves both optimal cost and optimal job-completion time in practice.

BibTeX

@inproceedings{caerus,
  author = {Zhang, Hong and Tang, Yupeng and Khandelwal, Anurag and Chen, Jingrong and Stoica, Ion},
  title = {{Caerus: Nimble Task Scheduling for Serverless Analytics}},
  booktitle = {USENIX NSDI},
  month = apr,
  year = {2021}
}