Desiderata for next generation of ML model serving

Sherif Akoush; Andrei Paleyes; Arnaud van Looveren; Clive Cox

Back to publications

Desiderata for next generation of ML model serving

Sherif Akoush, Andrei Paleyes, Arnaud van Looveren, Clive Cox

NeurIPS Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (DMML), 2022.

Abstract

Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.

Links

Cite this Paper

BibTeX


@InProceedings{publications/desiderata-for-next-generation-of-ml-model-serving,
  title = 	 {Desiderata for next generation of ML model serving},
  author = 	 {Akoush, Sherif and Paleyes, Andrei and van Looveren, Arnaud and Cox, Clive},
  booktitle = 	 {NeurIPS Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (DMML)},
  year = 	 {2022},
  url = 	 {/publications/desiderata-for-next-generation-of-ml-model-serving.html},
  abstract = 	 {Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.}
}

Endnote

%0 Conference Paper
%T Desiderata for next generation of ML model serving
%A Sherif Akoush
%A Andrei Paleyes
%A Arnaud van Looveren
%A Clive Cox
%B NeurIPS Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (DMML)
%D 2022	
%F publications/desiderata-for-next-generation-of-ml-model-serving
%U /publications/desiderata-for-next-generation-of-ml-model-serving.html
%X Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.

RIS


TY  - CPAPER
TI  - Desiderata for next generation of ML model serving
AU  - Sherif Akoush
AU  - Andrei Paleyes
AU  - Arnaud van Looveren
AU  - Clive Cox
BT  - NeurIPS Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (DMML)
DA  - 2022/10/22	
ID  - publications/desiderata-for-next-generation-of-ml-model-serving
UR  - /publications/desiderata-for-next-generation-of-ml-model-serving.html
AB  - Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.
ER  -

APA


Akoush, S., Paleyes, A., van Looveren, A. & Cox, C.. (2022). Desiderata for next generation of ML model serving. NeurIPS Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (DMML) Available from /publications/desiderata-for-next-generation-of-ml-model-serving.html.