Towards Better Data Discovery and Collection with Flow-Based Programming

Andrei Paleyes; Christian Cabrera; Neil D. Lawrence

Back to publications

Towards Better Data Discovery and Collection with Flow-Based Programming

Andrei Paleyes, Christian Cabrera, Neil D. Lawrence

Neurips Data-Centric AI Workshop (DCAI), 2021.

Abstract

Despite huge successes reported by the field of machine learning, such as voice assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for data-oriented activities. This paper explores the potential of flow-based programming (FBP) for simplifying data discovery and collection in software systems. We compare FBP with the currently prevalent service-oriented paradigm to assess characteristics of each paradigm in the context of ML deployment. We develop a data processing application, formulate a subsequent ML deployment task, and measure the impact of the task implementation within both programming paradigms. Our main conclusion is that FBP shows great potential for providing data-centric infrastructural benefits for deployment of ML. Additionally, we provide an insight into the current trend that prioritizes model development over data quality management.

Links

Cite this Paper

BibTeX


@InProceedings{publications/towards-better-data-discovery-and-collection-with-flow-based-programming,
  title = 	 {Towards Better Data Discovery and Collection with Flow-Based Programming},
  author = 	 {Paleyes, Andrei and Cabrera, Christian and Lawrence, Neil D.},
  booktitle = 	 {Neurips Data-Centric AI Workshop (DCAI)},
  year = 	 {2021},
  url = 	 {/publications/towards-better-data-discovery-and-collection-with-flow-based-programming.html},
  abstract = 	 {Despite huge successes reported by the field of machine learning, such as voice assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for data-oriented activities. This paper explores the potential of flow-based programming (FBP) for simplifying data discovery and collection in software systems. We compare FBP with the currently prevalent service-oriented paradigm to assess characteristics of each paradigm in the context of ML deployment. We develop a data processing application, formulate a subsequent ML deployment task, and measure the impact of the task implementation within both programming paradigms. Our main conclusion is that FBP shows great potential for providing data-centric infrastructural benefits for deployment of ML. Additionally, we provide an insight into the current trend that prioritizes model development over data quality management.}
}

Endnote

%0 Conference Paper
%T Towards Better Data Discovery and Collection with Flow-Based Programming
%A Andrei Paleyes
%A Christian Cabrera
%A Neil D. Lawrence
%B Neurips Data-Centric AI Workshop (DCAI)
%D 2021	
%F publications/towards-better-data-discovery-and-collection-with-flow-based-programming
%U /publications/towards-better-data-discovery-and-collection-with-flow-based-programming.html
%X Despite huge successes reported by the field of machine learning, such as voice assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for data-oriented activities. This paper explores the potential of flow-based programming (FBP) for simplifying data discovery and collection in software systems. We compare FBP with the currently prevalent service-oriented paradigm to assess characteristics of each paradigm in the context of ML deployment. We develop a data processing application, formulate a subsequent ML deployment task, and measure the impact of the task implementation within both programming paradigms. Our main conclusion is that FBP shows great potential for providing data-centric infrastructural benefits for deployment of ML. Additionally, we provide an insight into the current trend that prioritizes model development over data quality management.

RIS


TY  - CPAPER
TI  - Towards Better Data Discovery and Collection with Flow-Based Programming
AU  - Andrei Paleyes
AU  - Christian Cabrera
AU  - Neil D. Lawrence
BT  - Neurips Data-Centric AI Workshop (DCAI)
DA  - 2021/12/14	
ID  - publications/towards-better-data-discovery-and-collection-with-flow-based-programming
UR  - /publications/towards-better-data-discovery-and-collection-with-flow-based-programming.html
AB  - Despite huge successes reported by the field of machine learning, such as voice assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for data-oriented activities. This paper explores the potential of flow-based programming (FBP) for simplifying data discovery and collection in software systems. We compare FBP with the currently prevalent service-oriented paradigm to assess characteristics of each paradigm in the context of ML deployment. We develop a data processing application, formulate a subsequent ML deployment task, and measure the impact of the task implementation within both programming paradigms. Our main conclusion is that FBP shows great potential for providing data-centric infrastructural benefits for deployment of ML. Additionally, we provide an insight into the current trend that prioritizes model development over data quality management.
ER  -

APA


Paleyes, A., Cabrera, C. & Lawrence, N.D.. (2021). Towards Better Data Discovery and Collection with Flow-Based Programming. Neurips Data-Centric AI Workshop (DCAI) Available from /publications/towards-better-data-discovery-and-collection-with-flow-based-programming.html.