An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context

Andrei Paleyes; Christian Cabrera; Neil D. Lawrence

Back to publications

An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context

Andrei Paleyes, Christian Cabrera, Neil D. Lawrence

1st International Conference on AI Engineering – Software Engineering for AI, 2022.

Abstract

Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.

Links

Cite this Paper

BibTeX


@InProceedings{publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context,
  title = 	 {An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context},
  author = 	 {Paleyes, Andrei and Cabrera, Christian and Lawrence, Neil D.},
  booktitle = 	 {1st International Conference on AI Engineering – Software Engineering for AI},
  year = 	 {2022},
  url = 	 {/publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html},
  abstract = 	 {Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.}
}

Endnote

%0 Conference Paper
%T An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context
%A Andrei Paleyes
%A Christian Cabrera
%A Neil D. Lawrence
%B 1st International Conference on AI Engineering – Software Engineering for AI
%D 2022	
%F publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context
%U /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html
%X Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.

RIS


TY  - CPAPER
TI  - An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context
AU  - Andrei Paleyes
AU  - Christian Cabrera
AU  - Neil D. Lawrence
BT  - 1st International Conference on AI Engineering – Software Engineering for AI
DA  - 2022/05/16	
ID  - publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context
UR  - /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html
AB  - Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.
ER  -

APA


Paleyes, A., Cabrera, C. & Lawrence, N.D.. (2022). An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context. 1st International Conference on AI Engineering – Software Engineering for AI Available from /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html.