An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context

Andrei Paleyes; Christian Cabrera; Neil D. Lawrence

Back to publications

An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context

Andrei Paleyes, Christian Cabrera, Neil D. Lawrence

1st International Conference on AI Engineering – Software Engineering for AI.

Abstract

Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.

Links

Cite this Paper

BibTeX


@InProceedings{publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context,
  title = 	 {An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context},
  author = 	 {Paleyes, Andrei and Cabrera, Christian and Lawrence, Neil D.},
  booktitle = 	 {1st International Conference on AI Engineering – Software Engineering for AI},
  url = 	 {/publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html},
  abstract = 	 {Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.}
}

Endnote

%0 Conference Paper
%T An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context
%A Andrei Paleyes
%A Christian Cabrera
%A Neil D. Lawrence
%B 1st International Conference on AI Engineering – Software Engineering for AI	
%F publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context
%U /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html
%X Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.

RIS


TY  - CPAPER
TI  - An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context
AU  - Andrei Paleyes
AU  - Christian Cabrera
AU  - Neil D. Lawrence
BT  - 1st International Conference on AI Engineering – Software Engineering for AI
DA  - false	
ID  - publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context
UR  - /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html
AB  - Despite huge successes reported by the field of machine learning, such as speech assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for activities around data collection and analysis. We propose to consider flow-based programming with data streams as an alternative to commonly used service-oriented architectures for building software applications. To compare flow-based programming with the widespread service-oriented approach, we develop a data processing application, and formulate two subsequent ML-related tasks that constitute a complete cycle of ML deployment while allowing us to assess characteristics of each programming paradigm in the ML context. Employing both code metrics and empirical observations, we show that when it comes to ML deployment each paradigm has certain advantages and drawbacks. Our main conclusion is that while FBP shows great potential for providing infrastructural benefits for deployment of machine learning, it requires a lot of boilerplate code to define and manipulate the dataflow graph. We believe that with better developer tools in place this problem can be alleviated, establishing FBP as a strong alternative to currently prevalent SOA-driven software design approach. Additionally, we provide an insight into the trend of prioritising model development over data quality management.
ER  -

APA


Paleyes, A., Cabrera, C. & Lawrence, N.D.. (false). An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment Context. 1st International Conference on AI Engineering – Software Engineering for AI Available from /publications/an-empirical-evaluation-of-flow-based-programming-in-the-machine-learning-deployment-context.html.

Related People

Andrei Paleyes

Visiting Researcher, Cambridge University

View Profile

Christian Cabrera Jojoa

Assistant Research Professor, Cambridge University

View Profile

Related Projects

AutoAI

While excitement about the potential of artificial intelligence (AI) technologies continues to build, a gap is emerging between our aspirations for the benefits of AI and our ability to deploy these technologies to tackle real-world challenges.

Data-Oriented Architectures for AI-based Systems

Data Oriented Architecture (DOA) is a software architecture pattern that creates data-driven, loosely coupled, decentralised, and open systems. DOA achieves these goals by exposing systems' data as first class citizen to distributed, stateless, and asynchronous systems’ components. These design decisions enable DOA-based systems to achieve desirable properties such as data availability, reusability, and monitoring, as well as systems adaptability, scalability, and autonomy.

Evaluation of dataflow for ML deployment

When deploying machine learning algorithms in real-world systems, software developers face a new set of challenges. This project asks: "What are the underlying reasons for these challenges to arise?", and proposes that the problem lies with the modern trend of building software systems as microservices.

Interfaces

The Interfaces research programme develops interpretable, self-sustaining multi-agent AI systems, treating software as the interface between socio-technical needs and AI capabilities, with validation in healthcare and other high-stakes settings.