Causal fault localisation in dataflow systems

Andrei Paleyes; Neil D. Lawrence

Back to publications

Causal fault localisation in dataflow systems

Andrei Paleyes, Neil D. Lawrence

Proceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys), 2023.

Abstract

Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines.

Links

Cite this Paper

BibTeX


@InProceedings{publications/causal-fault-localisation-in-dataflow-systems,
  title = 	 {Causal fault localisation in dataflow systems},
  author = 	 {Paleyes, Andrei and Lawrence, Neil D.},
  booktitle = 	 {Proceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys)},
  year = 	 {2023},
  url = 	 {/publications/causal-fault-localisation-in-dataflow-systems.html},
  abstract = 	 {Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines.}
}

Endnote

%0 Conference Paper
%T Causal fault localisation in dataflow systems
%A Andrei Paleyes
%A Neil D. Lawrence
%B Proceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys)
%D 2023	
%F publications/causal-fault-localisation-in-dataflow-systems
%U /publications/causal-fault-localisation-in-dataflow-systems.html
%X Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines.

RIS


TY  - CPAPER
TI  - Causal fault localisation in dataflow systems
AU  - Andrei Paleyes
AU  - Neil D. Lawrence
BT  - Proceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys)
DA  - 2023/05/08	
ID  - publications/causal-fault-localisation-in-dataflow-systems
UR  - /publications/causal-fault-localisation-in-dataflow-systems.html
AB  - Dataflow computing was shown to bring significant benefits to multiple niches of systems engineering and has the potential to become a general-purpose paradigm of choice for data-driven application development. One of the characteristic features of dataflow computing is the natural access to the dataflow graph of the entire system. Recently it has been observed that these dataflow graphs can be treated as complete graphical causal models, opening opportunities to apply causal inference techniques to dataflow systems. In this demonstration paper we aim to provide the first practical validation of this idea with a particular focus on causal fault localisation. We provide multiple demonstrations of how causal inference can be used to detect software bugs and data shifts in multiple scenarios with three modern dataflow engines.
ER  -

APA


Paleyes, A. & Lawrence, N.D.. (2023). Causal fault localisation in dataflow systems. Proceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys) Available from /publications/causal-fault-localisation-in-dataflow-systems.html.