Machine Learning from Innovation to Deployment: A Strategic Research Agenda for AutoAI
Abstract
While excitement about the potential of artificial intelligence (AI) technologies continues to build, a gap is emerging between our aspirations for the benefits of AI and our ability to deploy these technologies to tackle real-world challenges. A significant proportion of attempted AI deployments fail, highlighting a suite of practical issues that arise when trying to integrate AI into real-world systems – from data management and use, to model performance, to user experience. Such failures not only hold back the economic potential of AI, they also expose individuals, communities and societies to new forms of harm.
Links
Cite this Paper
Achieving the full potential of AI – and its benefits for society and the economy – requires the ability to safely and effectively deploy AI systems at scale. As those deploying AI technologies seek to tackle more sophisticated tasks, AI systems are becoming larger and more complex. This complexity gives rise to challenges in the interpretation, explanation, accuracy, and fairness of AI systems. These challenges lie behind many of the failures we see today. They stem from the connection between system complexity and new types of technical and intellectual debt – our ability to deploy complex decision-making systems has increased, but further work is now needed to manage performance in deployment and scrutinise how these systems operate in practice.
Tackling these issues requires fresh approaches to system design that can manage the complex interactions that arise between machine learning components in a decision-making system. AutoAI offers this new perspective. It proposes a pathway to connecting machine learning and AI system design through AI-assisted techniques for system monitoring and maintenance.
The AutoAI Programme at Cambridge Computer Lab scales our ability to deploy safe and reliable AI solutions, driving innovation in machine learning architectures and techniques for deploying, maintaining, understanding, and redeploying AI systems. By investigating how AI systems can be decomposed into their component parts, how data availability and use can be more effectively managed in the development of AI, and how performance in deployment can be monitored, AutoAI will develop a new AI design and engineering paradigm.
Experience of embedding machine learning in large-scale industrial applications has identified integration of machine learning components in established systems as a major point of friction. Better integration requires research advances to develop (1) more sophisticated machine learning models that can be integrated with the wider system; (2) new software architectures and programming techniques to solve technical challenges at scale; (3) a shift towards data-oriented technical infrastructures and organisational practices; and (4) wider organisational, legal and policy interventions to influence AI adoption. AutoAI is addressing the entire pipeline of AI system development, from data acquisition to decision making. In so doing, it is developing policies that enable trustworthy access to data, building effective research collaborations that embed stakeholder needs in research design, and promoting an open culture of AI and software engineering.
This document sets out a strategic research agenda for the AutoAI Programme. It describes the core research themes and activities that the Programme supports, and provides a roadmap for the development of AI-assisted system design and monitoring tools.