Behavioral experiments for understanding catastrophic forgetting

Samuel J. Bell, Neil D. Lawrence
AI Evaluation Beyond Metrics (EBeM), IJCAI, 2022.

Abstract

We explore how the fundamental tool of experimental psychology, the behavioral experiment, can generate insight not only into humans and animals, but artificial systems too. Using catastrophic forgetting as a case study, we present a series of novel experiments with two-layer non-linear networks, and exploratory results revealing a more nuanced understanding of this complex behavior. Alongside our empirical findings, we demonstrate and discuss an alternative, behavior-first approach to investigating emergent neural network phenomena.