Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network

Isaac Sebenius; Alexander Campbell; Sarah E. Morgan; Edward T. Bullmore; Pietro Liò

doi:10.1109/MLSP52302.2021.9690626

Back to publications

Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network

Isaac Sebenius, Alexander Campbell, Sarah E. Morgan, Edward T. Bullmore, Pietro Liò

IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), 2021.

Abstract

Graph neural networks (GNN s) are a powerful class of model for representation learning on relational data and graph-structured signal, such as brain connectivity graphs derived from neuroimaging. To date, existing work applying graph learning methods to brain connectivity is limited to a single neuroimaging modality such as structural or functional MRI. In practice, the brain is best represented by multiple networks arising from different imaging modalities. We develop a gen-eral framework for jointly pooling multimodal graphs which share the same set of underlying nodes whilst differing in edge connectivity. Building on this approach, we propose a multimodal GNN (MM-GNN) model that incorporates mul-tiple types of neuroimaging-based brain connectivity. When applied to the task of classifying brain images from patients with schizophrenia and healthy control subjects, we observe that incorporating multimodal pooling dramatically improves performance over non-pooled networks and that MM-GNN matches or improves performance over multiple single-modal and non-GNN baselines. Finally, we demonstrate how our approach uses multimodal data to learn a unified, interpretable measure of the salience of individual brain regions of interest. In this way, MM-GNN represents a new method for leveraging diverse brain connectivity data to enhance the detection of mental health disorders and to understand their biological underpinnings.

Links

Cite this Paper

BibTeX


@InProceedings{publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network,
  title = 	 {Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network},
  author = 	 {Sebenius, Isaac and Campbell, Alexander and Morgan, Sarah E. and Bullmore, Edward T. and Liò, Pietro},
  booktitle = 	 {IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)},
  year = 	 {2021},
  doi = 	 {10.1109/MLSP52302.2021.9690626},
  url = 	 {/publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network.html},
  abstract = 	 {Graph neural networks (GNN s) are a powerful class of model for representation learning on relational data and graph-structured signal, such as brain connectivity graphs derived from neuroimaging. To date, existing work applying graph learning methods to brain connectivity is limited to a single neuroimaging modality such as structural or functional MRI. In practice, the brain is best represented by multiple networks arising from different imaging modalities. We develop a gen-eral framework for jointly pooling multimodal graphs which share the same set of underlying nodes whilst differing in edge connectivity. Building on this approach, we propose a multimodal GNN (MM-GNN) model that incorporates mul-tiple types of neuroimaging-based brain connectivity. When applied to the task of classifying brain images from patients with schizophrenia and healthy control subjects, we observe that incorporating multimodal pooling dramatically improves performance over non-pooled networks and that MM-GNN matches or improves performance over multiple single-modal and non-GNN baselines. Finally, we demonstrate how our approach uses multimodal data to learn a unified, interpretable measure of the salience of individual brain regions of interest. In this way, MM-GNN represents a new method for leveraging diverse brain connectivity data to enhance the detection of mental health disorders and to understand their biological underpinnings.}
}

Endnote

%0 Conference Paper
%T Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network
%A Isaac Sebenius
%A Alexander Campbell
%A Sarah E. Morgan
%A Edward T. Bullmore
%A Pietro Liò
%B IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)
%D 2021	
%F publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network
%R 10.1109/MLSP52302.2021.9690626
%U /publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network.html
%X Graph neural networks (GNN s) are a powerful class of model for representation learning on relational data and graph-structured signal, such as brain connectivity graphs derived from neuroimaging. To date, existing work applying graph learning methods to brain connectivity is limited to a single neuroimaging modality such as structural or functional MRI. In practice, the brain is best represented by multiple networks arising from different imaging modalities. We develop a gen-eral framework for jointly pooling multimodal graphs which share the same set of underlying nodes whilst differing in edge connectivity. Building on this approach, we propose a multimodal GNN (MM-GNN) model that incorporates mul-tiple types of neuroimaging-based brain connectivity. When applied to the task of classifying brain images from patients with schizophrenia and healthy control subjects, we observe that incorporating multimodal pooling dramatically improves performance over non-pooled networks and that MM-GNN matches or improves performance over multiple single-modal and non-GNN baselines. Finally, we demonstrate how our approach uses multimodal data to learn a unified, interpretable measure of the salience of individual brain regions of interest. In this way, MM-GNN represents a new method for leveraging diverse brain connectivity data to enhance the detection of mental health disorders and to understand their biological underpinnings.

RIS


TY  - CPAPER
TI  - Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network
AU  - Isaac Sebenius
AU  - Alexander Campbell
AU  - Sarah E. Morgan
AU  - Edward T. Bullmore
AU  - Pietro Liò
BT  - IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP)
DA  - 2021/10/25	
ID  - publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network
DO  - 10.1109/MLSP52302.2021.9690626
UR  - /publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network.html
AB  - Graph neural networks (GNN s) are a powerful class of model for representation learning on relational data and graph-structured signal, such as brain connectivity graphs derived from neuroimaging. To date, existing work applying graph learning methods to brain connectivity is limited to a single neuroimaging modality such as structural or functional MRI. In practice, the brain is best represented by multiple networks arising from different imaging modalities. We develop a gen-eral framework for jointly pooling multimodal graphs which share the same set of underlying nodes whilst differing in edge connectivity. Building on this approach, we propose a multimodal GNN (MM-GNN) model that incorporates mul-tiple types of neuroimaging-based brain connectivity. When applied to the task of classifying brain images from patients with schizophrenia and healthy control subjects, we observe that incorporating multimodal pooling dramatically improves performance over non-pooled networks and that MM-GNN matches or improves performance over multiple single-modal and non-GNN baselines. Finally, we demonstrate how our approach uses multimodal data to learn a unified, interpretable measure of the salience of individual brain regions of interest. In this way, MM-GNN represents a new method for leveraging diverse brain connectivity data to enhance the detection of mental health disorders and to understand their biological underpinnings.
ER  -

APA


Sebenius, I., Campbell, A., Morgan, S.E., Bullmore, E.T. & Liò, P.. (2021). Multimodal Graph Coarsening for Interpretable, MRI-Based Brain Graph Neural Network. IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP) doi:10.1109/MLSP52302.2021.9690626 Available from /publications/multimodal-graph-coarsening-for-interpretable-mri-based-brain-graph-neural-network.html.