JNS 2025: Integrating multimodal MRI data with graph neural networks for latent representation discovery

Ever since I entered RIKEN, I had always felt that I was out of place. It was hard for me to believe that I have what it takes to do brain science research. I still feel that way, but with this poster I presented at JNS 2025, I am starting to think that I may be headed somewhere.

The poster I presented at the JNS 2025

For a copy of relevant materials (e.g., presentation, paper) or any questions you may have, please feel free to reach out to me through the Contact Me gadget on this blog's sidebar.

Details

Title: Integrating multimodal MRI data with graph neural networks for latent representation discovery

Authors: May Kristine Jonson Carlon

Venue: The 48th Annual Meeting of the Japan Neuroscience Society

Date: July 24, 2025

Abstract

Multimodal magnetic resonance imaging (MRI) techniques such as resting-state functional MRI (rs-fMRI), diffusion-weighted imaging (DWI), and structural MRI (sMRI) provide rich data for investigating neural systems, but integrating these modalities into a unified analytical framework to uncover latent representations remains a challenge. We propose constructing multimodal brain graphs, where nodes represent regions of interest (ROIs) with volume information from sMRI and edges encode functional (rs-fMRI) and structural (DWI) connectivity as well as distances between centroids. These graphs will serve as input to graph neural networks (GNNs) trained using contrastive learning to create a shared latent space across modalities. This approach aims to facilitate the discovery of redundancies (distinct mechanisms performing similar functions) and synergies (complementary mechanisms working together) through information-theoretic methods. The primary focus is on optimizing the learning process, including graph construction, contrastive loss functions, and information-theoretic calculations. By leveraging GNNs and contrastive learning, this framework bridges computational techniques and brain data analysis to advance understanding of the collective behavior of neural systems. As this idea is in its conceptual phase, we welcome feedback on its feasibility and implementation strategies. Initial experiments will utilize data from the Human Connectome Project for validation and refinement.