TAI AIS 01: Why construct validity matters in AI-driven education

I am very much honored to speak at the inaugural session of Tokyo AI's AI x Society group. Speaking to a group of people brimming with passion is beyond motivating: it is as if I rediscover my own passion as well. 

Photo taken by a new found friend at the event

For a copy of relevant materials (e.g., presentation, paper) or any questions you may have, please feel free to reach out to me through the Contact Me gadget on this blog's sidebar.

Details

Title: Why construct validity matters in AI-driven education

Authors: May Kristine Jonson Carlon

Venue: Tokyo AI: AI x Society 01

Date: July 1, 2025

Abstract

Many artificial intelligence in education (AIED) systems use superficial signals like click rates or quiz scores as stand-ins for complex constructs such as curiosity, confidence, or understanding. But what are these models actually measuring? In this talk, I will argue that construct validity, or ensuring that a model meaningfully reflects the psychological concept it claims to measure, is still under-addressed in AIED. Misclassifications like labeling deep thinking as disengagement are more than just technical errors; they risk reinforcing problematic norms, punishing atypical learning behaviors, and misinterpreting student needs. I will introduce a practical framework that treats construct validity as a design challenge: one that requires theoretical grounding, transdisciplinary collaboration, and iterative reflection. The goal is to help researchers and developers build models that do not just predict outcomes, but better reflect the learners they aim to support.