CHI 2025: A Bayesian exploration on the motivational and behavioral impacts of chatbots in language learning

In 2020, CHI held its conference in Yokohama, but I unfortunately was not able to get a paper in. So I am incredibly happy to join this time around! It is a late-breaking work, but considering the back-breaking conditions I had recently, I am totally stoked by it!

The teaser video I created

For a copy of relevant materials (e.g., presentation, paper) or any questions you may have, please feel free to reach out to me through the Contact Me gadget on this blog's sidebar.

Details

Title: A Bayesian exploration on the motivational and behavioral impacts of chatbots in language learning

Authors: May Kristine Jonson Carlon, Julian Matthews, Yasuo Kuniyoshi

Venue: ACM (Association for Computing Machinery) CHI conference on Human Factors in Computing Systems

Date: May 1, 2025

DOI: 10.1145/3706599.3720088

Abstract

This study investigates the motivational and behavioral effects of chatbot interfaces on knowledge tasks in language learning, focusing on English as a Second Language (ESL) learners. As chatbots gain prominence in education, understanding how user factors—metacognitive awareness, help-seeking behavior, and technology acceptance—interact with interface design is critical for optimizing engagement and performance. We conducted an experimental study comparing conversational and informational chatbots during audio-based learning tasks. Bayesian analysis revealed that chatbot type minimally influenced perceived usefulness, with informational chatbots showing slightly higher engagement. Metacognitive awareness strongly predicted help-seeking tendencies, which positively impacted technology acceptance. These findings offer actionable insights for designing AI systems that enhance user engagement and learning outcomes while balancing reliance on automation. Our work contributes to understanding human-AI interaction in educational technology, emphasizing user-centered design's role in fostering equitable and effective learning experiences.