Synchrony Between Facial Expressions and Heart Rate Variability During Game-Based Learning: Insights from Cross-Wavelet Transformation

Elizabeth B. Cloude, Muhterem Dindar, Manuel Ninaus, Kristian Kiili

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

7 Downloads (Pure)

Abstract

Game-based learning (GBL) environments are designed to foster emotional experiences conducive to learning; yet, there are mixed findings regarding their effectiveness. The inconsistent results may stem from challenges in measuring and modeling emotions as multi-dimensional constructs during GBL. Traditional approaches often use one data channel and conventional statistics to study emotions, which limit our understanding of the multi-componential interactions that underlie emotional states during GBL. In this study, we merged non-linear dynamical systems (NLDS) theory with the component process model of emotion to examine interactions and synchrony among two emotion signals during GBL, facial expressions and heart rate variability (HRV), and assessed its relation to knowledge and learning gain. Data were collected from 58 participants (n = 58) at a university in Central Finland while they learned about pathology with a tower defense game called Antidote COVID-19. Results showed a significant improvement in knowledge after GBL. A NLDS technique called cross-wavelet transformation showed there were varying degrees of synchrony between facial expressions and HRV. Neutral expressions showed the highest degree of synchrony with HRV, followed closely by happiness and anger with HRV. However, the synchrony between facial expressions and HRV did not affect knowledge and learning gain. This research contributes to the field by studying emotions as multidimensional systems during GLB.

Original languageEnglish
Title of host publicationTechnology Enhanced Learning for Inclusive and Equitable Quality Education
Subtitle of host publication19th European Conference on Technology Enhanced Learning, EC-TEL 2024, Proceedings
EditorsRafael Ferreira Mello, Nikol Rummel, Ioana Jivet, Gerti Pishtari, José A. Ruipérez Valiente
PublisherSpringer
Pages90-104
Number of pages15
Edition1
ISBN (Electronic)9783031723155
ISBN (Print)9783031723148
DOIs
Publication statusPublished - 2024
Publication typeA4 Article in conference proceedings
EventEuropean Conference on Technology Enhanced Learning - Krems, Austria
Duration: 16 Sept 202420 Sept 2024
Conference number: 19
https://ea-tel.eu/ectel2024

Publication series

NameLecture Notes in Computer Science
Volume15159 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Technology Enhanced Learning
Abbreviated titleECTEL
Country/TerritoryAustria
CityKrems
Period16/09/2420/09/24
Internet address

Keywords

  • Emotions
  • Game-based Learning
  • Non-linear Dynamical Systems

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Synchrony Between Facial Expressions and Heart Rate Variability During Game-Based Learning: Insights from Cross-Wavelet Transformation'. Together they form a unique fingerprint.

Cite this