Technologies for Multimodal Interaction in Extended Reality—A Scoping Review

Research output: Contribution to journalArticleScientificpeer-review

157 Downloads (Pure)


When designing extended reality (XR) applications, it is important to consider multimodal interaction techniques, which employ several human senses simultaneously. Multimodal interaction can transform how people communicate remotely, practice for tasks, entertain themselves, process information visualizations, and make decisions based on the provided information. This scoping review summarized recent advances in multimodal interaction technologies for head-mounted display-based (HMD) XR systems. Our purpose was to provide a succinct, yet clear, insightful, and structured overview of emerging, underused multimodal technologies beyond standard video and audio for XR interaction, and to find research gaps. The review aimed to help XR practitioners to apply multimodal interaction techniques and interaction researchers to direct future efforts towards relevant issues on multimodal XR. We conclude with our perspective on promising research avenues for multimodal interaction technologies.
Original languageEnglish
Article number81
Number of pages35
JournalMultimodal Technologies and Interaction
Issue number12
Publication statusPublished - 10 Dec 2021
Publication typeA1 Journal article-refereed


  • multimodality
  • mixed reality
  • virtual reality
  • augmented reality
  • extended reality
  • human-computer interaction

Publication forum classification

  • Publication forum level 1


Dive into the research topics of 'Technologies for Multimodal Interaction in Extended Reality—A Scoping Review'. Together they form a unique fingerprint.

Cite this