Multimodal warnings in remote operation: The case study on remote driving

Pekka Kallioniemi, Alisa Burova, John Mäkelä, Tuuli Keskinen, Kimmo Ronkainen, Ville Mäkelä, Jaakko Hakulinen, Markku Turunen

Research output: Contribution to journalArticleScientificpeer-review

15 Downloads (Pure)


Developments in sensor technology, artificial intelligence, and network technologies like 5G has made remote operation a valuable method of controlling various types of machinery. The benefits of remote operations come with an opportunity to access hazardous environments. The major limitation of remote operation is the lack of proper sensory feedback from the machine, which in turn negatively affects situational awareness and, consequently, may risk remote operations. This article explores how to improve situational awareness via multimodal feedback (visual, auditory, and haptic) and studies how it can be utilized to communicate warnings to remote operators. To reach our goals, we conducted a controlled, within-subjects experiment in eight conditions with twenty-four participants on a simulated remote driving system. Additionally, we gathered further insights with a UX questionnaire and semi-structured interviews. Gathered data showed that the use of multimodal feedback positively affected situational awareness when driving remotely. Our findings indicate that the combination of added haptic and visual feedback was considered the best feedback combination to communicate the slipperiness of the road. We also found that the feeling of presence is an important aspect of remote driving tasks, and a requested one, especially by those with more experience in operating real heavy machinery.

Original languageEnglish
Article number44
JournalMultimodal Technologies and Interaction
Issue number8
Publication statusPublished - Aug 2021
Publication typeA1 Journal article-refereed


  • Human factors
  • Multimodal feedback
  • Remote operations

Publication forum classification

  • Publication forum level 1

ASJC Scopus subject areas

  • Neuroscience (miscellaneous)
  • Human-Computer Interaction
  • Computer Science Applications
  • Computer Networks and Communications


Dive into the research topics of 'Multimodal warnings in remote operation: The case study on remote driving'. Together they form a unique fingerprint.

Cite this