TY - JOUR
T1 - Multimodal warnings in remote operation
T2 - The case study on remote driving
AU - Kallioniemi, Pekka
AU - Burova, Alisa
AU - Mäkelä, John
AU - Keskinen, Tuuli
AU - Ronkainen, Kimmo
AU - Mäkelä, Ville
AU - Hakulinen, Jaakko
AU - Turunen, Markku
N1 - Funding Information:
Funding: This research was funded by Business Finland (grant number 251/31/2018), FIMA ry (Forum for Intelligent Machines), and the research parties University of Tampere (UTA, now TUNI) and Tampere University of Technology (TUT, now TUNI).
Publisher Copyright:
© 2021 by the authors. Licensee MDPI, Basel, Switzerland.
PY - 2021/8
Y1 - 2021/8
N2 - Developments in sensor technology, artificial intelligence, and network technologies like 5G has made remote operation a valuable method of controlling various types of machinery. The benefits of remote operations come with an opportunity to access hazardous environments. The major limitation of remote operation is the lack of proper sensory feedback from the machine, which in turn negatively affects situational awareness and, consequently, may risk remote operations. This article explores how to improve situational awareness via multimodal feedback (visual, auditory, and haptic) and studies how it can be utilized to communicate warnings to remote operators. To reach our goals, we conducted a controlled, within-subjects experiment in eight conditions with twenty-four participants on a simulated remote driving system. Additionally, we gathered further insights with a UX questionnaire and semi-structured interviews. Gathered data showed that the use of multimodal feedback positively affected situational awareness when driving remotely. Our findings indicate that the combination of added haptic and visual feedback was considered the best feedback combination to communicate the slipperiness of the road. We also found that the feeling of presence is an important aspect of remote driving tasks, and a requested one, especially by those with more experience in operating real heavy machinery.
AB - Developments in sensor technology, artificial intelligence, and network technologies like 5G has made remote operation a valuable method of controlling various types of machinery. The benefits of remote operations come with an opportunity to access hazardous environments. The major limitation of remote operation is the lack of proper sensory feedback from the machine, which in turn negatively affects situational awareness and, consequently, may risk remote operations. This article explores how to improve situational awareness via multimodal feedback (visual, auditory, and haptic) and studies how it can be utilized to communicate warnings to remote operators. To reach our goals, we conducted a controlled, within-subjects experiment in eight conditions with twenty-four participants on a simulated remote driving system. Additionally, we gathered further insights with a UX questionnaire and semi-structured interviews. Gathered data showed that the use of multimodal feedback positively affected situational awareness when driving remotely. Our findings indicate that the combination of added haptic and visual feedback was considered the best feedback combination to communicate the slipperiness of the road. We also found that the feeling of presence is an important aspect of remote driving tasks, and a requested one, especially by those with more experience in operating real heavy machinery.
KW - Human factors
KW - Multimodal feedback
KW - Remote operations
U2 - 10.3390/mti5080044
DO - 10.3390/mti5080044
M3 - Article
AN - SCOPUS:85113280963
SN - 2414-4088
VL - 5
JO - Multimodal Technologies and Interaction
JF - Multimodal Technologies and Interaction
IS - 8
M1 - 44
ER -