Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?

Oleg Spakov; Hanna Venesvirta; Jani Lylykangas; Ahmed Farooq; Roope Raisamo; Veikko Surakka

doi:10.1007/978-3-031-35702-2_24

Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?

Oleg Spakov, Hanna Venesvirta, Jani Lylykangas, Ahmed Farooq, Roope Raisamo, Veikko Surakka

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

1 Downloads (Pure)

Abstract

We studied two interaction techniques to perform secondary tasks in a driving simulator environment with the focus on driving safety. In both techniques, the participants (N = 20) used gaze pointing to select virtual task buttons. Toggling the controls was achieved by either mid-air gestures with haptic feedback or physical buttons located on the steering wheel. To evaluate each technique, we compared several measures, such as mean task times, pedestrian detections, lane deviations, and task complexity ratings. The results showed that both techniques allowed operation without severely compromising driving safety. However, interaction using gestures was rated as more complex, caused more fatigue and frustration, and pedestrians were noticed with longer delays than using physical buttons. The results suggest that gaze pointing accuracy was not always sufficient, while mid-air gestures require more robust algorithms before they can offer functionality comparable to interaction with physical buttons.

Original language	English
Title of host publication	Design, User Experience, and Usability
Subtitle of host publication	12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings
Editors	Aaron Marcus, Elizabeth Rosenzweig, Marcelo M. Soares
Publisher	Springer
Pages	333-352
Number of pages	20
ISBN (Print)	9783031357015
DOIs	https://doi.org/10.1007/978-3-031-35702-2_24
Publication status	Published - 2023
Publication type	A4 Article in conference proceedings
Event	International Conference on Human-Computer Interaction - , Denmark Duration: 23 Jul 2023 → 28 Jul 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14032 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Conference on Human-Computer Interaction
Abbreviated title	HCI International
Country/Territory	Denmark
Period	23/07/23 → 28/07/23

Keywords

in-vehicle interaction
mid-air gestures
multimodal gaze-based interaction
ultrasonic haptic feedback

Publication forum classification

Publication forum level 0

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-35702-2_24

Spakov et al (2023) Multimodal Gaze-based Interaction in Cars are Mid-air Gestures with Haptic Feedback Safer Than ButtonsAccepted author manuscript, 1 MB

https://urn.fi/URN:NBN:fi:tuni-202504033239

Cite this

Spakov, O., Venesvirta, H., Lylykangas, J., Farooq, A., Raisamo, R., & Surakka, V. (2023). Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons? In A. Marcus, E. Rosenzweig, & M. M. Soares (Eds.), Design, User Experience, and Usability: 12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings (pp. 333-352). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14032 LNCS). Springer. https://doi.org/10.1007/978-3-031-35702-2_24

Spakov, Oleg ; Venesvirta, Hanna ; Lylykangas, Jani et al. / Multimodal Gaze-Based Interaction in Cars : Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?. Design, User Experience, and Usability: 12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings. editor / Aaron Marcus ; Elizabeth Rosenzweig ; Marcelo M. Soares. Springer, 2023. pp. 333-352 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a656f29a2b134feab94f4580ede7cdd1,

title = "Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?",

abstract = "We studied two interaction techniques to perform secondary tasks in a driving simulator environment with the focus on driving safety. In both techniques, the participants (N = 20) used gaze pointing to select virtual task buttons. Toggling the controls was achieved by either mid-air gestures with haptic feedback or physical buttons located on the steering wheel. To evaluate each technique, we compared several measures, such as mean task times, pedestrian detections, lane deviations, and task complexity ratings. The results showed that both techniques allowed operation without severely compromising driving safety. However, interaction using gestures was rated as more complex, caused more fatigue and frustration, and pedestrians were noticed with longer delays than using physical buttons. The results suggest that gaze pointing accuracy was not always sufficient, while mid-air gestures require more robust algorithms before they can offer functionality comparable to interaction with physical buttons.",

keywords = "in-vehicle interaction, mid-air gestures, multimodal gaze-based interaction, ultrasonic haptic feedback",

author = "Oleg Spakov and Hanna Venesvirta and Jani Lylykangas and Ahmed Farooq and Roope Raisamo and Veikko Surakka",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; International Conference on Human-Computer Interaction, HCI International ; Conference date: 23-07-2023 Through 28-07-2023",

year = "2023",

doi = "10.1007/978-3-031-35702-2_24",

language = "English",

isbn = "9783031357015",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "333--352",

editor = "Aaron Marcus and Elizabeth Rosenzweig and Soares, {Marcelo M.}",

booktitle = "Design, User Experience, and Usability",

}

Spakov, O , Venesvirta, H , Lylykangas, J , Farooq, A , Raisamo, R & Surakka, V 2023, Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons? in A Marcus, E Rosenzweig & MM Soares (eds), Design, User Experience, and Usability: 12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14032 LNCS, Springer, pp. 333-352, International Conference on Human-Computer Interaction, Denmark, 23/07/23. https://doi.org/10.1007/978-3-031-35702-2_24

Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons? / Spakov, Oleg ; Venesvirta, Hanna ; Lylykangas, Jani et al.
Design, User Experience, and Usability: 12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings. ed. / Aaron Marcus; Elizabeth Rosenzweig; Marcelo M. Soares. Springer, 2023. p. 333-352 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14032 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Multimodal Gaze-Based Interaction in Cars

T2 - International Conference on Human-Computer Interaction

AU - Spakov, Oleg

AU - Venesvirta, Hanna

AU - Lylykangas, Jani

AU - Farooq, Ahmed

AU - Raisamo, Roope

AU - Surakka, Veikko

PY - 2023

Y1 - 2023

N2 - We studied two interaction techniques to perform secondary tasks in a driving simulator environment with the focus on driving safety. In both techniques, the participants (N = 20) used gaze pointing to select virtual task buttons. Toggling the controls was achieved by either mid-air gestures with haptic feedback or physical buttons located on the steering wheel. To evaluate each technique, we compared several measures, such as mean task times, pedestrian detections, lane deviations, and task complexity ratings. The results showed that both techniques allowed operation without severely compromising driving safety. However, interaction using gestures was rated as more complex, caused more fatigue and frustration, and pedestrians were noticed with longer delays than using physical buttons. The results suggest that gaze pointing accuracy was not always sufficient, while mid-air gestures require more robust algorithms before they can offer functionality comparable to interaction with physical buttons.

AB - We studied two interaction techniques to perform secondary tasks in a driving simulator environment with the focus on driving safety. In both techniques, the participants (N = 20) used gaze pointing to select virtual task buttons. Toggling the controls was achieved by either mid-air gestures with haptic feedback or physical buttons located on the steering wheel. To evaluate each technique, we compared several measures, such as mean task times, pedestrian detections, lane deviations, and task complexity ratings. The results showed that both techniques allowed operation without severely compromising driving safety. However, interaction using gestures was rated as more complex, caused more fatigue and frustration, and pedestrians were noticed with longer delays than using physical buttons. The results suggest that gaze pointing accuracy was not always sufficient, while mid-air gestures require more robust algorithms before they can offer functionality comparable to interaction with physical buttons.

KW - in-vehicle interaction

KW - mid-air gestures

KW - multimodal gaze-based interaction

KW - ultrasonic haptic feedback

U2 - 10.1007/978-3-031-35702-2_24

DO - 10.1007/978-3-031-35702-2_24

M3 - Conference contribution

AN - SCOPUS:85169474084

SN - 9783031357015

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 333

EP - 352

BT - Design, User Experience, and Usability

A2 - Marcus, Aaron

A2 - Rosenzweig, Elizabeth

A2 - Soares, Marcelo M.

PB - Springer

Y2 - 23 July 2023 through 28 July 2023

ER -

Spakov O , Venesvirta H , Lylykangas J , Farooq A , Raisamo R , Surakka V. Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons? In Marcus A, Rosenzweig E, Soares MM, editors, Design, User Experience, and Usability: 12th International Conference, DUXU 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Proceedings. Springer. 2023. p. 333-352. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-35702-2_24

Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?

Abstract

Publication series

Conference

Keywords

Publication forum classification

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this