Efficient and Accurate Hand-based Kinesthetic Interaction for Virtual Reality

Research output: Book/ReportDoctoral thesisCollection of Articles

Abstract

Current virtual reality (VR) technology is mainly based on visual and auditive modalities. Kinesthetic interaction, as a form of human-computer interaction, allows users to apply touch-based hand behaviors to interact with virtual objects and simultaneously provides force to simulate the feeling of touch. Implementing such interaction may largely improve the immersion of VR and extend the range of its applications. There are many devices that can be used to implement kinesthetic interaction, such as wearable haptic gloves and grounded force-feedback devices. Among them, grounded force-feedback devices are more mature, which can provide a reliable kinesthetic interface with robust and realistic force feedback. However, there is a major challenge while using force-feedback devices for VR interaction, that is, their small device workspace cannot be directly used to explore a large virtual environment. The current solution to this challenge is to employ a large control-display (CD) gain. While holding the device arm, this technique scales a small hand motion to a large motion of the cursor in the virtual world and thus increases the device workspace.

The aim of this dissertation is to enable efficient and accurate hand-based kinesthetic interaction for VR. The research was divided into three steps: problem understanding, development and application. First, the research was focused on the technique of CD gain. Multiple studies have argued that the mismatch between the hand motion and the cursor motion caused by using the CD gain method may affect kinesthetic interaction. However, it is unclear how it affects kinesthetic tasks in terms of task performance and user experience. The present research filled this gap and examined the effects of CD gain based on a kinesthetic task. Second, to address the issue of limited workspace for force-feedback devices, three multimodal kinesthetic interfaces were developed by using the user’s gaze as an input modality. These novel kinesthetic interfaces avoided the use of CD gain, and they also potentially relieved hand fatigue from prolonged operation. Third, the research explored medical applications of a kinesthetic VR interface and a vibrotactile VR interface associated with current popular VR equipment and force-feedback devices. To explore their practical usability, the two VR interfaces were compared with the state-of-the-art 2D interface (2D display + mouse) as the baseline.

The dissertation provides the following main contributions: First, the research experimentally demonstrated the effects of CD gain on kinesthetic interaction in terms of task performance and user experience. Thus, it provided an empirical basis for designing new kinesthetic interfaces using the CD gain method. Second, the research explored a novel design space for kinesthetic interfaces by using eye gaze as a kinesthetic input modality. It also contributed to understanding human kinesthetic perception in the virtual environment and simultaneously identified the critical factors for designing high-quality kinesthetic interfaces in terms of kinesthetic perception accuracy. Third, the research revealed the strengths and weaknesses of the kinesthetic VR interface and the vibrotactile VR interface. Therefore, it provided an empirical understanding for developing efficient, accurate and user-friendly interactive VR systems. More importantly, the research demonstrated the potential of the kinesthetic VR interface to be the next-generation user interface in the field of medical diagnosis and planning, and it encouraged further research in this area.
Original languageEnglish
Place of PublicationTampere
PublisherTampere University
ISBN (Electronic)978-952-03-2052-2
ISBN (Print)978-952-03-2051-5
Publication statusPublished - 2021
Publication typeG5 Doctoral dissertation (articles)

Publication series

NameDissertations in Interactive Technology
No.35
ISSN (Print)1795-9489

Fingerprint

Dive into the research topics of 'Efficient and Accurate Hand-based Kinesthetic Interaction for Virtual Reality'. Together they form a unique fingerprint.

Cite this