Eye Movement and Gaze in 3D

Our eyes are not just a sensor that we use to see but also a responder that continuously optimises vision with involuntary eye movements. For interaction in 3D we are interested in VOR (vestibulo-ocular reflex) eye movements that stabilise vision when we look at objects while we move our head and body and in vergence eye movements that adjust our vision to changes in focal depth of what we are looking at.

Gaze in 3D not only involves a wider range of eye movements but also exposed new design problems. In prior work we have for example explored placement of subtitles which is challenging: if they are placed at the focal depth of objects the user is looking at then they may be occluded by objects in the scene that are nearer, and if they are placed close up to avoid occlusion then they require the user to shift focus back and forth [1].

We have also explored gaze and eye movement for selection of objects at different depth. Depending on the user’s point of view, multiple objects can be positioned at different depth along their line of sight, resulting in ambiguity as to which object the user is attending to. In one approach, we are highlighting candidate objects and display motion around their outline to use pursuit eye movement for disambiguation [2]. In a different approach, we have shown how VOR eye movement can be utilised to estimate gaze depth. When the users move their head while looking at an object, the eyes rotate in the opposite direction to keep gaze steady. As the eyes are closer to the object than the head, they need to rotate faster, resulting in VOR gain. The VOR gain depends on the focal depth and can therefore be used to determine whether the user is looking that is nearer or farther in the scene [3,4].

  1. Subtitles in Interactive Virtual Reality: Using Gaze to Address Depth Conflicts
    Ludwig Sidenmark, Nicolas Kiefer and Hans Gellersen
    Workshop on Emerging Novel Input Devices and Interaction Techniques, IEEE VR ’19

  2. Outline Pursuits: Gaze-assisted Selection of Occluded Objects in Virtual Reality
    Ludwig Sidenmark, Christopher Clarke, Xuesong Zhang, Jenny Phu, and Hans Gellersen
    CHI ’20: ACM Conference on Human Factors in Computing Systems

  3. Monocular gaze depth estimation using the vestibulo-ocular reflex
    Diako Mardanbegi, Christopher Clarke, and Hans Gellersen. 2019
    ETRA '19: ACM Symposium on Eye Tracking Research & Applications

  4. Resolving Target Ambiguity in 3D Gaze Interaction through VOR Depth Estimation
    Diako Mardanbegi, Tobias Langlotz, and Hans Gellersen
    CHI ’19: ACM Conference on Human Factors in Computing Systems