Eye Tracking and Gaze-Based Selection

An eye tracker points a camera at the user’s eyes, estimates the gaze direction, and maps that direction to a point on the screen. On the surface this looks like a pointer with no hands required, and indeed eye tracking is the most independence-restoring input for people who cannot move their limbs at all. But the eye was not built to be a pointer. It is built to take in a scene in jumps, and using it for selection requires a layer of interpretation that turns a noisy, jumpy signal into discrete, intentional commands.

How the gaze point is estimated

The tracker estimates gaze direction from the geometry of the eye, and there are two dominant techniques. The first tracks the pupil: a camera image locates the pupil’s center and, relative to the camera, the direction the eye is looking. Because the pupil center alone is ambiguous when the head moves, the second technique adds a corneal reflection, the glint of an infrared light source reflected off the cornea. The vector between the pupil center and the corneal reflection stays roughly constant for a fixed gaze direction even as the head moves, so the pair of points gives a head-relative gaze estimate. This is the pupil-center corneal-reflection, or PCCR, method, and it is the basis of most commercial trackers.

The raw estimate is then calibrated. The user looks at a sequence of known screen points, the tracker records the pupil-glint vectors at each, and a mapping from those vectors to screen coordinates is fitted. The mapping is a geometric transform, and its accuracy degrades as the head moves away from the calibrated position, which is why trackers either restrict head movement or compensate for it continuously using the measured head pose.

The result is a stream of screen coordinates with two kinds of error. There is systematic error, the offset between the true gaze point and the estimated one, which calibration reduces but never removes, and which is typically on the order of half a degree of visual angle or more. There is also temporal noise, jitter in the estimate frame to frame. Both matter, and the system downstream must cope with both.

Why gaze is a poor pointer

The eye does not move smoothly across a scene the way a mouse moves. It moves in saccades, rapid jumps of up to several hundred degrees per second, separated by fixations, pauses of a few hundred milliseconds during which the eye is relatively still and the brain takes in information. Between saccades there is also microsaccadic drift, small involuntary movements during a fixation. A gaze point derived from this is therefore either jumping between fixations or jittering within one, and neither is a clean pointer signal.

This has a concrete consequence that defeats the naive approach. A user who simply looks at a target and expects it to activate will misfire constantly, because the eye lingers on things it is merely examining. Reading a line of text means fixating word after word, and each fixation would register as a potential selection if gaze were treated as a pointer with a click on contact. The eye looks at something for many reasons, only some of which are “select this”. The selection problem is therefore not estimation but disambiguation: the tracker reports where the user is looking, and the system must decide whether the user means to act.

Dwell as the standard disambiguation

The standard answer is dwell, borrowed from the head-pointer world and adapted to gaze. The user fixates a target, and when the gaze has stayed within a small radius for a configured dwell time, the target activates. The dwell time is the explicit signal of intent: a brief glance means reading, a sustained fixation means selection. The dwell mechanism itself, with its spatial radius and temporal interval, is described in the earlier article on debounce, dwell, and repeat filtering, and gaze reuses it wholesale.

Gaze dwell has two parameters with different effects than in other inputs. The radius must be larger than the microsaccadic jitter, or the gaze never stays inside it long enough to accumulate a full dwell; a radius of roughly a degree of visual angle is a typical order of magnitude. The dwell time trades speed against misfire, and for gaze the misfire is not a mechanical artifact but a cognitive one: a user who is thinking while looking at a target will accidentally hold it long enough to activate it. The Midas touch problem, the curse that everything looked at eventually activates, is the central failure mode of gaze input, and dwell time is the main defense.

Alternatives to dwell

Dwell is slow, because every selection costs a full dwell interval. Several schemes reduce that cost.

Switch-assisted gaze combines the gaze point with a physical switch: the user looks at the target and presses the switch to select it. This removes the dwell wait and the Midas problem entirely, because intent is expressed by the switch rather than by lingering, but it requires the user to have one reliable switch action, which the purely gaze-driven user does not.

Blink and wink detection uses deliberate eye gestures as the selection signal. A deliberate blink is longer than a reflexive one, so the system distinguishes them by duration, the same timing threshold used for hold detection in switch input. Blink selection is slow and physically tiring, because normal blinking must be suppressed to avoid accidental commands, and it is reserved for users who need it.

On-screen dwell targets avoid activating real interface controls directly. The user dwells on a floating panel to choose among actions such as click, scroll, or type, and the chosen action is then applied to the gaze point. This separates the selection of an action from the selection of a target, reducing the chance of acting on the wrong element.

Failure modes and the accuracy floor

Systematic error is the first failure. If the gaze estimate is offset by half a degree, the user looking directly at a small button has a gaze point that misses it, and dwell selects a neighbouring target instead. The remedy is larger targets, which is exactly what the target-size guidance in accessibility standards prescribes, and which is why gaze users benefit disproportionately from controls that meet the minimum target size.

Temporal noise is the second. Jitter around a target’s edge keeps the gaze point crossing the radius boundary, restarting the dwell timer and preventing selection. Smoothing the gaze signal reduces the jitter but adds latency, because smoothing is a low-pass filter over the frame stream, and too much smoothing makes the pointer lag behind the saccade.

The third failure is cognitive rather than technical: the user’s own reading behaviour. A user reading a densely linked page will dwell on links long enough to activate them accidentally, so gaze users must actively suppress the natural tendency to look at what they are reading. The interface cannot fix this; it can only raise the dwell time, which slows every legitimate selection too.

Eye tracking is the extreme end of the assistive-input spectrum: it restores pointing to people with almost no motor control, at the cost of an input signal that is fundamentally noisy and an intent signal that is fundamentally ambiguous. The estimation, the dwell disambiguation, and the accuracy floor together explain both its power and its limits.