Information about http://www.almaden.ibm.com/u/zhai/papers/magic/magic.pdf
Manual And Gaze Input Cascaded (MAGIC) Pointing …
Tags: almaden research center, computer input, core element, eye gaze, ibm almaden research center, large portion, manipulation, many compelling reasons, morimoto, new direction, popular press, research literature, san jose ca, shumin zhai, target acquisition, user interface, vision 1,Pages: 8
Language: english
Created: Fri Jun 11 18:57:20 1999
Display cached document
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Page 7
Page 8
Manual And Gaze Input Cascaded (MAGIC) Pointing
Shumin Zhai Carlos Morimoto Steven Ihde
IBM Almaden Research Center
650 Harry Road
San Jose, CA 95120 USA
+1 408 927 1112
{zhai, morimoto, ihde}@almaden.ibm.com
only in the research literature, but also in the popular press,
ABSTRACT
such as the July 1996 issue of Byte magazine [5]. One of the
This work explores a new direction in utilizing eye gaze for basic goals that numerous researchers have attempted to
computer input. Gaze tracking has long been considered as an achieve is to operate the user interface through eye gaze, with
alternative or potentially superior pointing method for pointing (target acquisition) as the core element. There are
computer input. We believe that many fundamental many compelling reasons to motivate such a goal, including
limitations exist with traditional gaze pointing. In particular, the following:
it is unnatural to overload a perceptual channel such as vision 1. There are situations that prohibit the use of the hands,
with a motor control task. We therefore propose an such as when the user's hands are disabled or
alternative approach, dubbed MAGIC (Manual And Gaze Input continuously occupied with other tasks.
Cascaded) pointing. With such an approach, pointing appears
to the user to be a manual task, used for fine manipulation 2. The eye can move very quickly in comparison to other
and selection. However, a large portion of the cursor parts of the body. Furthermore, as many researchers have
movement is eliminated by warping the cursor to the eye gaze long argued [3] [6], target acquisition usually requires
area, which encompasses the target. Two specific MAGIC the user to look at the target first, before actuating cursor
pointing techniques, one conservative and one liberal, were control. Theoretically this means that if the eye gaze can
designed, analyzed, and implemented with an eye tracker we be tracked and effectively used, no other input method
developed. They were then tested in a pilot study. This early- can act as quickly. Increasing the speed of user input to
stage exploration showed that the MAGIC pointing techniques the computer has long been an interest of HCI research.
might offer many advantages, including reduced physical 3. Reducing fatigue and potential injury caused by
effort and fatigue as compared to traditional manual pointing, operating keyboard and pointing devices is also an
greater accuracy and naturalness than traditional gaze important concern in the user interface field. Repetitive
pointing, and possibly faster speed than manual pointing. The stress injury affects an increasing number of computer
pros and cons of the two techniques are discussed in light of users. Most users are not concerned with RSI until
both performance data and subjective reports. serious problems occur. Utilizing eye gaze movement to
replace or reduce the amount of stress to the hand can be
Keywords beneficial.
Gaze, eye, computer input, eye tracking, gaze tracking, Clearly, to replace "what you see (and click on) is what you
pointing, multi-modal interface, Fitts' law, computer vision. get" with "what you look at is what you get" [4] [6] has
captivating appeal. However, the design and implementation
INTRODUCTION of eye gaze-based computer input has been faced with two
types of challenges. One is eye tracking technology itself,
which will be briefly discussed in the Implementation section
of the paper. The other challenge is the human factor issues
Using the eyes as a source of input in "advanced user involved in utilizing eye movement for computer input. Jacob
interfaces" has long been a topic of interest to the HCI field [7] eloquently discussed many of these issues with insightful
[1] [2] [3] [4]. Reports on eye tracking frequently appear not observations.
In our view, there are two fundamental shortcomings to the
In Proc. CHI'99: ACM Conference on Human Factors in existing gaze pointing techniques, regardless of the maturity
Computing Systems. 246-253, Pittsburgh, 15-20 May1999 of eye tracking technology. First, given the one-degree size
Copyright ACM 1999 0-201-48559-1/99/05...$5.00 of the fovea and the subconscious jittery motions that the eyes
constantly produce, eye gaze is not precise enough to operate
UI widgets such as scrollbars, hyperlinks, and slider handles
on today's GUI interfaces. At a 25-inch viewing distance to control of the cursor by hand near (or on) the target, or ignore
the screen, one degree of arc corresponds to 0.44 in, which is it and search for the next target. Operationally, a new object
twice the size of a typical scroll bar and much greater than the
Gaze position True target will be
size of a typical character.
reported by eye within the circle with
Second, and perhaps more importantly, the eye, as one of our tracker 95% probability
primary perceptual devices, has not evolved to be a control
organ. Sometimes its movements are voluntarily controlled
while at other times it is driven by external events. With the
target selection by dwell time method, considered more The cursor is
natural than selection by blinking [7], one has to be conscious warped to eye
Eyetracking tracking position,
of where one looks and how long one looks at an object. If boundary with
one does not look at a target continuously for a set threshold which is on or near
95% confidence the true target
(e.g., 200 ms), the target will not be successfully selected. On
the other hand, if one stares at an object for more than the set
threshold, the object will be selected, regardless of the user's
intention. In some cases there is not an adverse effect to a Previous cursor position,
far from target (e.g., 200
false target selection. Other times it can be annoying and
pixels)
counter-productive (such as unintended jumps to a web page).
Furthermore, dwell time can only substitute for one mouse Figure 1. The liberal MAGIC pointing technique:
click. There are often two steps to target activation. A single cursor is placed in the vicinity of a target that the user
click selects the target (e.g., an application icon) and a double fixates on.
click (or a different physical button click) opens the icon
is defined by sufficient distance (e.g., 120 pixels) from the
(e.g., launches an application). To perform both steps with
current cursor position, unless the cursor is in a controlled
dwell time is even more difficult.
motion by hand. Since there is a 120-pixel threshold, the
In short, to load the visual perception channel with a motor cursor will not be warped when the user does continuous
control task seems fundamentally at odds with users' natural manipulation such as drawing. Note that this MAGIC pointing
mental model in which the eye searches for and takes in technique is different from traditional eye gaze control, where
information and the hand produces output that manipulates the user uses his eye to point at targets either without a cursor
external objects. Other than for disabled users, who have no [7] or with a cursor [3] that constantly follows the jittery eye
alternative, using eye gaze for practical pointing does not gaze motion.
appear to be very promising.
The liberal approach may appear "pro-active," since the
MAGIC POINTING cursor waits readily in the vicinity of or on every potential
target. The user may move the cursor once he decides to
Are there interaction techniques that utilize eye movement to acquire the target he is looking at. On the other hand, the user
assist the control task but do not force the user to be overly may also feel that the cursor is over-active when he is merely
conscious of his eye movement? We wanted to design a looking at a target, although he may gradually adapt to ignore
technique in which pointing and selection remained primarily this behavior.
a manual control task but were also aided by gaze tracking.
Our key idea is to use gaze to dynamically redefine (warp) The more conservative MAGIC pointing technique we have
the "home" position of the pointing cursor to be at the explored does not warp a cursor to a target until the manual
vicinity of the target, which was presumably what the user input device has been actuated. Once the manual input device
was looking at, thereby effectively reducing the cursor has been actuated, the cursor is warped to the gaze area
movement amplitude needed for target selection. Once the reported by the eye tracker. This area should be on or in the
cursor position had been redefined, the user would need to vicinity of the target. The user would then steer the cursor
only make a small movement to, and click on, the target with manually towards the target to complete the target
a regular manual input device. In other words, we wanted to acquisition.
achieve Manual And Gaze Input Cascaded (MAGIC) pointing, As illustrated in Figure 2, to minimize directional uncertainty
or Manual Acquisition with Gaze Initiated Cursor. There are after the cursor appears in the conservative technique, we
many different ways of designing a MAGIC pointing technique. introduced an "intelligent" bias. Instead of being placed at the
Critical to its effectiveness is the identification of the target center of the gaze area, the cursor position is offset to the
the user intends to acquire. We have designed two MAGIC intersection of the manual actuation vector and the boundary
pointing techniques, one liberal and the other conservative in of the gaze area. This means that once warped, the cursor is
terms of target identification and cursor placement. likely to appear in motion towards the target, regardless of
The liberal approach is to warp the cursor to every new object how the user actually actuated the manual input device. We
the user looks at (See Figure 1). The user can then take hoped that with the intelligent bias the user would not have to
actuate input device, observe the cursor position and decide 3. A more natural mental model for the user. The user does
in which direction to steer the cursor. The cost to this method not have to be aware of the role of the eye gaze. To the
is the increased manual movement amplitude. user, pointing continues to be a manual task, with a
cursor conveniently appearing where it needs to be.
Gaze position True target will be 4. Speed. Since the need for large magnitude pointing
reported by eye within the circle operations is less than with pure manual cursor control, it
tracker with 95% is possible that MAGIC pointing will be faster than pure
probability manual pointing.
5. Improved subjective speed and ease-of-use. Since the
manual pointing amplitude is smaller, the user may
Eyetracking The cursor is perceive the MAGIC pointing system to operate faster and
boundary with 95% warped to the more pleasantly than pure manual control, even if it
confidence boundary of the operates at the same speed or more slowly.
gaze area, along the The fourth point warrants further discussion. According to the
initial actuation well accepted Fitts' Law [8], manual pointing time is
Initial manual vector
actuation vector logarithmically proportional to the A/W ratio, where A is the
movement distance and W is the target size. In other words,
Previous cursor targets which are smaller or farther away take longer to
position, far from target acquire. For MAGIC pointing, since the target size remains the
same but the cursor movement distance is shortened, the
Figure 2. The conservative MAGIC pointing technique pointing time can hence be reduced.
with "intelligent offset"
It is less clear if eye gaze control follows Fitts' Law. In Ware
To initiate a pointing trial, there are two strategies available and Mikaelian's study [3], selection time was shown to be
to the user. One is to follow "virtual inertia:" move from the logarithmically proportional to target distance, thereby
cursor's current position towards the new target the user is conforming to Fitts' Law. To the contrary, Silbert and Jacob
looking at. This is likely the strategy the user will employ, [9] found that trial completion time with eye tracking input
due to the way the user interacts with today's interface. The increases little with distance, therefore defying Fitts' Law.
alternative strategy, which may be more advantageous but
takes time to learn, is to ignore the previous cursor position In addition to problems with today's eye tracking systems,
and make a motion which is most convenient and least such as delay, error, and inconvenience, there may also be
effortful to the user for a given input device. For example, on many potential human factor disadvantages to the MAGIC
a small touchpad, the user may find it convenient to make an pointing techniques we have proposed, including the
upward stroke with the index finger, causing the cursor to following:
appear below the target. 1. With the more liberal MAGIC pointing technique, the
The goal of the conservative MAGIC pointing method is the cursor warping can be overactive at times, since the
following. Once the user looks at a target and moves the input cursor moves to the new gaze location whenever the eye
device, the cursor will appear "out of the blue" in motion gaze moves more than a set distance (e.g., 120 pixels)
towards the target, on the side of the target opposite to the away from the cursor. This could be particularly
initial actuation vector. In comparison to the liberal approach, distracting when the user is trying to read. It is possible
this conservative approach has both pros and cons. While to introduce additional constraint according to the
with this technique the cursor would never be over-active and context. For example, when the user's eye appears to
jump to a place the user does not intend to acquire, it may follow a text reading pattern, MAGIC pointing can be
require more hand-eye coordination effort. automatically suppressed.
Both the liberal and the conservative MAGIC pointing 2. With the more conservative MAGIC pointing technique,
techniques offer the following potential advantages: the uncertainty of the exact location at which the cursor
might appear may force the user, especially a novice, to
1. Reduction of manual stress and fatigue, since the cross- adopt a cumbersome strategy: take a touch (use the
screen long-distance cursor movement is eliminated from manual input device to activate the cursor), wait (for the
manual control. cursor to appear), and move (the cursor to the target
2. Practical accuracy level. In comparison to traditional manually). Such a strategy may prolong the target
pure gaze pointing whose accuracy is fundamentally acquisition time. The user may have to learn a novel
limited by the nature of eye movement, the MAGIC hand-eye coordination pattern to be efficient with this
pointing techniques let the hand complete the pointing technique.
task, so they can be as accurate as any other manual input
techniques.
3. With pure manual pointing techniques, the user, knowing The Almaden system uses two near infrared (IR) time
the current cursor location, could conceivably perform multiplexed light sources, composed of two sets of IR LED's,
his motor acts in parallel to visual search. Motor action which were synchronized with the camera frame rate. One
may start as soon as the user's gaze settles on a target. light source is placed very close to the camera's optical axis
With MAGIC pointing techniques, the motor action and is synchronized with the even frames. Odd frames are
computation (decision) cannot start until the cursor synchronized with the second light source, positioned off-
appears. This may negate the time saving gained from axis. The two light sources are calibrated to provide
the MAGIC pointing technique's reduction of movement approximately equivalent whole-scene illumination. Pupil
amplitude. detection is realized by means of subtracting the dark pupil
Clearly, experimental (implementation and empirical) work is image from the bright pupil image. After thresholding the
needed to validate, refine, or invent alternative MAGIC difference, the largest connected component is identified as
pointing techniques. the pupil. This technique significantly increases the
robustness and reliability of the eye tracking system. After
IMPLEMENTATION implementing our system with satisfactory results, we
discovered that similar pupil detection schemes had been
We took two engineering efforts to implement the MAGIC independently developed by Tomono et al [13] and Ebisawa
pointing techniques. One was to design and implement an eye and Satoh [14]. It is unfortunate that such a method has not
tracking system and the other was to implement MAGIC been used in the commercial systems. We recommend that
pointing techniques at the operating systems level, so that the future eye tracking product designers consider such an
techniques can work with all software applications beyond approach.
"demonstration" software.
Once the pupil has been detected, the corneal reflection (the
The IBM Almaden Eye Tracker glint reflected from the surface of the cornea due to one of the
light sources) is determined from the dark pupil image. The
Since the goal of this work is to explore MAGIC pointing as a reflection is then used to estimate the user's point of gaze in
user interface technique, we started out by purchasing a terms of the screen coordinates where the user is looking at.
commercial eye tracker (ASL Model 5000) after a market The estimation of the user's gaze requires an initial
survey. In comparison to the system reported in early studies calibration procedure, similar to that required by commercial
(e.g. [7]), this system is much more compact and reliable. eye trackers.
However, we felt that it was still not robust enough for a
variety of people with different eye characteristics, such as Our system operates at 30 frames per second on a Pentium II
pupil brightness and correction glasses. We hence chose to 333 MHz machine running Windows NT. It can work with
develop and use our own eye tracking system [10]. Available any PCI frame grabber compatible with Video for Windows.
commercial systems, such as those made by ISCAN Implementing MAGIC pointing
Incorporated, LC Technologies, and Applied Science We programmed the two MAGIC pointing techniques on a
Laboratories (ASL), rely on a single light source that is Windows NT system. The techniques work independently
positioned either off the camera axis in the case of the ISCAN from the applications. The MAGIC pointing program takes data
ETL-400 systems, or on-axis in the case of the LCT and the from both the manual input device (of any type, such as a
ASL E504 systems. Illumination from an off-axis source (or mouse) and the eye tracking system running either on the
ambient illumination) generates a dark pupil image. When the same machine or on another machine connected via serial
light source is placed on-axis with the camera optical axis, the port.
camera is able to detect the light reflected from the interior of
the eye, and the image of the pupil appears bright [11] [12] Raw data from an eye tracker can not be directly used for
(see Figure 3). This effect is often seen as the red-eye in flash gaze-based interaction, due to noise from image processing,
photographs when the flash is close to the camera lens. eye movement jitters, and samples taken during saccade
(ballistic eye movement) periods. We experimented with
various filtering techniques and found the most effective filter
in our case is similar to that described in [7]. The goal of
filter design in general is to make the best compromise
between preserving signal bandwidth and eliminating
unwanted noise. In the case of eye tracking, as Jacob argued,
eye information relevant to interaction lies in the fixations.
The key is to select fixation points with minimal delay.
Figure 3. Bright (left) and dark (right) pupil images Samples collected during a saccade are unwanted and should
resulting from on- and off-axis illumination. The glints, be avoided. In designing our algorithm for picking points of
or corneal reflections, from the on- and off-axis light fixation, we considered our tracking system speed (30 Hz),
sources can be easily identified as the bright points in
and that the MAGIC pointing techniques utilize gaze
the iris.
information only once for each new target, probably
immediately after a saccade. Our filtering algorithm was pointing is a touchpad: the user can choose one convenient
designed to pick a fixation with minimum delay by means of gesture and to take advantage of the intelligent offset.
selecting two adjacent points over two samples. The experimental task was essentially a Fitts' pointing task.
Subjects were asked to point and click at targets appearing in
EXPERIMENT
random order. If the subject clicked off-target, a miss was
Empirical studies, such as [3], are relatively rare in eye logged but the trial continued until a target was clicked. An
tracking-based interaction research, although they are extra trial was added to make up for the missed trial. Only
particularly needed in this field. Human behavior and trials with no misses were collected for time performance
processes at the perceptual motor level often do not conform analyses. Subjects were asked to complete the task as quickly
to conscious-level reasoning. One usually cannot correctly as possible and as accurately as possible. To serve as a
describe how to make a turn on a bicycle. Hypotheses on motivator, a $20 cash prize was set for the subject with the
novel interaction techniques can only be validated by shortest mean session completion time with any technique.
empirical data. However, it is also particularly difficult to
conduct empirical research on gaze-based interaction
techniques, due to the complexity of eye movement and the
lack of reliability in eye tracking equipment. Satisfactory
results only come when "everything is going right." When
results are not as expected, it is difficult to find the true
reason among many possible reasons: Is it because a subject's
particular eye property fooled the eye tracker? Was there a
calibration error? Or random noise in the imaging system? Or
is the hypothesis in fact invalid?
We are still at a very early stage of exploring the MAGIC
pointing techniques. More refined or even very different
techniques may be designed in the future. We are by no
means ready to conduct the definitive empirical studies on Figure 4. Experimental task: point at paired targets
MAGIC pointing. However, we also feel that it is important to
subject our work to empirical evaluations early so that The task was presented on a 20 inch CRT color monitor, with
quantitative observations can be made and fed back to the a 15 by 11 inch viewable area set at resolution of 1280 by
iterative design-evaluation-design cycle. We therefore 1024 pixels. Subjects sat from the screen at a distance of 25
decided to conduct a small-scale pilot study to take an initial inches.
peek at the use of MAGIC pointing, however unrefined. The following factors were manipulated in the experiments:
Experimental Design · two target sizes: 20 pixels (0.23 in or 0.53 degree of
viewing angle at 25 in distance) and 60 pixels in
The two MAGIC pointing techniques described earlier were diameter (0.7 in, 1.61 degree)
put to test using a set of parameters such as the filter's
· three target distances: 200 pixels (2.34 in, 5.37 degree),
temporal and spatial thresholds, the minimum cursor warping
500 pixels (5.85 in, 13.37 degree), and 800 pixels (9.38
distance, and the amount of "intelligent bias" (subjectively
in, 21.24 degree)
selected by the authors without extensive user testing).
Ultimately the MAGIC pointing techniques should be evaluated · three pointing directions: horizontal, vertical and
with an array of manual input devices, against both pure diagonal
manual and pure gaze-operated pointing methods (in the case A within-subject design was used. Each subject performed
of large targets suitable for gaze pointing). Since this is an the task with all three techniques: (1) Standard, pure manual
early pilot study, we decided to limit ourselves to one manual pointing with no gaze tracking (No_Gaze); (2) The
input device. A standard mouse was first considered to be the conservative MAGIC pointing method with intelligent offset
manual input device in the experiment. However, it was soon (Gaze1); (3) The liberal MAGIC pointing method (Gaze2).
realized not to be the most suitable device for MAGIC pointing, Nine subjects, seven male and two female, completed the
especially when a user decides to use the push-upwards experiment. The order of techniques was balanced by a Latin
strategy with the intelligent offset. Because in such a case the square pattern. Seven subjects were experienced TrackPoint
user always moves in one direction, the mouse tends to be users, while two had little or no experience.
moved off the pad, forcing the user adjust the mouse position,
often during a pointing trial. We hence decided to use a With each technique, a 36-trial practice session was first
miniature isometric pointing stick (IBM TrackPoint IV, given, during which subjects were encouraged to explore and
commercially used in the IBM Thinkpad 600 and 770 series to find the most suitable strategies (aggressive, gentle, etc.).
notebook computers). Another device suitable for MAGIC The practice session was followed by two data collection
sessions.
Although our eye tracking system allows head motion, at least true target. Such distance depends on eye tracking system
for those users who do not wear glasses, we decided to use a accuracy, which is unrelated to the previous cursor position.
chin rest to minimize instrumental error.
Experimental Results 1.8
Given the pilot nature and the small scale of the experiment, 1.6
we expected the statistical power of the results to be on the 1.4
weaker side. In other words, while the significant effects 1.2
revealed are important, suggestive trends that are statistically 1
non-significant are still worth noting for future research. No_Gaze
.8
First, we found that subjects' trial completion time Gaze1
.6
significantly varied with techniques: F(2, 16) = 6.36, p < Gaze2
.4
0.01. The total average completion time was 1.4 seconds with
the standard manual control technique (No_Gaze in Figure .2
5), 1.52 seconds with the conservative MAGIC pointing 0
technique (Gaze1), and 1.33 seconds with the liberal MAGIC small(20) large(60)
W
pointing technique (Gaze2). Note that the Gaze1 technique
had the greatest improvement from the first to the second Figure 6. Mean completion time (sec) vs. target size
experiment session, suggesting the possibility of matching the (pixels)
performance of the other two techniques with further practice.
1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1 No_Gaze
No_Gaze .8
.8 Gaze1
Gaze1 .6
.6 Gaze2
Gaze2 .4
.4
.2
.2 0
0 short(200) mid(500) long(800)
Session1 Session2 A
Figure 5. Mean completion time (sec) vs. experiment Figure 7. Mean completion time (sec) vs. pointing
session amplitude (pixels)
As expected, target size significantly influenced pointing In short, while completion time and target distance with the
time: F(1,8) = 178, p < 0.001. This was true for both the MAGIC pointing techniques did not completely follow Fitts'
manual and the two MAGIC pointing techniques (Figure 6). Law, they were not completely independent either. Indeed,
Pointing amplitude also significantly affected completion when we lump target size and target distance according to the
time: F(2, 8) = 97.5, p < 0.001. However, the amount of Fitts' Law Index of Difficulty ID = log2(A/W + 1) [15], we
influence varied with the technique used, as indicated by the see a similar phenomonon.
significant interaction between technique and amplitude: F(4, For the No_Gaze condition:
32) = 7.5, p < 0.001 (Figure 7). As pointing amplitude
increased from 200 pixels to 500 pixels and then to 800 T = 0.28 + 0.31 ID (rē=0.912)
pixels, subjects' completion time with the No_Gaze condition The particular settings of our experiment were very different
increased in a non-linear, logarithmic-like pace as Fitts' Law from those typically reported in a Fitts' Law experiment: to
predicts. This is less true with the two MAGIC pointing simulate more realistic tasks we used circular targets
techniques, particularly the Gaze2 condition, which is distributed in varied directions in a randomly shuffled order,
definitely not logarithmic. Nonetheless, completion time with instead of two vertical bars displaced only in the horizontal
the MAGIC pointing techniques did increase as target distance dimension. We also used an isometric pointing stick, not a
increased. This is intriguing because in MAGIC pointing mouse. Considering these factors, the above equation is
techniques, the manual control portion of the movement reasonable. The index of performance (IP) was 3.2 bits per
should be the distance from the warped cursor position to the second, in comparison to the 4.5 bits per second in a typical
setting (repeated mouse clicks on two vertical bars) [16].
For the Gaze1 condition: slower (4.3%) than those using pure manual pointing (1.41
T = 0.8 + 0.22 ID (rē=0.716) seconds). The US$20 cash prize was claimed by a subject
whose shortest mean completion time was 1.03 second,
IP = 4.55 bits per second achieved with the Gaze2 technique. The closest runner up
For Gaze2: was 1.05 second, also achieved with the Gaze2 technique.
Although some users performed in fact slower with the new
T = 0.6 + 0.21 ID (rē=0.804)
techniques, subjectively they tended to feel faster with MAGIC
IP = 4.76 bits per second pointing techniques. On a 5 (most unfavorable) to +5 (most
Note that the data from the two MAGIC pointing techniques fit favorable) scale, subjects gave an average rating of 1.5
the Fitts' Law model relatively poorly (as expected), although (spread from -1 to +3) to the Gaze1 technique and 3.5 (from
the indices of performance (4.55 and 4.76 bps) were much 2 to 4.5) to the Gaze2 technique. The overall positive
higher than the manual condition (3.2 bps). reaction from the users could be due to any of the following
factors: 1) The novel experience, which may or may not be
Finally, Figure 8 shows that the angle at which the targets
fundamentally beneficial; 2) the reduced physical effort.
were presented had little influence on trial completion time:
Users might have liked the fact that a big chunk of the
F(2, 16) = 1.57, N.S.
physical task was done automatically. Some subjects were
disappointed after the MAGIC pointing sessions when they
realized that the cursor would no longer move to the vicinity
1.6
of the target "by itself."
1.4
The targets used in the experiment varied from small (0.53
1.2 degree) to large (1.6 degree), resembling realistic targets in
1 practice. Notably, the traditional gaze pointing technique
works well only for large targets (2.0 by 1.6 degree in [3] and
.8
deteriorated rapidly when target was smaller than 1.5 degree.
.6
The reduced fatigue from pure manual pointing is self-
.4 evident, simply because less cursor movement is needed.
.2 On the other hand, the speed advantage, when there was one,
0 was not obvious. It is undoubtedly possible to improve the
Horizontal Diagonal Vertical performance of the MAGIC pointing techniques. First, many
aspects of the proposed techniques can be refined, including
No_Gaze Gaze1 Gaze2 optimizing the parameters in the gaze system's filter and in
the MAGIC pointing techniques themselves. The input device
transfer function was designed to accommodate both large
Figure 8. Mean completion time (sec) vs. target angle and small cursor movements. It is possible to optimize the
(degrees) transfer function for MAGIC pointing techniques. Second, the
The number of misses (clicked off target) was also analyzed. engineering aspects of the eye tracking system may also be
The only significant factor to the number of misses is target improved. Many subjects commented that the eye tracker
size: F(1,8) = 15.6, p < 0.01. Users tended to have more performance varied over time, probably due to their head
misses with small targets. More importantly, subjects made motions during the session. In order to achieve the best
no more misses with the MAGIC pointing techniques than with results, we turned off the camera's servo mode and used a
the pure manual technique (No_Gaze 8.2 %, Gaze1 7%, chin-rest. Some subjects did not stay steady in the chin rest as
Gaze2 7.5%). asked. Some subjects also noticed the delay in the tracking
system, which depended on how quickly a pair of samples
DISCUSSIONS AND CONCLUSIONS was detected, which in turn depends on noise in the system.
In the ideal case, the delay can be as small as one sampling
The performance data from this pilot study shows both
period (33 ms). Other times it may take several samples to
promises and shortcomings with the very first implementation
find a pair of adjacent points.
of MAGIC pointing techniques. First, the MAGIC pointing
approach actually worked. All subjects were able to operate In summary, the pros and cons of the two techniques were
the two novel techniques with minimal instruction. By the end demonstrated both in the performance data and in subjects'
of the experiment, subjects had less than 10 minutes of comments. The conservative MAGIC pointing method was
exposure to each technique, but were able to perform at a truly "conservative." Its average speed was slower than the
speed similar to their manual control skills. In the second "liberal" and the manual technique, although such a
session of the experiment, on average, subjects using the difference tended to shrink with practice. Some subjects
liberal MAGIC pointing technique performed slightly faster commented that the conservative technique required more
(6.8%) and those using the conservative technique slightly effort to coordinate the timing of eye-hand cooperation.
Others found it less distracting and more "discreet" than they Conference on Human Factors in Computing Systems and
found the liberal technique. Some also pointed out that it took Graphics Interface. 1987. Toronto.
them several trials to get used to the conservative technique, 4. Jacob, R.J.K. What You Look At is What You Get: Eye
specifically the uncertainty of not knowing exactly where the Movement-Based Interaction Techniques. in CHI'90: ACM
cursor would appear. Interestingly, the intelligent offset, Conference on Human Factors in Computing Systems. 1990:
designed to reduce the directional uncertainty, was not Addison-Wesley/ACM Press.
unnoticed by some users who pointed out that the
conservative technique had greater "tracking error": the 5. Joch, A., What Pupils Teach Computers. Byte, 1996(July):
cursor was farther from the target. Clearly we need to further p. 99-100.
test the conservative technique without the offset. 6. Jacob, R.J.K., The Use of Eye Movements in Human-
Overall subjects liked the liberal technique better for its Computer Interaction Techniques: What You Look At is What
responsiveness. This may change in a more realistic setting You Get. ACM Transactions on Information Systems,, 1991.
where pointing is mixed with other tasks, in which case the vol. 9(no. 3): p. 152-169.
more discreet conservative technique may become more 7. Jacob, R.J.K., Eye Movement-Based Human-Computer
favorable. Interaction Techniques: Toward Non-Command Interfaces,
Based on the results of this pilot experiment, we are refining in Advances in Human-Computer Interaction, H.R. Hartson
the proposed MAGIC technique. Alternative techniques may and D. Hix, Editors. 1993, Ablex Publishing Co.,: Norwood,
also be designed in future research. N.J. p. 151-190.
The IBM Almaden Gaze tracker described in the paper points 8. Fitts, P.M., The information capacity of the human motor
to the rapid improvement in eye tracking technology. The system in controlling the amplitude of movement. Journal of
price (and size) of commercial eye tracking equipment has Experimental Psychology, 1954. 47: p. 381-391.
dropped significantly in the last decade, from over US$100k 9. Silbert, L. and R. Jacob, The Advantage of Eye Gaze
to around US$20k. Our system hardware cost was around Interaction, . 1996: Unpublished manuscript.
US$2000 (US$200 for camera and US$1500 for frame 10. Morimoto, C., et al., Pupil detection and tracking using
grabber), in addition to the computer (which also ran the multiple light sources, . 1998, IBM Almaden Research
applications used by subjects). As computer power and the Center: San Jose.
price of cameras and video processing hardware continue to
exponentially improve, it is conceivable that in the future 11. Young, L. and D. Sheena, Methods & designs: Survey of
mainstream computers will all be equipped with technology eye movement recording methods. Behavioral Research
similar to that which we used in this experiment. Such a Methods & Instrumentation, 1975. 7(5): p. 397--429.
prospect calls for continued, in-depth research on eye-based 12. Hutchinson, T.E., et al., Human-computer interaction
interaction techniques. This work attempts to serve as one using eye-gaze input. IEEE Transactions on Systems, Man,
stepping stone in this process. and Cybernetics, 1989. 19(Nov.Dec): p. 1527--1533.
ACKNOWLEDGMENTS
13. Tomono, A., I. Muneo, and Y. Kobayashi. A TV camera
system which extracts feature points for non-contact eye
This study was conducted as part of the IBM Blue Eyes movement detection. iSPIE vol. 1194. Optics, Illumination,
project, led by Myron Flickner, who provided us great and Image Sensing for Machine Vision IV. 1989.
support. Barton Smith developed the Fitts' Law testing
14. Ebisawa, Y. and S. Satoh. Effectiveness of pupil area
program used in the experiment. Dragutin Petkovic was a
detection technique using two light sources and image
constant source of support and inspiration. Dave Koons, Rob
difference method. 15th Annual Int. Conf. of the IEEE Eng.
Barrett, and Arnon Amir contributed to brainstorming
in Medicine and Biology Society. 1993. San Diego, CA.
discussions; and Johnny Accot provided much-needed help
with data processing and analysis. 15. MacKenzie, I.S., Fitts' law as a research and design tool
in human computer interaction. Human Computer
REFERENCES Interaction, 1992. 7: p. 91-139.
1. Levine, J.L., An Eye-Controlled Computer, . 1981, IBM TJ 16. MacKenzie, I.S., A. Sellen, and W. Buxton. A
Watson Research Center: Yorktown Heights, New York. comparison of input devices in elemental pointing and
dragging tasks. in CHI'91: ACM Conference on Human
2. Bolt, R.A. Eyes at the interface. in Human Factors in
Factors in Computing Systems. 1991. New Orleans,
Computer Systems. 1982. Gaithersburg, Maryland: ACM.
Louisiana.
3. Ware, C. and H.H. Mikaelian. An evaluation of an eye
tracker as a device for computer input. in CHI+GI: ACM