Information about http://sonification.de/publications/media/HuntHermannPauletto2004-IWS.pdf
Interacting with sonification systems: closing…
Tags: abstract objects, adh, andy hunt, bielefeld university, computer systems, control loop, control loops, exploring data, feedback loop, germany university, interaction, interactive control, ohm, perception, sandra, texture, thomas hermann, uni bielefeld, university germany, york ac,Pages: 6
Language: english
Created: Wed Apr 28 11:02:50 2004
Display cached document
Page 1
Page 2
Page 3
Page 4
Page 5
Page 6
Interacting with sonification systems: closing the loop
Andy Hunt, Thomas Hermann, Sandra Pauletto
University of York, UK, Bielefeld University, Germany, University of York, UK
{adh@ohm.york.ac.uk, thermann@techfak.uni-bielefeld.de, sp148@ohm.york.ac.uk }
Abstract objects and obtaining from them instant and continuous
This paper stresses the importance of the human user being feedback of their position, speed and texture.
tightly embedded within an interactive control loop for Such control loops of human action and continuous
exploring data sets using sound. We consider the quality of feedback from the world become embedded deep within
interaction, and how this can be improved in computer systems our mind-body system.
by learning from real-world acoustic interactions. We describe Therefore it is hardly surprising that, later in life, we
how different sonification methods can utilise the human
become rapidly frustrated with computer systems that
feedback loop to enhance the perception and analysis of the
data under investigation. Some considerations are given engage with us in a very different and more limited
regarding systems and applications. manner [1]. Here, too often, the interaction is dictated by
the computer. A prompt is given, or a list of options
1. Introduction presented as icons or a menu. We have to choose from
the selection offered by the computer at every stage of
the process, and thus the interaction becomes a series of
This paper discusses the way that humans interact with
stilted prompt-choice cycles; a far cry from the way that
sound in everyday life. It focuses on how we gain
we have learnt to interact with the everyday world. It is
feedback from a combination of senses, helping us to
as if we have designed our computer systems to always
obtain our sense of reality, and thus to understand the
remain outside our control loop. We seem to expect them
world better. We consider the nature of `Control
always to be under `third-party' control; things to which
Intimacy'; the quality of interaction that we take for
we give instructions. The result of this is that we rarely
granted in manipulating everyday objects, but which is
gain the same intimacy of control with a computer as we
so often lacking in limited-interaction, visual-only
do with objects in everyday life. A common observation
interfaces.
is that much of our time working with computers is spent
Musical instruments are examined as a particular type of
in navigating the interface, rather than completing the
human-device interface (tried and tested over long
task.
periods of time) which allow such control intimacy to
It matters too whether or not you are part of the control
develop to high levels. We then consider the special case
loop. Many passengers become travel sick whereas this
of computer interaction where the human is allowed to
condition rarely affects drivers. When you are
form intimate control loops with the system, using sound
controlling an object you know what to expect, as by
(and other senses) as feedback.
definition you are initiating the reactions and can thus
We outline what can be learnt about integrating
prepare your mental apparatus for the result. Maybe you
interaction into exploratory data analysis techniques. We
have had the experience of being in a room where
then consider some challenging application areas which
someone else is in charge of the TV remote control. You
can be tackled in a fresh way specifically using
cannot believe how much they are `playing around with
continuous interaction with sonic feedback.
it', driving to distraction everyone else in the room.
However when you have it, everything is different, and
2. Control Loops in Human Interaction you are `simply seeing what's on the next channel'. It
matters greatly whether you are in the control loop or
As human beings, from the moment we are born we not.
begin to interact with the world. In fact a baby's first This paper considers how we can bring more real-world
action in the world is to cry to make a sound. As we interaction into our computing interfaces, by placing the
grow we learn first how to control our bodies, and then human operator firmly in charge of a continuous control
how to interact with objects around us. The way that the loop wherever possible.
world works its physical laws, the constants and the
variables becomes coded into our developing brain. 2.1 Control intimacy
We learn to take for granted that dropped objects fall to A child playing with wooden blocks and a person
the ground, and that when we reach for an object we feel operating a typical computer interface are both
it and see it and hear it as we touch it. Watch a young interacting with external objects. It is just that the
child playing with a pile of bricks and you will notice quality of the interaction is different. The extent to
how she develops her movements by interacting with which the interaction directly affects the object is one
aspect of the control intimacy being exhibited; the other initiate actions, and receive constant and immediate
aspect being how well the human manages this control. sonic results.
Real-world objects seem to exhort us to spend time with
them, and as we do, we subconsciously learn more about 2.3 Tuning parameters for individuals
them, and master the skills of manipulating them until An observation about the individuality of interacting
the control becomes almost automatic. with sound became clear to the first author during his
We are all aware of situations where we are controlling experiences as an amateur radio operator. It is well-
an object and almost forget that we are doing it. Car known to `Radio Hams' that there is quite an art to
drivers often report that they are shocked to find `tuning in' the radio to pick out a particularly weak
themselves at their destination, without knowing how signal. Somehow you need to be able to pick out the
they got there; even though the act of driving is an signal you are trying to listen to, in spite of the fact that
extremely complex interactive process. Many good there are much louder interfering signals nearby in the
performing musicians feel that their fingers are somehow frequency spectrum, and background noise, and all
playing the music by themselves. In musical manner of fluctuating signal levels and characteristics
performances, their minds appear to be concentrating on due to propagation conditions. To do this requires a fine
higher-level modes of expression, whilst their bodies are balance with the tuning control, and the signal
managing the physical act of manipulating the modulation controls, and sometimes even movement of
instrument. In fact, most musicians will recount the the antenna. When two people are listening to the same
terrifying feeling of suddenly becoming aware of what radio signal, but only one is at the controls, it is quite
their fingers are doing, and as a result the performance common for the signal to be audible only to the person at
grinds to a halt! the controls.
Csikszentmihalyi [2] called this type of disembodied What can we learn from such an observation? Perhaps
interaction flow. He explains how it is found freely in when a sound is made by a system, we ought to consider
children as they play, and less so in adult life. Certainly who the sound is intended for. Is it just for the person `in
in most computer interfaces the flow is never allowed to the loop', since s/he is the one controlling the system
happen, due to the constant choices, and the stop-start parameters? Or, is the sound intended for everyone?
style of the interaction caused by the emphasis on Where data is being portrayed as sound, for example in a
reading words, processing and selecting options. To hospital environment, it is important that everyone
shed some light on how to improve this state of affairs, recognises the sound. However, where the sound is
let us consider the special case of interaction where the being controlled interactively by a person, we might need
goal is to generate sound. to be aware that the operator could be inadvertently
tuning the system for themselves. More complex sounds
2.2 The special case of interacting with sound (which could appear as annoying or unpleasant) can be
Engineers use sound to deduce the internal state of quite acceptable to people who are in the control loop.
engines and complex machinery such as washing The more general point to be inferred from the above
machines. Sound warns us of dangers outside our example is that humans can use physical interaction to
relatively narrow field of view. It is also the medium by control the generation and modulation of sound in order
which much human communication takes place via to extract data from a noisy signal. Section 3 studies this
speech and singing. area in more detail.
Whenever we interact with a physical object, sound is Musical instruments are a special case of sound
made. It confirms our initial contact with the object, but generating device where the main intention is that other
also tells us about its properties; whether it is solid or people do indeed listen to the sound. Having said that, if
hollow, what material it is made of etc. The sound you are sharing a house with someone practicing an
synchronises with both our visual and tactile `views' of instrument (particularly if the player is a beginner), the
the object. As we move the object, the sounds it makes observation that `it-matters-whether-you-are-in-control-
give us continuous feedback about its state. Sound is a loop' becomes obvious.
temporal indicator of the physical state of the world In the next section we look at human interaction with
around us. instruments in more detail.
The act of making sound may be satisfying to human
beings precisely because they are in a very tightly- 2.4 Musical instruments as exemplars
responsive control loop. This does not by definition The sonic response of physical objects is so deeply
mean that other people find the sound satisfying. Think ingrained in the human psyche that sound and music has
of times when a person mindlessly `drums' his fingers on been a fundamental part of every known human society.
the table to help him think. He is part of the control In this section, we take a closer look at human interaction
loop, and so is expecting the moment-by-moment sonic with musical instruments; since much can be learned
response. The whole process often remains at the from this about what makes good quality real-time
subconscious level, and he is unaware he is doing it. interaction.
However, to other people in the vicinity (not in the loop) In a previous paper [3] we described the attributes of
the sound can be intensely annoying. Therefore, we see most acoustic musical instruments as follows:
that there is something special about being the one to · there is interaction with a physical object.
· co-ordinated hand and finger motions are the purposes of facilitating communication or
crucial to the acoustic output. interpretation" [6]
· the acoustic reaction is instantaneous. Humans must be equipped with several senses for a good
· the sound depends in complex ways on the reason: that they are complementary, and are needed in
detailed kinds of interaction (e.g. on collaboration to gain a full sense of the world around us.
simultaneous positions, velocities, There are several areas where sound offers
accelerations, and pressures). improvements over visual feedback.
The physical interaction with the instrument causes an "The main differences of sound displays over visual
instantaneous acoustic reaction. This allows the player displays are that sound can:
to utilise the everyday object manipulation skills he has · represent frequency responses in an instant (as
developed all his life. The player's energy is directly timbral characteristics)
responsible for activating the sonic response of the · represent changes over time, naturally
system; when the player stops, the sound dies away. The · allow microstructure to be perceived
mapping of system input to sonic output [4] is complex; · rapidly portray large amounts of data
many input parameters are cross-coupled, and connected · alert the listener to events outside the current
in a non-linear manner to the sonic parameters. This can visual focus
make an instrument difficult to play at first, but offers · holistically bring together many channels of
much scope for increased subtlety of control over time. information" [7]
As the player practices, he becomes better and better. So, the use of sound allows us to gain alternative insights
This allows the control intimacy to increase to a level into the data under examination. Until recently the sheer
where the physical operation of the instrument becomes computing power required to generate the sound output
automatic. At this point the player often experiences the has meant that, by necessity, the act of sonification was a
`flow' of thinking at levels much higher than complex non-interactive process. Data was loaded, parameters
physical interface manipulations. were selected, the algorithm set going, and some time
We should also not underestimate the importance of later the sound emerged. Too often in computing
tactile feedback. A good performer will rarely look at technology, when this time-lag is eliminated by
her instrument, but will instead rely on the years of improvements in processor speed, the style of interaction
training, and the continuous feel of the instrument which remains; and interaction is limited to setting parameters,
is tightly coupled to the sound being produced. The then listening to a completed sound. As stated in section
human operator learns to wrap his mind-body system 2, this stilted interaction prevents any form of control
around the instrument to form a human-machine entity. intimacy from developing. In the following section we
So, it seems from considering how people interact with examine how to re-introduce interaction into the art of
musical instruments, that devices intended for making sound.
exploration need to have certain characteristics. These
include a real-time sonic response, a complex control 3.2 Interacting with Sonification
mapping which permits learning, and tactile feedback Now that computers can run fast enough to generate
tightly coupled to the sonic response. sound in real-time, we should re-design our data-to-
sound algorithms to take advantage of the rich
3. The use of sound in exploratory data analysis possibilities of continuous human interaction. How are
In this section we consider the use of sound in computers we to allow a `flow' experience of data sonification to
as a way of understanding data taken from the world take place?
around us. We describe how sound can be used to This question was examined by the 2004 Interactive
portray data, and explain the importance of continuous Sonification workshop, organised by the first two authors
human interaction with the sound generating process. [7] and is summarised in [8]. At this gathering
researchers from diverse disciplines described the magic
3.1 Sonification that occurs when sound is generated in real-time under
human control. Although musical instruments also
The general term Auditory Displays is employed to generate sound in real-time under human control, their
describe the use of sound in computers to portray primary is artistic expression. In contrast, the goal of an
information. It covers a wide range of topics including interactive sonification system is to allow humans to
alarm signals, earcons and sonification techniques, most explore and understand the intrinsic properties of a
of which are discussed by the International Community particular data set. In other words, it is an analysis tool.
for Auditory Display (ICAD) [5]. Sonification is the In sections 4 and 5, we describe toolkits that we have
more specific term used to describe the rendering of data developed which enable such interaction to be explored,
sets as sound, or: and some interactive sonification applications which are
". . the transformation of data relations into in progress at the time of writing. There are two basic
perceived relations in an acoustic signal for approaches to the incorporation of interaction into a
sonification algorithm. The first involves taking data
attributes and converting them into sound (so called
parameter mapping), whilst allowing the user to interact analysis tasks are still to be invented. Model design
with this process. The second involves designing a entails wide possibilities for bringing task-oriented needs
sonification model which is inherently interactive. into the concrete realisation of a model. Complex sound
Where data is time-ordered (for example where it has responses can occur, but humans respond well to this
been gathered from a time-evolving source) it is sensible type of reaction. The MBS concept and its benefits are
to retain this time order by mapping the data onto sound discussed in detail in [9].
variables. Traditionally, the entire data set is converted In this paper we focus on the aspects of engagement and
into a sound file, which is then listened to non- flow, which have been shown to play an important role in
interactively, rather like a CD. However, interaction can the use of interactive auditory systems such as musical
be built into the process to allow a human being to instruments.
explore the data much more freely. For example the The following three aspects of acoustic real-world
position in the data can be moved continuously, interactions cause human users to increase their
`scrubbing' through the data and instantly hearing the engagement with the system:
sonic result. Alternatively the data could play back (i) sound complexity,
continuously in a loop while the sonification algorithm is (ii) low-latency correspondence to human controls,
tuned by the user, rather analogous to the Radio Ham (iii) attention.
example given in section 2.3. Concerning (i), the complexity of sounds from real-
The next section describes how sonification models can world acoustic systems is much higher than that of most
be designed and used for exploring non-time-based data sounds used in computer systems. This is because real-
sets. world systems typically possess complex dynamic
behaviour involving nonlinearities as well as stochastic
3.3 Using model-based sonification to enhance user components, whereas synthesised sounds are often
interaction generated by rather `sterile' algorithms such as FM-
Traditional sonification schemes are based on clearly synthesis. Our auditory system is so well tuned to, and
separated computation and playback phases, as pointed experienced with, analysing real-world sounds that it
out above. In contrast, the framework of Model-based appreciates complexity, often interpreting this as
sonification (MBS) involves "interacting with data- `beauty' of sound. In contrast, even complex stochastic
driven virtual acoustic objects" - which is by design time series generated in computer contexts (e.g. from
inherently interactive. This approach is almost chaotic systems) fail to please or convince the listener.
orthogonal to previous techniques: whereas in parameter For instance plucking a guitar string will never lead to
mapping sonification the data is used to provide controls the exactly same sound, whereas sonification systems
(e.g. playing instructions) for a given instrument (sound typically reproduce sound accurate to the single bit.
synthesis algorithm), in Model-based sonification the Model-based sonification provides exactly this `mind of
data is used to establish the instrument or algorithm its own' to a data-driven dynamical acoustic system.
itself. This means that with an MBS system the user is Since high-quality interactions (those that go far beyond
given the responsibility of interacting with the a simple triggering) are unique excitation patterns, the
sonification model, and (only) by this means causes the resulting sound will also be a unique reaction to this
sonification to generate sound. MBS is thus different unrepeatable stimulus. Sound complexity is not granted
from parameter mapping sonification, in that there is no automatically by the use of an MBS approach. Instead
mapping from data to model-parameters - instead the we need to learn from real-world acoustics, which
data become (in most models) part of the model provides inspiration on suitable ways to create complex
configuration and thus do not explicitly but implicitly sonic dynamics, resulting in sounds where users can rely
determine the sonification. on their highly developed listening skills.
Model-based sonification is a concept in which a virtual Concerning (ii), low-latency is an important factor in
acoustic object is established, dependent on the data creating engagement and for facilitating the user's
under analysis. It thus provides a method of mediating transition from conscious mode to flow mode. Low-
between abstract data spaces and the infinite space of latency sound generation is useful for guiding
possible instruments. Concrete models usually specify exploratory activities since the immediate response
the laws of dynamics that govern the temporal evolution allows the user to directly refine his control activities. It
of the dynamical elements constituting the 'virtual is also important for increasing the synchronisation of
instrument'. Typically sonification models are set up other modalities occurring in the interaction, such as
first to be in a state of equilibrium so that they do not tactile and visual feedback. For example the user hears
produce any sound without being excited into a non- the resulting sound at the same time as they experience
equilibrium state. Most MBS models are dissipative the tactile feedback from the control device.
(because, for example, sound radiation represents energy Concerning (iii), attention; users often focus their
loss) which causes the sound to vanish after interaction attention in order to enhance perception. Think for
ceases. instance of a photograph you are looking at, wondering
Model design is a very creative process. Some example why you took a picture of a boring landscape. Later you
models have been described in previous work, remember that you were taking a photo of a bird. It is
[9][10][11], but the best suited models for specific almost invisible on the picture, but your attention
`magnified' at the time. Attention is the magnifying lens At the University of York we have been developing an
through which users experience and explore the world! Interactive Sonification Toolkit [12] which allows rapid
Attention is often directed towards correlations between prototyping of the transition from data to sound, coupled
the user's activities and a system's response to it. Even with real-time user interaction. It is constructed in PD
faint correlations can then receive significant [13] so that the end product is also cross-platform and
magnification by attention, but only for the user in the open-source. Pd allows real-time sound synthesis,
control loop. Attention is tightly coupled to points (i) creation of graphical user interfaces, refinement of the
and (ii). Complexity of sound grants the availability of `program' during runtime, easy interfacing with many
many possible sources of correlations between sound and sorts of sensors/controllers, e.g. via MIDI or OSC, and
the system feedback. Low-latency is an important factor platform independency.
for ensuring that these correlations are easily detectable This is similar in concept to an interactive sonification
in the interaction. Attention is thus related to a user's platform produced in Bielefeld, based on a graphical
engagement with the system, since the occurrence of simulation system Neo/NST [14], which is particularly
structure on multiple complexity levels keeps alive the strong in data computation, data mining, data
user's interest in practising and improving in the visualisation, and rapid prototyping. However, it is
interaction. weaker in real-time sound synthesis and limited to the
Model-based sonification helps to implement these Linux platform. All sonification models mentioned
aspects automatically since it incorporates an interaction above have been implemented using Neo/NST, using
style which is more like real-world acoustic interactions. Neo displays for graphical data representation and
However, the designer has the freedom to refine the interaction.
sonification so that the aspects mentioned above come Interactive sonification systems have to consist of several
better into play. For instance: components, which (a) need tight interaction, (b) are
- by allowing non-linear couplings of the dynamical computationally expensive, or (c) demand special
elements, so that the sonification model exhibits a platforms. These are expanded below.
rich acoustic behaviour. This may be a) All sonification systems involve data-related
computationally costly but the evolution of computations. Interaction with the display (such as
computation power makes it merely a matter of selection) requires intermediate representations to be
time. recomputed. This demands a tight connection between
- by enhancing the modes of interaction. For instance the controls and the data computation engine. Neo/NST
a sonification model triggered by a computer mouse here provides a good platform, and related alternatives
is `poorer' than one in which users bring in the for powerful data processing are MatLab or Octave.
multi-dimensional controls of a whole articulated b) Specifically for sonification models, CPU power is
hand, which in turn is poorer than interactions with never enough. It is useful to distribute specialised
tangible interfaces that take the user even closer to rendering routines onto an extra machine. Where
real-world acoustic interaction. appropriate, sonification models can be divided into a
- by designing sonification models so that subtle high-level part (where low-level synthesis instructions
changes of the excitation pattern (e.g. of position or are computed) and a low-level sound engine (where the
velocity) are directly related to subtle changes of the sound signal is actually generated). For the first part,
sound. As an example think of a sonification model simulation systems such as Neo/NST or languages like
with which the user can interact by clicking on a Smalltalk are appropriate. For sound computation, there
graphical representation of the data points. One are several candidates, e.g. Csound, PD, or Supercollider.
possible paradigm of model excitation would be to c) The third aspect concerns controls for sonification
give the entire excitation to the nearest data node. In models. Many suitable interfaces, such as an audio-
this case any click within the vicinity of the node haptic ball interface [15] or computer-vision-based
will cause the same sonification. If, however, the gestural interfaces are works-in-progress and demand
excitation energy is distributed between the nearest their own machine. Other interfaces (e.g. certain
neighbours according to their distance, then subtle joysticks, Phantom device, etc.) are only supported on
changes in the activation position will result in special platforms/OS and do not allow a tight direct
subtle changes in the sound. combination.
In this way, we hope that the above aspects prove helpful In summary, it seems that any isolated platform is so far
in the design process of engaging exploratory inappropriate for solving the whole range of problems
sonification models. encountered in interactive multimodal displays. Many
different aspects need to come together in order to enable
4. Software for interactive sonic data analysis a quick and effortless design of systems. Heterogeneous
Sonification systems which allow us to link data sets to solutions not only offer the chance to distribute the
their acoustic representation face several requirements in computation better over several machines, but also to use
terms of interfaces, structure, and performance. Here we optimised components according to the respective needs.
briefly introduce interactive sonification toolkits (ISTs). An intelligent architecture for such complex systems is
We then step back and regard the general requirements currently under development at Bielefeld University and
for software being used in sonification systems. will be presented elsewhere.
5. Applications particularly with sound. We have also summarised the
In this section we briefly outline projects that are in the work done in building interaction into sonification
early stages of development. techniques, and the inherently interactive method of
model-based sonification.
5.1 Analysis of time-stamped data In conclusion, the research community needs to be
At present the PD-based toolkit described above is being acutely aware of the quality of interaction that is
modified to suit several different projects. Two of these provided in human interfaces, in order to maximise the
are funded by EPSRC, allowing advanced data mining of capabilities of the human mind-body system.
helicopter flight data and physiotherapy muscle data
respectively. The data produced by a helicopter test 7. Acknowledgements
flight cannot be adequately shown on a computer screen Thanks to the Sensors Design Consultancy team of ERA
at a reasonable resolution whilst giving an overall Technology for the input and collaboration regarding
picture; so we are using sonification to allow engineers their existing work on audio interfaces for landmine
to navigate the entire data set in a matter of seconds. detection. Thanks to EPSRC for funding the work
Physiotherapists wish to know more about the qualitative described in sections 4 and 5, under the project
aspects of the signals produced by the movement of `Improved data mining through an interactive sonic
muscles, and sound has allowed new insights above and approach'.
beyond the traditional visual plots [12].
5.2 Landmine detection 8. References
The number of anti-personnel landmines buried around [1] A. Hunt, Radical User Interfaces for Real-time Musical
the world is estimated to be between 50 and 70 million. Control, Ph.D. thesis, University of York, 2000,
Their impact on third world countries is devastating in http://www-users.york.ac.uk/~elec18/download/adh_thesis
terms of local economies and their impact on the local [2] M. Csikszentmihalyi, Beyond Boredom and Anxiety:
population. Humanitarian landmine detection and Experiencing Flow in Work and Play, reprint, Jossey
clearance is currently a slow process, because of the high Bass Wiley, 2000.
false alarm rate associated with current detector [3] Hermann, T. & Hunt, A.D, The discipline of Interactive
technology Sonification, Proc. 1st Int. Workshop on Interactive
Sonification, Bielefeld, Germany, January 2004.
Sponsored by the UK Department for International
http://www.interactive-sonification.org
Development, ERA Technology have developed a
[4] A. D. Hunt, M. Paradis, and M. Wanderley, "The
prototype hand-held detector which uses both ground- importance of parameter mapping in electronic
penetrating radar (GPR) and metal detection (MD) to instrument design," Journal of New Music Research, vol.
significantly reduce the effect of false alarms and also 32, no. 4, pp. 429440, December 2003, special issue on
detect minimum metal (plastic) landmines. New Musical Performance and Interaction.
The key factors in the design of the new detector are [5] International Community for Auditory Display
affordability and ease of use. ERA have developed an http://www.icad.org/
audio interface which uses frequencies in the 100Hz to [6] Gregory Kramer et al., Sonification Report, National
3kHz range to give continuous feedback to the user of Science Foundation,1997
the GPR detector. The depth of the target is given by the http://www.icad.org/websiteV2.0/References/nsf.html
frequency of the output signal, and the size of the target [7] http://www.interactive-sonification.org
is given by the amplitude of the signal. [8] Andy Hunt and Thomas Hermann, "The importance of
interaction in sonification," in Proceedings of the Int.
"A key feature of the design is a special
Conf. on Auditory Display. ICAD, 2004, submitted.
(patented) man-machine acoustic interface. This
[9] Thomas Hermann, PhD, Sonification for Exploratory
approach utilises the inherent capabilities of Data Analysis, Bielefeld University, 2002
humans to `process' information and keeps the [10] Hermann, T., Meinicke, P., & Ritter, H. Principal Curve
`man-in-the-loop'." [16] Sonification, Proc. Int. Conf. on Auditory Display p81--
The metal detector also produces its own audio tone, and 86, 2000
together the operator has sonic feedback of the objects on [11] Hermann, T.& Ritter, H. Crystallization Sonification of
and below the surface. High-dimensional Datasets, Proc. Int. Conf. on Auditory
The University of York, Department of Electronics, are Display p76--81, 2002
planning to work with ERA to investigate ways of [12] Pauletto, S & Hunt, A., "An interactive sonification
combining the separate signals from the two sensors, and toolkit" in Proceedings of the Int. Conf. on Auditory
to optimise the presentation of the audio information for Display. ICAD, 2004.
users from many cultural backgrounds. [13] www.pure-data.org
[14] H. Ritter, The Graphical Simulation Toolkit Neo/NST
http://www.techfak.uni-
6. Conclusions
bielefeld.de/ags/ni/projects/neo/neo_e.html
In this paper we have stressed the importance, for [15] Hermann, T., Krause, j., & Ritter, H. Real-Time Control
complex data analysis, of the human user being in a of Sonification Models with an Audio-Haptic Interface,
tightly-coupled intimate control loop. We have Proc. Int. Conf. on Auditory Display p82--86, 2002
illustrated this with examples from everyday interaction, [16] http://www.era.co.uk/docs/electronics/Minetect.pdf