Will Your Digital Butlers Betray You? …

                                Will Your Digital Butlers Betray You?

                                                                           Frank Stajano
                                                                     University of Cambridge


ABSTRACT                                                                             is always the sword of Damocles that some intrusive data
The cost of data storage is now so low that there is little                          mining may occur retroactively at a later date.
necessity ever to delete anything. The consequence is denied                            The time shift inherent in denied oblivion is responsible
oblivion--digital systems that remember forever and can be                           for another major threat, namely that data acquired for one
data-mined retroactively, years after the event, ignoring any                        purpose by one agent will later be accessed and searched for
privacy promise under which the original data may have                               another purpose by another agent. The regulations under
been acquired.                                                                       which data was acquired may have changed; the agent that
  Even for systems under your own control, though, the sit-                          originally acquired the data may have gone out of business;
uation is alarming. As your capacious digital butlers faith-                         but the data itself is still there, ready to be mined by its new
fully collect as much data as possible about you, your private                       owner who, in practice and despite theoretical claims to the
information is increasingly likely to become compromised.                            contrary, is no longer bound by the original rules. Captured
  New solutions are needed. But technical countermeasures                            data tends to have a much longer lifetime than the privacy
alone are not the whole story.                                                       policy under which it was captured1 .
                                                                                        The interception-of-communications scenario, while not
Categories and Subject Descriptors: K.4.1 [Computers                                 unrealistic, is just one of many possible examples. The com-
And Society]: Public Policy Issues--Privacy.                                         mercial world offers many more, from supermarket loyalty
General Terms: Security.                                                             cards to adware that spies on your web browsing habits.
Keywords: Denied Oblivion, Data Mining.                                              Garfinkel's Database Nation will raise your awareness on the
                                                                                     extent to which data collection and dossier building already
                                                                                     takes place in today's society.
1.     DENIED OBLIVION                                                                  Perhaps one of the most worrying aspects of the prob-
  Processor speeds have increased by three orders of mag-                            lem is the apathy of the general public towards it: most
nitude (1, 000×) over the past 20 years. A more significant                          people can be bribed out of their shopping privacy by the
though less frequently glorified improvement has occurred                            1% discount offered by the loyalty card. Except for small
in mass storage capacity: the size of today's hard disks has                         vocal minorities, the complacency of the public extends to
increased by about 4.5 orders of magnitude (30, 000×) over                           much more intrusive developments such as the ubiquitous
the same time span. It is now possible to collect data to an                         Minority Report-style CCTV surveillance to which people
extent that was previously unthinkable.                                              are subjected in the UK, or the US-VISIT program under
  Thirty years ago, no three-letter agency would have had                            which all the non-American attendees of this "Workshop on
the budget to monitor all the international telephone traffic                        Privacy in the Electronic Society" were fingerprinted and
of a country with the population of Canada. Yet a rough                              photographed at their port of arrival in the US2 . Both prac-
estimate, the numerical details of which have been omitted                           tices are dismissed by many honest members of the public
in this concise position paper, shows that a 10 M$ server                            as a small price to pay in order to ensure "the safety of the
farm could now transcribe all that speech into text in real                          country".
time. Much more significant, though, is the fact that a whole                           "Denied oblivion" simply means that those fingerprints
month's worth of searchable full-text transcripts would fit                          taken from you at the airport will stay on file forever. Poli-
into a single 300 GB hard disk. The running costs of storage                         cies may come and go, but this acquisition will never be
have become practically nil.                                                         undone.
  There is no economic requirement ever to delete anything.
Whatever was once digitized is now stored forever. This                              2.   A CONTROVERSIAL DIGITAL BUTLER
property, which I shall call denied oblivion, is the source
                                                                                       Intrusive data acquisition about you by other parties is
of many new privacy problems. The privacy violation may
                                                                                     not necessarily the most worrying development. Consider
not occur right now; but, since everything is logged, there
                                                                                     also legitimate data acquisition about you by devices under
                                                                                     your control, later misused against you by adversaries who
                                                                                     take over these devices against your wish.
Permission to make digital or hard copies of all or part of this work for            1
personal or classroom use is granted without fee provided that copies are              Assuming, optimistically, that one was in place at the time
not made or distributed for profit or commercial advantage and that copies           of acquisition.
                                                                                     2
bear this notice and the full citation on the first page. To copy otherwise, to        This practice, introduced in January 2004 for citizens of
republish, to post on servers or to redistribute to lists, requires prior specific   certain nations, was widened to most of the remaining ones
permission and/or a fee.                                                             in October 2004. It would not be entirely illogical to predict
WPES'04, October 28, 2004, Washington, DC, USA.                                      that something similar will eventually extend to domestic
Copyright 2004 ACM 1-58113-968-3/04/0010 ...$5.00.                                   citizens too.
   Let me introduce a hypothetical recording device--call it       and operated by the secret police, who will also beat you up
"Omnirec"--that would forever store everything you hear.           if you attempt to turn it off.
This is feasible today: modern digital dictaphones already            Encrypting the content before storing it, with a key only
compress voice-grade audio to 10 MB/h, so the 24×7 stor-           known to the wearer, is an obvious first attempt towards a
age requirement amounts to less than 100 GB/year. A few            solution. Unfortunately it still leaves the wearer at liberty
years from now, a portable Omnirec would also be capable           to reveal the content to third parties, therefore not altering
of transcribing and indexing all speech, thereby making it         the status of the Omnirec as a potential spying device. It
searchable; and it might be capable of recording video as well     also doesn't prevent a determined adversary from obtaining
as audio. With an only slightly more daring stretch of the         the content from the wearer--it just forces this adversary to
imagination, the Omnirec might eventually sample your au-          resort to more convincing attacks, of the kind in which the
ditory and visual neurons rather than external microphones         locution "brute force" reverts to its literal meaning as op-
and cameras. This invention would be a wonderful mem-              posed to the one usually attributed to it by cryptographers.
ory prosthesis: you would be able to recall, instantly and            Various steganographic solutions have been proposed to
accurately, any event at which you had been present.               prevent the extraction of encrypted data under duress. None
   This invention would also, however, raise a number of           is entirely satisfactory when the threat model includes tak-
serious privacy concerns, both towards others and towards          ing you to Orwell's "Room 101".
you. The most significant difference between the prosthetic
and the wetware memory has to do with transferability of           3.   SUMMING UP:
content from the original viewer to other persons. With
standard human memory you can tell a third party what you               THE RIGHT TO REMAIN SILENT
saw and heard; but this indirect and imperfect report is quite        The threats to privacy are changing.
different from a video. There is also the significant difference      From a technological standpoint, the spectacular improve-
that the recipient has no reliable way to distinguish between      ments in capacity and affordability of mass storage are re-
the objective facts and your own (intentional or unconscious)      sponsible for denied oblivion: data, once acquired, is never
additions and omissions.                                           forgotten. Since data will outlive the privacy policy under
   Imagine you visit my home, where all walls are covered          which it was acquired, retroactive data mining is a first,
in books. Later, by reviewing past Omnirec footage at              obvious privacy problem.
your leisure, you could compile a list of all the thousands           Our growing reliance on digital butlers, from cellphones
of books in all the rooms that you visited, learning the titles    to car navigation systems, means that more and more data
of many more of my books than you could possibly have no-          nuggets about us are being logged on an ongoing basis, with
ticed on your own. Imagine further that someone asked you          ever-increasing temporal, spatial and semantic resolution.
whether I have an embarrassing or controversial title--say         This in itself would not be a privacy threat if not for the
Mein Kampf or How to build an atom bomb. You didn't                fact that your digital butlers can be "forced to speak" with
notice it when you visited; but with the Omnirec you could         much less effort and risk than that needed to force you to
search all the books that entered your field of vision, instead    say something you wouldn't. The more our butlers become
of just saying "I don't know". If you found it, you could then     privy to intimate information about our us and our lives,
show this third party the video frames of the book on my           the more serious this threat becomes. Transferring informa-
shelf--a much more convincing and damning report than              tion from brain to butler enhances availability but threatens
just "I saw it". (And let's not get into forgeries. . . )          confidentiality.
   It is easy to see that showing Omnirec media to a third            Sometimes people tolerate surveillance and intrusion in
party would therefore make both of you guilty of eavesdrop-        exchange for the promise of greater security. Wouldn't the
ping. As a matter of fact, under many jurisdictions, if you        job of the police be a lot easier if they knew everything
had been using your Omnirec without my knowledge, you              about every citizen? Of course. They might even be able to
would be deemed guilty of some form of spying regardless           prevent crimes, rather than stop them. Yet not many would
of whether you showed the recordings to anyone else. Some          like the idea of having to live in a society in which the secret
may consider this an exaggeration--a consequence of the            police kept detailed files on everyone.
accidental fact that, with existing or foreseeable technology,        As technologists we enjoy devising access control crypto-
there is no way to prevent you from showing your Omnirec           tricks that might make the butlers a little safer. Technical
recordings to others. If it were possible to build a mem-          countermeasures, however, can always be overcome. At a
ory prosthesis with the same guaranteed non-transferability        higher level, therefore, it is important to discuss principles.
property as the human memory, then there might be grounds             To the extent that your digital butlers are increasingly
to consider single-user operation of the Omnirec as quite dis-     knowledgeable about your thoughts, as the purposefully ex-
tinct from spying, just as we have no objections to people         aggerated Omnirec example illustrates, their lack of "rights"
having a good memory, or to people writing down accurate           is a problem. There is practically nothing--in our cultural
debriefing notes after having witnessed something of which         perception of right and wrong, much less in law--to pro-
they want to keep a record.                                        tect them when information is being techno-tortured out of
   So long as enforcing this limitation is technologically im-     them. Your digital butlers do not enjoy the "right to remain
possible, though, the Omnirec remains a sinister spying de-        silent". Yet, given what might be in them, to dismiss this
vice that others might not like you using. Both reactions to       issue as ridiculous just because they are not sentient beings
and justifications for the Omnirec may be similar to those         is to leave the back door open to the thought police.
that apply to Mann's Wearcam. The device is also, how-
ever, a dangerous double-edged sword, as illustrated by the
nightmare scenario in which your Omnirec is actually owned