A Study of Vote Verification Technologies …

                  A Study of Vote Verification Technologies
                  for the Maryland State Board of Elections

                                  Executive Summary

        This Executive Summary presents the principal findings of two studies of vote
verification technologies that were commissioned in 2005 by the Maryland State Board
of Elections (SBE). The first, or the technical study, was conducted by researchers at the
University of Maryland, Baltimore County (UMBC). The second, or the usability study,
was conducted by researchers at the University of Maryland, College Park.

       We note that while these studies were commissioned by the SBE, they were
conducted independently of the SBE and, independently of one another. This should
provide the citizens and decision-makers in the State of Maryland with a high degree of
confidence that the studies are impartial and scientifically sound.

                     Part I: Technical Study Executive Summary

        Scholars at UMBC, working through the National Center for the Study of
Elections of the Maryland Institute for Policy Analysis and Research, conducted a
technical review of vote verification systems for the Maryland State Board of Elections
(SBE). Initially, the review was supposed to include up to seven systems from the
following organizations and individuals: VoteHere (Sentinel); SCYTL (Pnyx.DRE); Prof.
Ted Selker, MIT (VVAATT); Diebold's VVPAT; Democracy Systems, Inc.
(VoteGuard); IP.Com; and Avante. We determined that IP.Com did not represent a true
vote verification technology, and Avante and Democracy Systems, Inc., declined to
participate in the study. We also examined the SBE's procedures for "parallel testing" of
the Diebold AccuVote-TS (touch screen) voting system in use in Maryland and used this
as a baseline against which to evaluate the vote verification systems.

        In conducting our analysis, we received demonstrations from the vendors, and we
examined the vendors' hardware, software, and documentation to determine if their
products did what their vendors claim that they do. That is, do they enable voters who
use the touch screen voting system in use in the State of Maryland to verify that their
votes were cast as intended, recorded as cast, and reported as recorded, and do they
permit post-election auditing? We examined such issues as:

           implementation
           impact on current state voting processes and procedures
           impact on voting
           functional completeness
           security against fraud, attack and failure
           privacy
           reliability
           accessibility
       We also compared these systems to one another and to the state's current voting
system and procedures, which includes the SBE's use of parallel testing around that
system.

       We note several specific concerns about these products, including the following:

       1. Only one of these products, the Diebold VVPAT, provides for a pure paper
          solution.
       2. All of these products would impose significant one-time implementation and
          on-going management burdens (cost, effort, security, etc.) on the SBE and the
          state's 24 Local Boards of Elections.
       3. All would increase the complexity of the act of voting.
       4. All would increase the amount of time required to vote.
       5. All would at least double the amount of effort required to administer elections.
       6. All would adversely affect voter privacy.
       7. These products would have both potentially positive and potentially negative
          impacts on security and election integrity.
       8. None can be considered as fully accessible to persons with disabilities and
          none of them fully meets the accessibility standards of Section 508 of the
          Rehabilitation Act.
       9. Integration of these systems will require the cooperation of Diebold to
          develop and/or ensure the viability of a working interface between the
          vendors' products and the Diebold system.

       Our principal findings are, first, that each of the systems we examined may at
some point provide a degree of vote verification beyond what is available through the
Diebold System as currently implemented. But this is true only if the system were fully
developed, fully integrated with the Diebold DREs and effectively implemented.

        Our second principal finding is that none of these systems is yet a fully
developed, commercially ready product. None of these products had been used in an
election in the U.S. (SCYTL has been used outside the U.S. and a different version of the
Diebold VVPAT has been used in the U.S.).

       Were the State of Maryland to decide to acquire any of these products, the vendor
would have to invest additional money and effort to produce an actual product and make
the product ready for use in actual elections. Indeed, nearly all of these vendors are
looking for some level of external support to fully develop and commercialize their
products.

        In our expert opinion, it is a bad idea for governments to buy products that are not
functionally complete and that either do not have positive records in the market place or
that cannot be fully and effectively tested in simulated elections to ascertain their
performance characteristics.




                                                                                           2
        Therefore, based on the evidence from this study, we cannot recommend that the
State of Maryland adopt any of the vote verification products that we examined at this
time.

        We would note that no election system ­ regardless of the technology involved ­
is foolproof nor is any election system completely immune or secure from fraud and
attack. Indeed, there is a long and inglorious history of election fraud in the U.S. that
involves nearly all methods and technologies of voting, especially paper voting systems.
Moreover, it would be prohibitively costly to make any election totally secure.

        Finally, regardless of what the State of Maryland does in the near term with
regard to vote verification and vote verification systems, in future elections, it should
expand the use of parallel testing. The state should also undertake a full-scale assessment
of the security procedures and practices around its current voting system. We say this
even with the knowledge that current security procedures are reasonable and prudent and
that the SBE's system of parallel testing, as currently implemented, reduces considerably
the possibility of fraud and attack on the system.

                     Part II: Usability Study Executive Summary

        The University of Maryland's Center for American Politics and Citizenship, along
with the Human-Computer Interaction Lab, conducted a usability study of four vote
verification systems and a voting system with no verification unit for the Maryland State
Board of Elections.

The major findings from the expert review by human-computer interaction experts are:

   ·   There was a perceived trade-off between usability and security. In all cases, the
       verification system appeared to reduce the usability of the voting process
       compared to the Diebold AccuVote-TS, which had no verification unit.

   ·   The Diebold AccuVote-TSx with the AccuView Printer Module (paper printout,
       referred to as AccuView Printer) was rated most favorably. However,
       suggestions were made for improvement and questions were raised about the
       paper record's utility when used for a long ballot.

   ·   Privacy concerns were raised about each of the four vote verification systems.

The major findings from the field test involving more than 800 Marylanders are:

   ·   All of the systems were viewed favorably, including the Diebold AccuVote-TS
       with no verification unit.

   ·   The Diebold with AccuView Printer was rated the most favorably in terms of
       voter satisfaction, but not substantially better than the AccuVote-TS with no
       verification unit or the VoteHere Sentinel.


                                                                                            3
   ·   The MIT (audio) system was found to be distracting and it failed to generate as much
       confidence as other systems. It also was criticized by some users because of sanitary
       concerns related to the repeated use of the same headset.

   ·   Participants needed the least amount of help when using the Diebold AccuVote-
       TS system (no verification unit). The Diebold with AccuView Printer system
       (paper trail) came next. Voters received more help using the VoteHere (internet
       or telephone), MIT (audio), and Scytl (monitor) systems.

The major findings concerned with election administration are:

   ·   Adding any of the four verification systems greatly increased the complexity of
       administering an election.

   ·   The paper spool in the Diebold AccuView Printer had to be changed frequently,
       and changing it was fairly complex.

   ·   It was difficult and time consuming to set up the Scytl system.

   ·   The Scytl, MIT, and Diebold AccuVote-TS with no verification unit were out of
       commission for some portions of the study (but not enough to affect the results).

   ·   Diebold provided outstanding response to service calls. Scytl (based in Spain)
       provided poor service. No service calls were made to MIT or VoteHere.

Recommendations

   ·   On the basis of usability and some administrative considerations, we cannot
       recommend that the State of Maryland purchase any one of the vote verification
       systems (or system prototypes) that were reviewed. There are some important
       tradeoffs between usability and other considerations, including the security of the
       vote.

   ·   We recommend that the voter interface of AccuVote-TS (with no printer unit) be
       modified to incorporate some of the improvements made to the interface of the
       AccuVote-TSx with the AccuView Printer system.

   ·   The AccuVote-TS with no verification unit became inoperative while an
       individual was voting under normal circumstances. This had a direct impact on
       the usability of the system and caused concern among voters. An explanation was
       provided but it was beyond the scope of this study to confirm it. We recommend
       this situation be addressed.




                                                                                           4