Post Election Audits

Post Election AuditsA number of states have enacted requirements for mandatory manual audits (in randomly selected precincts) of the voter-verified paper records produced by the voting systems in use in those states. These audits are designed to verify that the electronic voting systems (either DRE voting machines or optical scan voting systems) are accurately recording and counting the votes. In the randomly-selected precincts, a hand count of the voter-verified paper records is compared to the totals reported by the electronic voting system. (Click on the map to see which states require post election audits.)

Two states without voter-verified paper record requirements (Kentucky, Pennsylvania) also have audit requirements. These were written into statute decades ago, apparently prior to widespread adoption of (paperless) direct recording electronic (DRE) voting systems. It is unclear whether – or how – these states are carrying out their statutory audit requirement. Texas requires audits of optical scan paper ballot systems only; counties with DREs have no voter-verified paper records to audit. The audit provisions in the various states illustrate a variety of manual audit requirements in several states using voter verified paper records, as well as two provisions from states that do not (yet) require VVPR. Some apply generically to both direct recording electronic systems equipped with voter-verified paper audit trails (VVPAT) and optical scan systems, since both offer voter-verified paper records which can be compared to an electronic tally. Others refer specifically to DRE + VVPAT systems (e.g. Washington). In every case (except Kentucky and Pennsylvania) the paper record verified by the voter is the one used in the manual count. The quantity of ballots to audit is most often stated as the ballots in some percentage of the total precincts, although some provisions audit by other units (by machine, ballot batches, etc.).

Why Audit Election Results?
No voting system is perfect. Nearly all US elections today are counted using electronic voting systems. Such voting systems have produced result-changing errors through problems with hardware, software, and procedures.1 Errors can also occur in hand counting of ballots or in the compiling of results. Even serious error can go undetected if results are not audited effectively.

Well-designed and properly performed post-election audits can significantly mitigate the threat of error, and should be considered integral to any vote counting system. A post-election audit in this document refers to hand-counting votes on paper records and comparing those counts to the corresponding vote counts originally reported, as a check on the accuracy of election results, and resolving discrepancies using accurate hand counts of the paper records as the benchmark. Such audits are arguably the most economical component of a quality voting system, adding a very small cost2 for a large set of benefits.

The benefits of such audits include:

  • Revealing when recounts are necessary to verify election outcomes
  • Finding error whether accidental or intentional
  • Deterring fraud
  • Providing for continuous improvement in the conduct of elections
  • Promoting public confidence in elections

Post-election audits differ from recounts. Post-election audits routinely check voting system performance in contests,3 regardless of how close margins of victory appear to be. Recounts repeat ballot counting in special circumstances, such as when preliminary results show a close margin of victory. Post-election audits that detect errors can lead to a full recount.

When an audited contest is also recounted, duplicate work can be avoided. Voting systems should have reliable audit records. Best effort audits should be performed even if the technology does not support optimal audits, or even if the laws do not permit optimal remedies. No single model for post-election audits is best for all states. Election traditions, laws, administrative structure and voting systems vary widely. Nonetheless, there are guiding principles that apply across all states. As states develop their own audit models, the public should have the opportunity to help shape those regulations.4

Best Practices for Post Election Audits
In Post-Election Audits: Restoring Trust in Elections, the Brennan Center teamed with the Samuelson Law, Technology & Public Policy Clinic at Boalt Hall School of Law (UC Berkeley), as well as several election officials and leading academics (collectively, the “Audit Group”), to make several recommendations for conducting post-election audits. Many of these recommendations are amplified in “Principles and Best Practices for Post-Election Audits,” which is available online.

All states should look to statistical sampling methods tied to the margin of victory to improve their criteria for how many units to audit for more effective auditing. A well designed audit can provide a large chance of correcting the outcome if it was wrong. Such risk-limiting audits are being piloted in California, Colorado and Ohio; Colorado law requires moving to risk-limiting audits by 2014. Currently only North Carolina legally requires the use of statistical methods in the selection process, while Oregon, New Mexico and New Jersey laws require taking the margin of victory into account when determining what (fixed) percentage to audit. (New Jersey’s law is not yet implemented). Ten California counties conducted pilot risk-limiting audits recently. Among other state grants, the U.S. Election Assistance Commission awarded California $230,000 in federal grant money to fund up to 20 such pilot audits following elections held in California counties throughout 2012.

The following steps are critical for a good audit:

Auditing All Ballots Good audit protocols mandate that all ballots – early and absentee ballots, UOCAVA ballots, regular and provisional ballots, and aggregation at the tally server – be audited for accuracy.

Using Transparent and Random Selection Processes for All Auditing Procedures Audits are much more likely to prevent fraud, and produce greater voter confidence in the results, if the ballots, machines or precincts to be audited are chosen in a truly random and transparent manner, observable by the public with sufficient notice.

Conducting in a Timely Manner Audits should be conducted before results are finalized, so that if the audit reveals problems, official totals can be corrected.

Implementing Effective Procedures for Addressing Evidence of Fraud or Error If audits are to have a real deterrent effect and catch widespread, systemic problems, jurisdictions must adopt clear procedures for dealing with audit discrepancies when they are found.. Such procedures must ensure that outcome-changing errors are not ignored, otherwise vote tampering succeeds.

Encouraging Rigorous Chain of Custody Practices. Audits of voter-verifiable paper records will deter attacks and identify problems only if states have implemented solid chain of custody and physical security practices that will allow them to make an accurate comparison of paper and electronic records.

Florida and Minnesota – A Tale of Two Elections

A compelling case for post-election audits can be drawn through a comparison of the 2008 Senate race in Minnesota and the 2006 Congressional race in Florida’s 13th District. On election night in 2008, based on the electronic tallies, Norm Coleman was reported to be the winner of the Minnesota Senate rase. Only because Minnesota used paper ballot optical scan systems statewide and only because election workers hand-counted all of the almost 3 million paper ballots that were cast in the election could Minnesota determine the true winner of the election: Al Franken was eventually found to have won the race.

In stark contrast, in the 2006 Congressional race in Florida’s 13th District, candidate Vern Buchanan was reportedly ahead of candidate Christine Jennings by 369 votes.901 However, in Sarasota County, one of the five counties in the District, a staggering 18,000 votes were not recorded for the Congressional race. That was a higher under-vote rate (almost 13%) than in any of the other counties (in other counties, the highest under-vote rate was just under 6%, and the others were between 2% and 3%). Unlike Minnesota, however, in 2006 Sarasota County used paperless DREs. Therefore, there were no independent records of the votes cast in the polling places in that county. Some, including the U.S. Government Accountability Office, ultimately concluded that the under-vote was the result of a confusing touch screen ballot that caused voters to overlook the Congressional race. But because there was no evidence (paper ballots) that could be reviewed to confirm the intention of the voters, there was no way to dispute the electronic result. Following a lengthy legal battle Vern Buchanan was sworn in.

Ron Rivest at the 2007 Post Election Audit Summit
Joe Hall speaking at the 2007 Post-Election Audit Summit
Risk-Limiting Post-Election Audits

The risk-limiting audit5 is the gold standard of audits. Risk-limiting means that if the machine-reported count is incorrect, “there is a large, pre-specified chance that the audit will reveal the correct outcome.”6

Risk-limiting post-election audits are designed to minimize the size of the audit when the outcome is correct, while with very high probability correcting the outcome, if it is incorrect, by counting all the ballots. The audit continues until there is sufficiently strong statistical evidence that the apparent outcome is right, or until all the ballots have been manually counted. There are several factors that determine the size of the audit. Two are the closeness of the race being audited and the total number of ballots cast in that race.

To understand why, imagine an election with 100,000 votes where the machine results show candidate A beating candidate B by 100 votes. A relatively small number of votes for B that either were incorrectly counted for A or not counted at all could change the result and determine that B was the actual winner. Since a few potentially election-changing discrepancies might not be uncovered by a small audit, a large audit is needed. If, however, the machine results show A beating B by a wide margin of 20,000 votes, but B actually beat A, there would have to be a large number of B votes given to A or not counted at all to change the outcome. Therefore, if only a relatively small number of audit units is examined, it would be highly likely that a large number of wrongly recorded votes would be uncovered. A third factor is the size of the batches for which auditable totals are available. The smaller the batches, the fewer ballots will have to be examined, with individual ballot audits being the most efficient.

The following food example from Philip Stark is instructive.7 Suppose there are 100 bags of 100 jelly beans each, with some bags having a mixture of flavors and others consisting of a single flavor only. Suppose also that each bag is covered with aluminum foil, so that nobody can tell which is which by looking at the bags. I love coconut jelly beans and I want to estimate the number of coconut beans in all 100 bags.

One option would be to choose a bag at random, open it, and count all the beans. I could then estimate the total number of coconut beans by multiplying the number in that bag by 100. If I chose a bag that contained only coconut beans, I would estimate that all 10,000 beans were coconut; if the bag consisted of entirely a different flavor, I would estimate that none of the 10,000 beans was coconut; and if I picked a mixed bag, I would assume the ratio of all 10,000 beans was the same as that in the bag I had picked.

Suppose instead the jelly bean bags are all opened by someone else, dumped into a large pot, and stirred well. Suppose I then choose 100 beans at random from the large pot and count the number of coconut beans in that group. The estimate I get in this case will be far more reliable than the estimate I would get by looking at the contents of a single bag, even though in both cases I’m examining 100 jelly beans. To get a similarly reliable estimate on the number of coconut jelly beans in all the bags by drawing individual bags at random, I would have to examine far more bags and count many more jelly beans.

The basic structure of a risk-limiting audit follows the following framework: Hand count ballots until the evidence is strong that the outcome is correct. The number of ballots counted will depend on the errors you observe and the particular method being used. If you see no errors or predominantly errors that, if corrected, help the apparent winner, you need to look at fewer ballots than if you see errors that, if corrected, predominantly help the apparent loser. In sum, the number of ballots that need to be examined depends on the data.

There are also time-saving techniques for doing a risk-limiting audit of all of the ballot races simultaneously, [3. Stark, Philip B. 2010. Super-simple simultaneous single-ballot risk-limiting audits. In Proc. EVT/WOTE'10, the Electronic Voting Tech. Workshop / Workshop on Trustworthy Elections.] although hand-counting multiple races at once may be significantly harder than counting just one race by the sort and stack method. In 2009 Colorado modied its election law to require risk-limiting audits by 2014.8

The following year the American Statistical Association issued a statement endorsing risk-limiting post-election audits.[34] In the same year AB 2023 became law in California. AB 2023 authorizes “the Secretary of State to establish a post-canvass risk-limiting audit pilot program in ve or more voluntarily participating counties for the purpose of verifying the accuracy of election results.”9

  1. For example, in Pottawattamie County, Iowa, in the June 2006 primary election for County Recorder, the original optical scan count showed challenger Oscar Duran defeating the incumbent, John Sciortino. A hand count showed that Sciortino actually had won handily; the scanners had been misprogrammed. In Napa County, California, after the March 2004 primary, the 1% manual tally discovered that the optical scanners had been miscalibrated and were failing to detect the dye-based ink commonly used in gel pens. The ensuing recount recovered almost 6700 votes (but no outcomes changed).
  2. For instance, in Minnesota after the 2006 general election, the cost of the wages for election judges (pollworkers) to count votes has been estimated at $24,500 to $27,000 statewide – 9 to 10 cents per hand-counted vote, and about 1.2 cents per voter in the election (http://www.ceimn.org/files/CEIMNAuditReport2006.pdf). While audit costs will vary depending on the scope of the audits and other considerations, they can be expected to be a small fraction of election administration costs.
  3. We use “contest” to refer to any ballot item (such as an election to public office or a ballot initiative) – not to a challenge to the results, as in some states.
  4. From ElectionAudits.org
  5. Douglas W. Jones and Barbara Simons, Broken Ballots, 2012, pp. 336-338.
  6. American Statistical Association. 2010. Statement on risk-limiting post-election audits, http://www.amstat.org/outreach/pdfs/Risk-Limiting_Endorsement.pdf Apr. 17, 2010.
  7. Stark, Philip B. 2010. Risk-limiting vote-tabulation audits: The importance of cluster size. Chance 23(3): 9-12.
  8. Colorado, State of. 2009. x 1-7-515: Risk limiting audits – pilot program. Col. Revised Statutes, Title 1, Article 7, Part 5. Originated as House Bill 09-1335, Section 12.
  9. California, State of. 2010. x 15560: Post-canvass risk-limiting audit pilot program. Cal. Elections Code, Div. 15, Chapt. 8.5. Jul. 19, 2010. Originated as Assembly Bill 2023.

Download PDF Download this page in PDF format