Acting Co Director Mark Lindeman testified at the Pennsylvania Senate State Government Committee’s “Public hearing on the Department of State’s Risk Limiting Audit Pilot.” His written testimony is included below and the video can be viewed here.
Submitted Testimony of Mark Lindeman, Verified Voting Hearing on Risk-Limiting Audit Pilots
Senate State Government Committee August 24, 2021
Chairman Argall, Chairman Street, and Members of the Senate State Government Committee, I thank you for the opportunity to speak with you this morning.
My name is Mark Lindeman. I am currently acting co-director of Verified Voting, a national non-partisan non-profit organization that promotes the appropriate use of technology in elections to foster justified public confidence. Since its founding by computer scientists in 2004, Verified Voting has promoted voter-verified paper ballots and routine, rigorous post-election audits to confirm, rather than rely upon, the accuracy of computerized voting systems. As you know, Pennsylvania made great progress in 2019 and 2020 on both those fronts – paper ballots and post-election audits. As a specialist in risk-limiting audits and other post-election audit methods, I have served as a member of Pennsylvania’s audit work group since early 2019, and I have traveled around the state assisting implementation efforts, including the pilot risk-limiting audit of the 2020 presidential election.
This morning, I want to briefly cover three main topics. First, I will explain from my perspective how risk-limiting audits and other robust post-election audits fit into the landscape of election administration. Second, I will offer some background on the presidential election pilot, and briefly on other pilots. Third, I will offer some broad recommendations for post-election audits and election administration in Pennsylvania. Our work group has been extremely collegial, but I cannot speak for other members: these opinions are my own.
Post-election audits
Last week, a bipartisan task force of the National Association of Secretaries of State offered a helpful high-level description of post-election audits in general: “Post election audits are designed to ensure the accuracy of election results, verify the integrity of the election, and promote voter confidence in the election process.” 1 These goals are best served by routine, robust, transparent audits that take place before election results are finalized and provide assurance that the results are accurate. The state’s audit work group has pursued essentially those goals since 2019. Milestones include successful risk-limiting audit pilots in Mercer and Philadelphia Counties after the November 2019 municipal elections, and wider pilots of the 2020 primary election, the general election, and the 2021 primary election. My testimony today will focus on the general election pilot.
These ongoing pilots are part of a broader project: to engage election officials throughout Pennsylvania in designing, refining, and implementing audits that provide high assurance in election outcomes. I have found both state and local election officials to be committed to these efforts, but constrained by limited time and resources, especially after the COVID-19 pandemic erupted in March 2020. Audits in Pennsylvania are a work in progress – and the progress has been substantial.
The pilot risk-limiting audit of the 2020 general election
In particular, the pilot audit of the November 2020 general election represents a major milestone. This pilot was conducted in addition to the statutorily required statistical sample of 2% of votes or 2,000 votes. The statistical sample audit is useful, but it has important limitations. For instance, the statute does not specify an audit method. The work group learned that some counties have hand-counted the audit samples, while others have used scanners other than the ones used on election day. Election security experts agree that audits should not rely on voting data captured by computerized systems, including digital ballot images. Instead, human beings should examine the very same paper ballots that voters could verify.
The statistical sample law also is limited by the arbitrary size of the audit. A 2% audit of uncontested races can be grossly burdensome; a 2% audit of a close contest may not provide adequate evidence that the outcome is correct. In agreement with election security experts,2 Verified Voting has advocated for risk-limiting audits, which by design provide strong evidence that a full hand count would confirm the election outcome. All the pilots I have mentioned are of risk-limiting audits, or RLAs for short.
To be clear, these pilots, including the presidential election pilot, have not been full risk-limiting audits. The reason is inherent in the definition of an RLA. By definition, if the voted ballots contradict the reported election outcome, a risk-limiting audit is very likely to lead to a full hand count that corrects the outcome – before the results are finalized. In my view and that of other work group members, a full RLA in Pennsylvania would have been impractical last November given the tight certification deadline and the many demands on state and county election officials.
Participation in the pilot was voluntary, but extending the deadline for participation through late January made it possible for many more counties to take part.
Department of State staff and work group members provided extensive advice and assistance to county officials during the pilot. The non-profit company VotingWorks provided and supported the Ario software used to help coordinate the audit and gather the results. As you know, ultimately 63 of 67 counties took part in the pilot. Local officials were provided with a list of ballots to retrieve and manually review; they entered vote interpretations in the Ario software tool.
A risk-limiting audit can use any of several methods; this pilot used a method called ballot polling. Much as a public opinion poll uses a random sample of adults or likely voters, a ballot polling audit uses a random sample of voted ballots. Broadly, a ballot polling RLA seeks a preponderance of votes for the reported winner in the sample that provides strong evidence that the candidate indeed had more votes overall.
In close elections, ballot polling audits can require rather large samples. For the 2020 presidential election in Pennsylvania, experts estimated that if the reported vote counts were accurate, a statewide sample of about 50,000 voted ballots would be very likely to provide strong evidence that the outcome was right.3 However, the Department of State rightly anticipated that not all counties would participate. For the 65 counties that initially submitted data for the audit, the department set a sample size of 48,000 ballots, so that these counties would not be burdened by other non participation. Ultimately, two more counties declined to participate, and so the completed sample included 45,529 individual ballots. The county samples varied as widely as the counties themselves vary in size: Cameron County audited 10 ballots, while Philadelphia County audited over 5,200.
Let me talk briefly about the statistical properties of the audit sample, without becoming hypertechnical. Briefly, in the 63 participating counties, the reported margin was Biden defeating Trump by 2.85 points. The margin in the audit sample was 2.67 points. So the difference between the reported margin and the audit sample margin was 0.18 points. That is well within the statistical margin of error if the reported vote counts were accurate. In fact, it is about as close as one would expect for a random sample of this size. 4 Contrariwise, the result is not at all what we would expect if the election was marred by widespread miscounts favoring the reported winner. So even though this pilot was not a full risk-limiting audit, it still provides strong evidence that the reported winner – Joe Biden – received more votes. At the same time, the pilot gave election officials around the state hands-on experience with audit methods and helped lay the groundwork for future audits.
The 2021 primary pilot: a different method
I should note that the pilot of the 2021 municipal primary used a different audit method, called batch comparison. Batch comparison is a more traditional audit method. For instance, suppose that each precinct is treated as a batch, and you have vote totals for each candidate in every precinct in the state. Then to conduct an audit, you could randomly sample from among those precincts, hand-count the ballots from each one, and examine any differences between the hand-count results and the originally reported totals for each batch. In practice, entire precincts may not be the most practical batches. For instance, some counties do not sort their mail and absentee ballots by precinct, so hand-counting all the ballots from a precinct may be difficult. As you can see, batch comparison has potential advantages over ballot polling, and also poses some challenges. Now election officials around the state have hands-on experience with both methods, and that is a good thing for the future.
Next steps for the legislature
Let me briefly offer some broad recommendations to your committee on how best to support the progress that Pennsylvania continues to make on election verification.
First: recognize the progress. Strangely and sadly, election officials have been under siege from members of the public who have been led to believe that massive irregularities occurred in the 2020 election. This is deeply unfair and destructive.
Election administration in 2020, during a harrowing pandemic, with the new requirements of Act 77, and in many counties using new voting systems – with limited staff and limited training – required courage and perseverance. Election officials are not infallible superheroes, but in 2020, they were real-world heroes. Stand up for them.
Second: provide support. Successful post-election audits are the culmination of extensive planning – not just of the audits themselves, but of voter registration processing, absentee ballot processing, ballot management and reconciliation, and other processes. Given the wide disparities in resources across Pennsylvania counties, the legislature has an important role to play in equipping all counties to meet a high standard of election excellence. In that effort, the Department of State appropriately plays a large role, even if audit responsibilities themselves are moved to another government agency. In my experience, the department staff is competent, conscientious, and wildly overworked. Even the most talented and dedicated people cannot be expected to support 67 county election directors as one part of their broad responsibilities. I urge the committee to look for ways to increase capacity at the state level, as well as to provide material support to counties as they continue to modernize their election equipment and processes.
Third: remove obstacles. In particular, the state’s post-election certification deadline is stringent given all the procedures that must be completed before results are certified. Robust audits, such as risk-limiting audits, whose workload is unpredictable may be difficult or impossible to complete in the current timeline. Rushing quality assurance is not a path to election excellence. Election officials need time to do this important work.
Finally: seek to legislate principles, not details. Here are some key principles:5 audits should be routine; they should be robust; they should be timely; they should examine paper ballots, not just digital artifacts; they should be observable, and the procedures should be clearly documented; election materials’ chain of custody should be preserved before and during the audit. Such principles are appropriate for legislation. Admittedly, the line between principle and detail is not clearly defined. But be wary of requiring specific audit methods that may be infeasible or inefficient, and of imposing other requirements that could make audits less accurate or less observable.
I know that advice is easy and solutions are hard. I thank you for your interest in these urgent topics, and I am happy to help in any way I can as Pennsylvania continues to make strides in demonstrating the trustworthiness of its elections.
1 “NASS Task Force on Vote Verification: Post-election Audit Recommendations,” August 2021, available at https://www.nass.org/sites/default/files/Summer%202021/NASS%20Vote%20Verification%2 OTask%20Force%20Recommendations.pdf
2 For instance, see the National Academies of Sciences, Engineering, and Medicine’s consensus study report Securing the Vote: Protecting American Democracy (NAP Press, 2018), available for download at https://www.nap.edu/catalog/25120/securing-the-vote-protecting american-dem ocr acy .
3 More specifically, an RLA with that sample size would have about a 90% chance of confirming the outcome at a 10% risk limit. In an RLA with a 10% risk limit, if a full hand count would change the outcome, the RLA guarantees at least a 90% chance of leading to a full hand count – i.e., no more than a 10% chance of failing to correct the outcome, no matter how close the actual result.
4 The margin between Biden and Trump in the sample had about a 50% chance of falling within about 0.3 points of the true margin in these counties; a 75% chance of falling within about 0.5 points; and a 95% chance of falling within about 0.9 points. That 95% “confidence” standard is conventionally used for margins of error, so we can say that the margin of error for the margin between Biden and Trump was 0.9 points. (This number should not be confused with the margin of error for either candidate’s vote share.)
5 See also Principles and Best Practices for Post-Election Tabulation Audits, available at https://verifiedvoting.org/publication/principles-and-best-practices-for-post-election ta bulat ion- audits/ , and the NASS task force statement referenced in footnote 1.