Proponents of Instant Runoff Voting commonly claim that IRV has the benefit of increasing voter turnout. Their logic is that with ordinary delayed (not "instant") runoffs, fewer voters tend to show up to cast votes in the decisive two-candidate runoff election.
However, analysis of the entire history of IRV elections in San Francisco reveals no increase in turnout. Actually it showed a decrease in turnout, albeit a statistically insignificant one. One possible hyptothesis advanced in the media about why that is:
Quote from Almost Nobody You Know Will Vote For the Next Mayor of San Francisco by Chris Roberts, SF Weekly, 5 Oct 2011: Voter turnout was high in 1999, when the mayoral race broke down into easy, left vs. right lines. With ranked choice voting, a dozen candidates are asking for second and third place votes, and – as [USF professor Corey] Cook says – "there are a dozen shades of gray." Too much ambiguity reduces turnout, as does local media's handling of the top candidates. "The more that papers print that Ed Lee is a shoo-in," said a consultant working for one of Lee's chief rivals, "the less people are going to vote."
Warren D. Smith, a Princeton math Ph.D. who has been studying elections for over a decade, explains as follows. [This analysis was excised and compiled by W.D. Smith and Clay Shentrup from a thread at the CES discussion forum on Google Groups with several contributors. Later David Cary pointed out a problem which caused us to do some rechecking, correcting, and updating with new datapoint from year 2011.]
San Francisco's first Instant Runoff (IRV) election occurred in November 2004, see http://www.usatoday.com/tech/news/techpolicy/evoting/2004-11-05-ranked-evote-snafu_x.htm and other news sources. The turnout percentages of election in SF are tabulated on the sfgov web site http://www.sfgov2.org/index.aspx?page=1670. Here are the numbers. Warning: The calculations below were using 40% for the 2011 turnout based on press accounts, but the correct value was later claimed to be 42.47% on the sfgov site.
I. SUMMARY:
| quantity | IRV era (Nov.2004 thru 2011) | pre-IRV old era (Nov.1960 thru March 2004) |
|---|---|---|
| #ELECTIONS | 14 | 70 |
| HARMONIC_MEAN_TURNOUT% | 40.43 | 45.81 |
| GEOMETRIC_MEAN_TURNOUT | 43.91 | 48.53 |
| ARITHMETIC_MEAN_TURNOUT | 47.06 | 50.91 |
| STD_DEV(TURNOUT) | 18.66 | 14.81 |
| TURNOUT±1σ | 47.06±4.98 | 50.91±1.77 |
| DIFFERENCE | 3.85±5.29 | |
Conclusion based on this: although the IRV era had lower turnouts, the difference's sign is not statistically significant. (There certainly is not evidence IRV increases turnout, at least not using this crude analysis.)
II. THE RAW DATA PLUS SEVERAL ANALYSES:
1. TURNOUTS IN THE IRV ERA (Nov.2004 thru 2011, data in reverse chronological order):
2. TURNOUTS IN THE OLD (PRE-IRV) ERA (Nov.1960 thru March 2004, reverse chronological order):
More sophisticated analyses: The above has been the simplest possible analysis. However, reasons then came to our attention for considering various more complicated corrections to it.
Jan Kok pointed out that many of the mid-year elections before 1992 had very high turnout (we do not know why). November elections are usual, mid-year elections unusual.
Here is the data with mid-year elections removed. Again, the IRV era has lower turnout than the pre-IRV era, but this is now even less statistically significant.
| quantity | IRV era (2004-2011) | pre-IRV era (1960-2004) nonRunoffs | pre-IRV Runoffs |
|---|---|---|---|
| #ELECTIONS | 8 | 35 | 10 |
| HARMONIC_MEAN_TURNOUT% | 45.81 | 51.64 | 39.03 |
| GEOMETRIC_MEAN_TURNOUT | 49.92 | 43.62 | 42.25 |
| ARITHMETIC_MEAN_TURNOUT | 53.61 | 56.73 | 44.68 |
| STD_DEV(TURNOUT) | 18.60 | 15.40 | 12.93 |
| TURNOUT±1σ | 53.61±6.58 | 56.73±2.60 | |
| DIFFERENCE | 3.12±7.07 | ||
Raw data: Mid-year {elections and runoff elections} removed:
Also note that in the above dataset the runoffs tend to have lower average turnout than both nonrunoff and IRV elections. But Jan Kok then observed that odd years were quite different from even years. The runoffs in the odd years were in some cases higher than the main election! (Turnout was 2.8% lower on average.) And the average of all the odd-year runoff turnouts is greater than the average of the odd-year IRV turnouts (46.98 vs 37.27%).
Here's the odd-year-only dataset:
| Year | Main | Runoff |
|---|---|---|
| 2009 | 22.58 | |
| 2007 | 35.62 | |
| 2005 | 53.61 | |
| 2003 | 45.67 | 54.46 |
| 2001 | 29.62 | 16.58 |
| 1999 | 44.95 | 48.84 |
| 1997 | 30.41 | |
| 1995 | 51.86 | 46.56 |
| 1993 | 37.21 | |
| 1991 | 47.61 | 50.73 |
| 1989 | 44.84 | |
| 1987 | 51.18 | 40.36 |
| 1985 | 27.02 | |
| 1983 | 45.99 | |
| 1981 | 25.22 | |
| 1979 | 55.06 | 51.91 |
| 1977 | 51.75 | |
| 1975 | 72.70 | 66.42 |
| 1973 | 60 |
Warren Smith then noticed that the pre-IRV turnouts seem to show a declining trend with time. Just listing the 70 pre-IRV turnouts in reverse chronological order:
and least-square fitting y=Ax+B where x=[1, 2, 3,..., 70]
we find
Going back to our original comparison
| quantity | IRV era (Nov.2004 thru 2011) | Pre-IRV old era (Nov.1960 thru March 2004) |
|---|---|---|
| TURNOUT±1σ | 47.06±4.98 | 50.91±1.77 |
| DIFFERENCE | 3.85±5.29 | |
we could argue that to compensate for the time-declining trend (so that we can compare apples with apples, i.e. same-time to same-time), we should subtract half of the decline, i.e. 26.56/2=13.28 from the pre-IRV average number. (And call that 13.28±1 to crudely estimate the error in the estimate of the decline.) This would yield
| quantity | IRV era (Nov.2004 thru 2011) | Pre-IRV era (corrected for time-decline) |
|---|---|---|
| TURNOUT±1σ | 47.06±4.98 | 37.63±2.03 |
| DIFFERENCE | 9.43±5.38 (IRV turnout greater) | |
This would now turn the tables and support the view IRV increases turnout – except that again this is too small an effect to be statistically significant. One could go further: one could argue that 0.60-per-year decline rate also ought to be reckoned as continuing during the IRV-years also, which would upon adding 3.5 years of further decline, further boost IRV's (decline-compensated) turnout advantage from 9.43±5.38 to 11.53±5.38 which would then be starting to become statistically significant (98% confidence that the decline-corrected IRV turnout exceeds the decline-corrected non-IRV turnout). Of course, to buy that you'd need to believe in this continuing decline and that it is occurring for reasons wholy unrelated to IRV versus non-IRV so that it is valid to compensate for it when assessing the effect of IRV/nonIRV. Obviously it is mathematically impossible for the decline to continue forever (in either time direction) and also the decline apparently actually accelerated (more than doubled?!) after the switch to IRV... so there is some reason to question both those assumptions... so perhaps therefore we should only use half the decline-compensation, not the whole ball of wax. If so, the statistical significance vanishes.
So ok, I think we now have done enough analysis to reach a conclusion.
The evidence from San Francisco up to 2011 does not yield a statistically significant conclusion at the 99% level that IRV increases turnout, and also does not yield a statistically significant conclusion that IRV decreases turnout. The effect, if any, is small and can be regarded using different analysis techniques as having moved in either the positive or negative direction.
We thank David Cary for pointing out an arithmetic error in our decline-trend calculation (now corrected).