How accurate is the GE sample vote count ?

0

By Ngiam Shih Tung

The Elections Department released early indications of polling results for the first time during the 2015 General Election by publicly announcing the result of sampling checks within two hours of the close of polls. Sampling checks have been carried out since 2001 or earlier, but the results were not made known to all candidates or the public until this year.

The sampling check is carried out by Counting Assistants drawing a sample of 100 ballots from each counting table after opening the ballot boxes and mixing the ballot papers together. As each counting table corresponds to one polling district and there are about 2,000-3,000 voters per polling station, the 100-ballot sample per polling district corresponds to three to four per cent of votes cast and is a large enough sample to make a good estimate of the final polling result

As shown in Table 1, the margin of error varies from ±1.2 percentage points for a large GRC, Pasir Ris-Punggol with 66 polling districts, to ±4.4 percentage points for Potong Pasir, the smallest Single Member Constituency (SMC) with only 5 polling districts.

For simplicity, no adjustments are made to account for the variation in number of voters in each polling district, and the error margin is calculated for a 50:50 vote split.  The Straits Times reported that the error margin for sampling counts is ±4% but this is a worst case and only applies to Potong Pasir which is the smallest SMC by far.

On average, the uncertainty in the sampling check is within 1.3 percentage points for GRCs and 3.3 percentage points for SMCs.

Table 1 - Error margins for selected constituencies and SMC and GRC averages
Number of polling districts Sample size Margin of error at 50% vote share
Pasir Ris-Punggol GR 66 6,600 1.2%
Aljunied GRC 50 5,000 1.4%
East Coast GRC 32 3,200 1.7%
Bukit Panjang SMC 11 1,100 3.0%
Potong Pasir SMC 5 500 4.4%
GRC Average 45 4,500 1.5%
SMC Average 9 900 3.3%
Error margin is calculated as half the width of the two-sided 95 per cent confidence interval (Normal approximation) at 50 per cent vote share. All polling districts within a constituency are assumed to have an equal number of voters.

 

In a first-past-the-post election, however, what counts is not the actual vote share but crossing the 50 per cent threshold.

If we ignore three-cornered fights and make use of the normal approximation again, we can calculate a “victory threshold” or minimum sampling check result for which a candidate can be 95 per cent certain of victory.  This is shown in Table 2 for selected constituencies.

Note that the victory threshold is slightly lower than would be obtained if we simply added the error margin from Table 1 to 50 per cent because the victory threshold is a one-sided rather than a two-sided test. If you’re not statistically-inclined, don’t worry about it - that effect is small in this case.

In Aljunied, despite the appearance of a nail-biting finish, the 52:48 sampling check result was above the victory threshold for the Workers’ Party and the PAP in fact had only a 0.2 per cent chance of winning in that constituency once the sampling check result was known [See note 1].

In Punggol East, however, the sampling check result of 51 per cent for the PAP was lower than the victory threshold of 52.4% and the PAP could only be 75 per cent certain that it would win there. Put another way, Lee Li Lian still had a 25 per cent chance of winning even after seeing the sampling check results.

In all the other constituencies, the winning party’s sampling check result significantly exceeded the victory threshold, so the final result was not in doubt once the sampling checks were completed.

Table 2 - Victory threshold for selected constituencies.
No. of polling districts Victory threshold Actual PAP share in sampling check Probability of PAP win given sampling check result
Pasir Ris-Punggol (6-member GRC) 66 51.0% 73% 100%
Aljunied (5-member GRC) 50 51.2% 48% 0.2%
East Coast (4-member GRC) 32 51.5% 61% 100%
Sengkang West (SMC) 13 52.3% 63% 100%
Punggol East (SMC) 12 52.4% 51% 75%
Hougang (SMC) 9 52.7% 42% 0.0%
Potong Pasir (SMC) 5 53.7% 68% 100%
1. Victory threshold is defined as the minimum sample count result for which a candidate can be 95 per cent certain of receiving over 50 per cent of the actual vote (one-sided test).
2. Probability of PAP win is the probability that the actual PAP vote share is > 50 per cent, given the observed sampling check result.

The sampling check came closest to the actual vote share in Pasir Ris-Punggol GRC with a difference of only 0.1 percentage points while the largest difference of 2.6 percentage points was observed in MacPherson GRC. This is to be expected as the sample size in Pasir Ris-Punggol was much larger than that in MacPherson and the sampling check gets more accurate as the sample size increases.  The observed differences between sampling checks and actual vote counts were all within the expected 95 per cent error margins except for one constituency, but one out of 29 is not surprising, statistically. We also did not take rounding errors into account, which slightly widen the margin of error.

The sample check is a form of a quick count, which is used in developing democracies where there are concerns with regards to the compilation of electoral results by the central government. In Singapore’s case, there is no obvious need for a quick count as the entire counting process can be observed by candidates’ counting agents and elections results have always been announced within a few hours. Nonetheless, given that the Elections Department has chosen to conduct sampling checks, the decision to publicly reveal sampling check results is a welcome one.

Notes

[1] Some additional uncertainty is caused by the sampling check results only being reported as whole number percentages. The true PAP sampling check result from 5,000 samples could have been as high as 48.5 per cent rather than the reported 48 per cent.  However, this would still have given the PAP only a 1.8 per cent chance of victory once the sampling check results were known.

The author is an Engineering Manager by vocation, Shih Tung has served as an election official, polling agent or counting agent in four elections.