Turnout bias in post-election surveys

It is a well-known fact that reported turnout in post-election surveys is higher than official turnout rates. This overestimation of turnout in surveys may be due to different factors: coverage error, overrepresentation of voters (nonresponse bias) and overreporting of actual abstainers (measurement bias).  Unfortunately, there is only very few data available to test how much these factors contribute to the overall turnout bias. What is needed for such an analysis is validated data, i.e. one has to know if a survey respondent correctly reported her turnout or if she lied about it. Even rarer than this validated data, though, is turnout information about non-respondents. Using a unique data-set from the canton of Geneva, Switzerland, Pascal Sciarini and me are able to examine the components of the turnout bias. The data include not only information about reported and validated turnout among survey respondents, but also the validated turnout among non-respondents. We use these data to analyse several aspects of turnout bias, which so far lead to two publications.

In a first paper we focus on vote overreporting and its link with survey participation and political participation. We argue that turnout is typically a topic with a connection between response propensity and measurement error, as factors related to political involvement (usual voting behaviour) are likely to influence both survey participation and misreport. To take this link into account, we run a Heckman selection model in which the outcome equation estimates the probability of overreporting conditional on the probability of participation in the survey. In most other data, though, no information about nonresponse is available, so that we offer a solution that may still correct for the nonresponse bias. Applying this correction to U.S. data shows that not correcting for response propensity may indeed lead to inaccurate estimates of the determinants of overreporting and, ultimately, of turnout.

Sciarini, P. and A.C. Goldberg (2016). Turnout bias in postelection surveys: Political involvement, survey participation and vote overreporting. Journal of Survey Statistics and Methodology 4(1): 110-137.

Our second paper starts with a more general analysis of the components of turnout bias. Using two different data-sets, we show that voter overrepresentation accounts for a larger share (60-65%) of total turnout bias than misreporting. This speaks against a widespread view that measurement error is more important. In a second step we then concentrate on nonresponse bias and analyse individual predictors of survey participation. We are even able to break down the nonresponse problem into its two main components – contact and cooperation – and can thus see which type of person is lost at which point of the survey process.

Sciarini, P. and A.C. Goldberg (2017). Lost on the way. Nonresponse and its influence on turnout bias in post-election surveys. International Journal of Public Opinion Research 29(2): 291-315.