Experimental Design
During the period of our study, Afghanistan was an active warzone. To keep our field staff safe, we selected polling centers that met three safety criteria: (i) achieving the highest security rating given by the International Security Assistance Force (ISAF) and the Afghan National Police (ANP); (ii) being located in a provincial center, which are much safer than rural areas; and (iii) being scheduled to operate on election day by the Independent Elections Commission. Our experimental sample comprises 471 polling centers (7.8 percent of polling centers operating on election day) in 19 of the 34 provincial capitals in Afghanistan.
On election day (September 18, 2010) we randomly announced the use of photoquick count by delivering letters to 238 of the 471 polling centers in our experimental sample. We call this treatment our Letter Treatment. We instructed our Afghan field staff to deliver letters to polling center managers after 10am and before polling concluded at 4pm. Staff visited all 471 polling centers the following day in order to take a picture of the election returns form. The letter announced to polling center managers that researchers would photograph declaration of results forms the following day (September 19) in order to document discrepancies arising during aggregation.
Two points about the experimental protocol bear emphasis. First, if we had not notified managers of monitoring on election day, they would have been unaware of our treatment as no election staff should be present at the polling center on the day after the election. Correspondingly, our staff report encountering election officials while they were taking photographs on the day following the election at only 2 of our 471 polling centers. Second, the experimental sample was known only to the research team. Polling center managers in the treatment group were informed of their status, but no election officials had a means of determining which sites we had selected as controls. We asked polling center managers to acknowledge receipt by signing the letter. Managers at 17 polling centers (7 percent of centers receiving letters) refused to sign. We designate a polling center as treated if the manager received a letter (Letter Treatment = 1). Our results remain robust to redefining treatment as both receiving and signing a letter.
To inform our treatment assignment, we fielded a baseline survey of households living in the immediate vicinity of 450 of the polling centers in our experimental sample a month before the election (August 2010). On election day, we added 21 polling centers in Kabul after obtaining additional funding. We do not have baseline survey data for these 21 polling centers and so place them all in a single strata when assigning treatment. To ensure balance, we stratify treatment on province. In the 450 polling centers for which we have baseline data, we also stratify on the share of respondents from the baseline survey reporting at least occasional access to electricity and on respondents reporting that the district governor carries the most responsibility for keeping elections fair. We estimate all core specifications both with and without stratum fixed effects.
For the 2014 paper, to measure the effect of election fairness on legitimacy, we combine the results of the letter intervention with data from a post-election survey which we conducted in December 2010, roughly three months after the election and only shortly after the Independent Election Commission certified final results. Respondents came from households living in the immediate vicinity of 450 of the 471 polling centers in our experimental sample, for a total of 2,904 respondents. To obtain a representative sample of respondents living near polling centers - generally neighborhood landmarks such as mosques, schools or markets - enumerators employed a random walk pattern starting at the polling center, with random selection of every fourth house or structure until either six or eight subjects had been surveyed. In keeping with Afghan custom, men and women were interviewed by field staff of their own gender. Respondents within households were randomly selected using Kish grid. The survey had 50 percent female respondents. Enumerators conducted the survey in either Dari or Pashto. We measure perceptions of government using individuals’ responses to four questions. The first two questions probe attitudes that might contribute to procedural legitimacy; the second two questions measure attitudes related to outcome legitimacy, i.e. competence in service provision. We use all four questions since any single question is unlikely to capture fully a citizen’s conception of legitimacy. While we sought to distinguish between legitimacy related to procedure and competence, our specific questions may straddle the concepts. The survey asks other questions about attitudes to government as well.