Experimental Design
Researchers implemented the survey in August 2009 in Distrito Federal (DF), Brazil. The DF is run by one local government, which decides and implements the bulk of expenditures on public education (including primary education and excluding higher education). In 2009, the local government was also responsible for the bulk of cash transfer spending under the program Bolsa Escola and Vida Melhor. The survey was conducted by surveyors hired by a local polling organization, who did not have any knowledge of the purpose of the study. They interviewed a random sample of 2,003 individuals in twelve of the twenty districts that compose the DF. The company used the sampling method they commonly use in political poll, by approaching and surveying individuals in areas of the DF with large levels of foot traffic (markets, bus terminals, shopping centers), during different days of the week. Participation in the survey was voluntary and started with socio-demographic questions.
The survey followed with the “treatments:” randomized information shocks on how the local government allocated resources in the previous years. In the first one, the No information treatment group, individuals received no information shocks and were simply asked to rate the local (state) government. This group serves as a “control group" for our analysis. In all three other treatment groups, subjects received information shocks phrased similarly. In the More education treatment group, the surveyor would read “Did you know that, compared to 2006, the state government increased in 2007 the share of total public expenditures allocated to public education by 9%?” This treatment provides information that the local government is spending more resources on public education, without revealing any tradeoff in public spending (lower share of public spending in other categories). In the More education, less cash treatment group, the surveyor would read “Did you know that, compared to 2006, the state government increased in 2007 the share of total public expenditures allocated to public education by 9% but reduced the share allocated to social assistance by 9%?” This treatment says that the local government is spending a higher fraction of resources on public education, and also reveals a possible tradeoff: the fraction of public spending associated with cash transfers has been reduced. Finally, in the More cash treatment group, the surveyor would read “Did you know that compared to the first year in the previous state government, the current state government increased in its first year the share of total public expenditures allocated to social assistance from 1.3% to 3.1%?” Each subject was then asked to rate the current administration of the local government (which started in 2007) by giving it a grade from 0 to 10. This is the main outcome of interest in this analysis.
Finally, each respondent was asked two questions. First, participants were asked “What is more important for the state government to achieve?.” Then, they were asked “Which of the following two numbers do you believe to be greater? (a) The amount actually spent in improving public education for every R$100 allocated by the local government to public education spending; (b) The amount actually spent in increasing cash transfers for every R$100 allocated by the local government to cash transfer spending. This question measures the perception of the relative effectiveness (and measure of diversion of funds/corruption) in the two types of public spending.
Researchers also assessed whether individuals update their priors about government spending by examining whether individuals exposed to the “More education, less cash" information shock associate more the state government administration under consideration with improvements in public education and less with increases in cash transfers, when compared to individuals not exposed to any information by conducting a follow-up survey with a very similar sample.
The main analysis uses OLS regression, with individual’s government rating being a dependent variable, and treatment, income, and the interaction of treatment and income as independent variables. Researchers use four different measures of low income in the analysis: (i) log of monthly household income; (ii) log of monthly personal income; (iii) a dummy for household income below the median in the sample; (iv) a dummy for household income in the first quartile of the sample. To take into account respondents that reported to have zero income and those that did not report their income level, dummy variables were added for those two sets of respondents throughout, as well as their interactions with the treatment indicators. The regression equation also includes individual controls. Standard errors are clustered at the surveyor level.
Researchers also designed and implemented an auxiliary experiment to deal with two main-intervention problems: first, because income differences are not randomly assigned, people with different incomes may differ in many unobservable dimensions; second, the poor may not think that public education spending would directly benefit them. Thus, researchers exogenously vary income and offer an education opportunity that would directly benefit the recipients. This experiment was conducted in the DF state in the first week of November 2012. The participants in the experiment were 80 randomly chosen parents who had one child enrolled in either the fourth or fifth grade in a large public school of the district of Varjao. Parents were recruited with letters distributed to the child inviting one parent to come to the school at the end of any day of the week the experiment was conducted.
Parents were offered R$5 to attend the study; the show-up rate was 83%. One surveyor was assigned to each participant to read the survey questions in a school room. Surveyors were randomly ordered at the beginning of the day and assigned according to availability throughout the rest of the day. No communication across subjects was allowed during the entire experiment. For each subject, total participation took on average around 15-20 minutes. Parents were randomly assigned to one of two treatments, according to a random number generator. In any treatment, the experiment began with the surveyor offering the parent the opportunity to receive different types of benefits. Parents were first offered an initial monthly payment for November and December 2012 (which they received with probability 1 or 0.25). The amount of this first payment was randomly varied across treatment groups: for the Low income treatment it was R$10 per month, and for the High income treatment it was R$210 per month. All cash transfers were unconditional. At the second benefit stage, subjects were asked seventeen questions, each one a choice between: (i) R$10 + R$X to be added to their initial monthly payment and (ii) R$10 to be added to their monthly payment and free, individual, weekly, three-hour long, after-school Math and Portuguese tutoring sessions for their child for November and December 2012. The amount R$X started at zero and was increased by R$5 increments question after question. Researchers added another level of (cross-)randomization, regarding the stakes of each parent's choices. Half of the parents were informed that 25% of participants in their group would have their first benefit implemented as well as one of their decisions from the seventeen questions, and that decision would be randomly chosen from the seventeen questions. For the other half, instead of 25%, the probability of implementation was 100%. This design allows for (i) the WTP elicitation for the free tutoring sessions for two months; (ii) assessment of how such WTP varies when the household level of income is randomly increased for two months. Researchers broke the benefits into two parts to make sure the seventeen questions eliciting the WTP for tutoring were identical to participants across treatment conditions. By doing this, researchers were able to reduce concerns related to visual reference points affecting the decisions; the two treatments would otherwise display different cash payment values. This in turn could have made the amount that individuals are willing to forgo for the free tutoring sessions seem relatively bigger or smaller depending on the treatment. Another important implication is that with identical questions, there are no differences in the stakes associated with each one of the seventeen questions across the two treatments. The main outcome variable - the parent's willingness to pay for tutoring – is equal to the largest amount that the parent is willing to forgo to get free tutoring for the child.
To estimate the treatment effects, researchers run OLS regression, where WTP for tutoring is the dependent variable. The regression equation controls for whether the parent faced a 100% chance of implementation of one of her choices, as opposed to 25%, and whether the parent received the High income treatment. Researchers also include log of household income, gender indicators (for the parent and for the child), age (parent and child), employed parent indicator, religion dummies, parent's marital status dummies, schooling (parent and child), number of children in the household, dummy on whether the household has been receiving conditional cash transfers from the government, parent's race dummies, number of days the child missed class in the last two months, number of grades the child has already failed, and surveyor dummies. Researchers cluster the standard errors at the surveyor level. Finally, researchers also display the treatment effects on the WTP for tutoring by examining directly across treatments the cumulative distribution for that variable.