Intervention(s)
We implement a SMS campaign to increase civil servants' compliance. Specifically, we run 3 nationwide field experiments. In the first large-scale experiment (which we call ``Benchmark Experiment"), text messages were crafted in a way that incorporates behavioral insights in dimensions related to information provision, social norms, and loss aversion as well as some weak forms of monitoring and auditing. This intervention was implemented with civil servants from a school maintenance program (from the Ministry of Education) across the country.
We run a second experiment in 2016 (called ``Follow-Up Experiment") to further explore the role of social norms and the salience of social benefits along other implementation details that might be relevant to transform this campaign in a public policy. Regarding social norms, we introduce the distinction between descriptive and injunctive norms (Cialdini et al 2004), taking into account evidence suggesting that they may have differential effects (Cialdini 2007). We also vary the reference group (school district versus the country as a whole) and the use of quantitative or qualitative norms. This approach will allow us to learn more about what type of social norms can be more effective in inducing compliance.
We also run a third experiment (which we call "External Validity Experiment") with a different national program in order to explore external validity issues. Our design does not suffer of the standard external validity problem in which a small subset of a population is part of a experiment given the large-scale nature of our intervention. However, we want to learn whether the basic results of our Benchmark Experiment can be replicated in a population of civil servants with different characteristics. We test the role of social norms and monitoring, the most promising interventions found in the Benchmark Experiment, in an monthly intervention that was implemented from September 2016 to January 2017 with all the civil servants from CUNA MAS, an early childhood development program.
We describe the details of these experiments in this section.
1. Benchmark Field Experiment
Each SMS contains a fixed and a variable component. The fixed component includes the bureaucrat's first name and the deadline for task compliance. These fixed elements are based in behavioral insights. The use of personalized messages has been shown to be an effective strategy. On the other hand, the use of exogenous deadlines has been proven to be more useful when agents suffer from procrastination. The variable component is the main behavioral lever that we use to induce a change in bureaucrats' behavior. We describe this component below.
Maintenance bureaucrats were assigned to one of six groups. Bureaucrats in the control group receive no SMS. The remaining bureaucrats receive an SMS with behavioral content at fixed points during the intervention cycle. In total, each bureaucrat in any of the treatment groups receives up to five SMS. These SMS shared the same behavioral insight over the cycle but varied in terms of the type of maintenance activity that was emphasized. For instance, near the beginning of the intervention cycle, bureaucrats receive SMS emphasizing the withdrawal of maintenance funds, whereas near the end of the cycle SMS emphasize the filing of expense reports. Bureaucrats only receive a particular if they have not complied with the activity being emphasized in that SMS.
Bureaucrats in the reminder/warning treatment receive SMS with an alert and the URL of the PRONIED website where the bureaucrat can obtain more information. Reminders are one of the most popular tools used in behavioral science to influence behavior and the inclusion of an alert is motivated by the need to prime a sense of urgency to comply with maintenance activities. Reminders are motivated by the existence of limited attention problems and are tools that can potentially change the inter-temporal allocation of mental resources to enforce compliance.
Bureaucrats in the monitoring treatment receive SMS with information regarding the amount of transfers not yet withdrawn from the bank or not yet declared on the expense report, depending on the timing of the message in the intervention cycle. It is expected that this information creates the impression on bureaucrats that their actions are being observed and, as a consequence, comply with maintenance policies. This treatment should not be surprising for a fully rational agent since it is perfect knowledge that the program is able to observe funds withdrawal and the expense reporting. Therefore, by making salient a fact that is common knowledge among civil servants, it is possible to re-create some critical dimensions of monitoring systems in a cost-effective way.
Bureaucrats in the social norm treatment receive SMS with a message emphasizing that most bureaucrats are complying in their reference group (UGEL). Social norms are understood in this paper as a set of informal rules and unwritten codes that establish what we expect of others and what others expect from us. Following Cialdini et al. 2004, it is possible to establish a useful distinction between norms that inform us about what is typically done (descriptive norms) and norms that inform us about what is typically approved or disapproved (injunctive norms). We used a qualitative descriptive norm to minimize the risk of backlash effects, considering a body of evidence that suggests that providing actual levels of conformity with a social norm can induce more people to deviate from it if their baseline expectations regarding conformity with the norm were higher. In the follow-up experiment we further explore variants of social norms, including quantitative norms and alternative reference groups.
Bureaucrats in the shaming treatment receive SMS with information regarding the possible publication of a list with the names of those bureaucrats who fail to comply with the reporting of expenses. The goal is to induce concern regarding potential reputational loss in order to motivate compliance, especially when baseline non-compliance behavior is deeply rooted. This treatment arm is based on a large body of evidence indicating that people are more likely to comply when their behaviors are observed.
Finally, bureaucrats in the auditing treatment receive SMS with a soft threat of auditing. Specifically, they are told that they will be visited for supervision of their maintenance activities. Schools are already visited on regular basis by UGEL representatives for several matters, including (of course) the development of maintenance activities. In that sense, the intervention is simply making salient an event that civil servants will face over the course of the academic year. However, given the scale of the intervention, the probability of facing a visit is low at a given moment of time. We take advantage of this fact to induce compliance among civil servants by reminding them about the fact that they will be visited by UGEL officials.
2. Follow-Up Field Experiment
We implemented a new large-scale field experiment in 2016 with the goal of further exploring the role of non-monetary incentives. Taking as starting point the results for the benchmark experiment in 2015, we designed a new large scale intervention to address the following questions:
a) What type of social norms are more relevant?
b) Does making salient the social benefits of investing in school infrastructure an alternative way to enforce compliance among civil servants?
c) Are the effects persistent over time?
d) Does the duration of the SMS campaign matter?
Regarding a), we extend our analysis of social norms by incorporating treatments targeted to address the distinction between descriptive and injunctive social norms. We further explore the role of social norms by breaking down the descriptive social norm treatment into quantitative and qualitative versions as well as modifying the reference group. We proceed in the same way to break down the injunctive social norm into 2 reference groups: parents and principals.
With respect to b), we vary the dimension of social benefit to consider messages that emphasize the importance of a good quality infrastructure for students' health (well-being social benefit treatment), for the pride of the school community (pride social benefit treatment) and for contributing to the students' learning process (learning social benefit treatment).
c) addresses whether civil servants can be induced to comply with the maintenance policies beyond a one-shot SMS.
Finally, d) relates to understanding a critical component of a SMS campaign design: the treatment duration. We experimentally vary the number of SMS delivered to civil servants. One group receives a short duration SMS campaign of 4 SMS delivered in a given period. A second group receives a long duration campaign of 7 SMS.
3. External Validity Field Experiment
We run an additional field experiment in a different population of civil servants to shed light of the applicability of our intervention in other settings. Although we recognize that running additional experiments would not exhaust all relevant dimensions of external validity, we believe this exercise will shed light on relevant issues to consider when it comes to understanding the applicability of our results to other settings. In addition, by implementing the Follow-Up Experiment in 2016 with the population of civil servants, we also address the external validity of our results by controlling for aggregate time-specific shocks (Rosenzweig et al 2016).