Evaluating the Effects of School Reopening in the Pandemic: Evidence from São Paulo

To measure the impact of school reopening, we will take advantage of the fact that schools will reopen progressively. We combine administrative data for the universe of students and school staff and mobile survey data for a sample comprising both students and school staff enrolled and working in schools that have reopened and those enrolled and working in schools that are still closed.

Approximately 20% of the municipalities in São Paulo State authorized schools to reopen for optional sports and cultural activities on September 7, 2020 and for regular activities (only for high school students) on October 7, 2020. The return of school activities might depend on several characteristics, such as the state of the pandemic in the municipality, mayor's characteristics, etc.

In this project, we will combine quasi-experimental and experimental methods to evaluate the causal effect of school reopening on the outcomes of interest. The experiment will comprise an encouragement design, in which we send nudges to motivate students to return to
school -- randomly assigning within municipalities where schools are open. We will send motivation SMS to students themselves (grades 10 to 12) or a students' parent or legal guardian (grades 6 to 9, if those are allowed to resume regular
classes in the months that follow).

We can use this experiment to assess the reduced-form effects of school reopening on the outcomes of interest by simply comparing treated schools with non-treated schools within municipalities that chose to reopen schools. If we have access to attendance data for the schools that reopen, from administrative records, we can use the assignment to nudges as an instrumental variable to estimate the causal effect of school reopening on health and education outcomes.

In order for the simple comparison described above to generate causal effects, we need to assume that the messages do not have a direct effect on the outcomes of interest but affect them only through the attendance in reopened schools. We will verify if this is the case by sending placebo messages. Specifically, we will send the same messages to students enrolled in schools that did not reopen. Since we will have information on students in both types of schools, we can evaluate if messages have a direct effect on the outcomes of interest.

2020-11-16

2021-01-29

We will assess the effects of schools reopening on the following variables - obtained both through administrative data and mobile based phone surveys:
 Quarterly grades, by school subject, based on these activities (admin
data);
 Student dropouts (admin data);
 Number of positive Covid-19 cases in the family of students, school staff and municipalities that reopen (admin and phone survey data);
 Number of Covid-19 related hospitalizations in the family of students, school staff and municipalities that reopen (admin and phone survey data);
 Number of Covid-19 deaths in the family of students, school staff and municipalities that reopen (admin and phone survey data).
 Labor supply of household members (phone survey only).

Besides, we also have information on various other baseline variables: the history of access to the distance learning platform since May; attendance and grades for the rst quarter of the year, provided by administrative records; predicted risk of dropouts at the student-level, on a 0-100 scale. We will use them for estimating heterogeneous treatment effects.

We plan on reaching at least 18.258 students equally divided between the treatment, placebo, and control groups (although the intervention might end up reaching as many as 400,000 students).

The content of the messages will be the same, but the students in the treatment group will be randomly assigned among those in municipalities that reopen schools. The placebo group, on the other hand, will be composed of a random set of students of the municipalities that did not reopen schools.

We will send two messages a month for each student during three months. To implement the intervention, we will partner with Brazilian EdTech Movva, specialized in nudges via text messages, withexpertise in implementing randomized trials and partnering with researchers to
conduct evaluations to assess the effectiveness of programs and policies. Movva is in charge of designing and sending text messages in the context of our experimental intervention.

Randomization done in office by a computer by Movva

Schools, stratified by municipality.

Yes

At least 142 (up to 644) municipalities

At least 18,000 (up to 664,500) high-school students

1/3 in the treatment group (nudges in municipalities where schools reopen), 1/3 in the placebo group (nudges in municipalities where schools do not reopen), 1/3 in the control group (no nudges)

This design allows us to detect an effect size of nudges on attendance of at least 0.039 s.d., about 2% of the mean attendance in schools that reopen.

Pre-Analysis Plan