|
Field
Abstract
|
Before
This study proposes a Randomized Control Trial impact evaluation of providing internal efficiency information to public and private, primary and secondary, and urban and rural schools in Argentina. The information is provided through brief and easy-to-understand reports that will allow schools to compare themselves with (a) their country, province and locality, in the most recent year; and (b) with itself, in the last three years. The reports will also include a letter from the national and provinicial ministries to signal the importance of the use of information, an executive summary that extracts the main points of the presented information and some questions to accompany the principal's and teachers' reading of the report.
|
After
Many developing countries provide schools with diagnostic feedback on their performance. Yet, evidence on these initiatives focuses on the effect of providing information on student achievement. We conducted four experiments in Argentina to estimate the effect of providing schools with information on internal efficiency (e.g., passing, repetition, and dropout rates), which is more readily available and easier to understand.
In 2017, we carried out two studies. In study 1, which covered 23 out of Argentina’s 24 provinces, we randomly assigned schools to one of three groups: (a) one in which they received report cards that compared their internal efficiency indicators to those of all other schools in the country; (b) one in which they received similar reports that only compared them to schools of the same management type (i.e., public or private); or (c) one that did not receive any reports. In study 2, in the Province of Tucumán, we randomly assigned schools to one of four groups: (a) one of the three groups in study 1; or (b) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country and a professional development workshop for principals.
In 2018, we conducted two more studies. In study 3, which covered 23 provinces, we randomly assigned schools to one of two groups: (a) one in which they received report cards that compared their internal efficiency indicators to those of all other schools in the country; or (b) one that did not receive any report cards. In study 4, in the Province of Salta, we randomly assigned schools to: (a) a group that received a print version of the report; or (b) one that did not receive any reports.
|
|
Field
Last Published
|
Before
June 04, 2018 09:16 PM
|
After
August 02, 2018 06:49 PM
|
|
Field
Primary Outcomes (End Points)
|
Before
school enrollment, grade promotion, grade repetition, drop-out rates, learning outcomes
|
After
number of enrolled students, passing rates, grade failure rates, repetition rates, dropout rates, learning outcomes
|
|
Field
Experimental Design (Public)
|
Before
In 23 of 24 provinces, schools were randomly assigned to one of two groups: group 1, which receives internal efficiency information in the beginning of 2017; or group 2, which receives internal efficiency information in the beginning of 2018. In addition, there will be two versions of group 1: group 1A, which receives information in which the comparison will be done with all schools (public and private); and group 1B, which will receive information in which the comparison will be done with schools with the same type of management (in other words, public schools will be compared will other public schools, and private schools will be compared with other private schools)
Randomization was done with three stratification levels: province, education level and size (specifically, terciles of total number of students, including primary and secondary). The purpose of the stratification was to maximize comparability of schools in the different experimental groups, increasing statistical power. This strategy resulted in 7,269 schools in Group 1A, 7313 schools in group 1B and 7,360 schools in group 2.
In one of the 24 provinces, schools were randomly assigned to one of four groups: groups 1A and 1B that receive internal efficiency information in the beginning of 2017; group 2 that receives internal efficiency information in the end of 2018; and group 3 that receives internal efficiency information and training to understand this information in the beginning of 2017. In this province, randomized assignment was also stratified as in the other 23 provinces. In other words, two lotteries were done. This strategy resulted in 211 schools in group 1A, 215 schools in group 1B, 213 in group 2, and 216 school in group 3.
|
After
In 2017, we carried out two studies. In study 1, which covered 23 out of Argentina’s 24 provinces, we randomly assigned 14,404 primary schools and 8,016 secondary schools to one of three groups: (a) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country; (b) one in which schools received similar report cards that only compared them to schools of the same management type (i.e., public or private); or (c) one that did not receive any reports. In study 2, in one of the 24 provinces, we randomly assigned 605 primary schools and 398 secondary schools to one of four groups: (a) one of the three groups in study 1; or (b) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country and a professional development workshop for principals. All treatment schools in 2017 accessed the report cards through an online platform.
In 2018, we conducted two more studies. In study 3, which covered 23 provinces, we randomly assigned 13,776 primary schools and 8,109 secondary schools to one of two groups: (a) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country; or (b) one that did not receive any report cards. All treatment schools accessed the report cards through the same online platform from 2017, but we made it more difficult for schools to opt out of the report. In study 4, in one of the 24 provinces, we randomly assigned 629 primary schools and 241 secondary schools to: (a) a group that received a print version of the report; or (b) one that did not receive any reports.
|
|
Field
Randomization Method
|
Before
Randomization done in office by a computer
|
After
Randomization was conducted in Stata 15 in a computer
|
|
Field
Planned Number of Observations
|
Before
22,797 schools
|
After
Study 1: 14,404 primary schools and 8,016 secondary schools
Study 2: 605 primary schools and 398 secondary schools
Study 3: 13,776 primary schools and 8,109 secondary schools
Study 4: 629 primary schools and 241 secondary schools
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
As explained above, in 23 of 24 provinces, the assignment resulted in 7,269 schools in Group 1A, 7313 schools in group 1B and 7,360 schools in group 2. In 1 of 24 provinces, the randomization resulted in 211 schools in group 1A, 215 schools in group 1B, 213 in group 2, and 216 school in group 3.
|
After
We ran two lotteries in 2017. In study 1, which covered 23 out of Argentina’s 24 provinces, we randomly assigned 14,404 primary schools and 8,016 secondary schools to one of three groups: (a) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country (group 1A); (b) one in which schools received similar report cards that only compared them to schools of the same management type (i.e., public or private) (group 1B); or (c) one that did not receive any reports (group 2). In study 2, in one of the 24 provinces, we randomly assigned 605 primary schools and 398 secondary schools to one of four groups: (a) one of the three groups in study 1; or (b) one in which schools received report cards that compared their internal efficiency to that of all other schools in the country and a professional development workshop for principals (group 3).
We also ran two lotteries in 2018. In study 3, which covered 23 provinces, we randomly assigned 13,776 primary schools and 8,109 secondary schools to one of two groups: (a) one in which schools received report cards that compared their internal efficiency indicators to those of all other schools in the country (group 1A); or (b) one that did not receive any reports (group 2). In study 4, in one of the 24 provinces, we randomly assigned 629 primary schools and 241 secondary schools to one of three groups: (a) one in which schools received web-based report cards that compared their internal efficiency to that of all other schools in the country (group 1A); (b) one in which schools received similar printed report cards (group 1C); or (b) one that did not receive any reports (group 2).
|