Public Call for the Connected Education Program

Last registered on March 08, 2019

View Trial History

Pre-Trial

Trial Information

General Information

Title

Public Call for the Connected Education Program

RCT ID

AEARCTR-0003933

Initial registration date

March 07, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

March 08, 2019, 3:56 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Brazil

Region

South America

Primary Investigator

Name

Tassia Cruz

Affiliation

Fundação Getulio Vargas

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Daniel Grimaldi

PI Affiliation

BNDES

Contact Investigator

PI Name

Fabio Roitman

PI Affiliation

BNDES

Contact Investigator

Additional Trial Information

Status

In development

Start date

2018-04-27

End date

2022-12-31

Keywords

Education

Additional Keywords

Technology

JEL code(s)

I280

Secondary IDs

Abstract

This study analyzes the impact of a comprehensive technology project in Brazilian public schools. This project is the first structured initiative with the objective of fostering the adoption of technology within a larger policy, the “Connected Education Program.” This initiative – called Public Call for the Connected Education Program – provides additional funds for a group of randomly selected schools to implement the program. These additional funds are at least 300% higher than what this group of schools was receiving from other sources to implement technology projects. The Connected Education Program was developed under the premise that the potential of technology in education is only achieved by integrating four dimensions – vision, infrastructure, expertise and content – which must be in balance. This study focuses on two key research questions. First, does the “Public Call for the Connected Education Program” lead to an improvement in student learning? Second, what are the mechanisms of change? We will conduct a randomized controlled trial within the municipalities selected to receive the intervention. Our study is designed as a matched-pair cluster randomization with three levels (municipalities, schools and students) and treatment at level two (schools). Within the municipalities selected to receive support we have randomized 50% of the municipal schools to receive the treatment. Within the schools, we will evaluate school, teachers and students’ outcomes. The evaluation will focus on students of the 5th grade municipal schools. Our key hypothesis is that the interaction between the four dimensions is what will generate the intervention’s impact.

External Link(s)

Website for the Public Call (for municipalities to enroll in the program)

Registration Citation

Citation

Cruz, Tassia, Daniel Grimaldi and Fabio Roitman. 2019. "Public Call for the Connected Education Program." AEA RCT Registry. March 08. https://doi.org/10.1257/rct.3933-1.0

Former Citation

Cruz, Tassia, Daniel Grimaldi and Fabio Roitman. 2019. "Public Call for the Connected Education Program." AEA RCT Registry. March 08. https://www.socialscienceregistry.org/trials/3933/history/42825

Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Experimental Details

Interventions

Intervention(s)

This Public Call is the result of a partnership between the Brazilian Development Bank (BNDES) and the Ministry of Education (MEC). The objective of this Public Call is to select projects for the implementation and use of digital technologies for public education within the scope of the Connected Education Program, in order to support its adoption in public education networks. Projects should include actions and/or investments integrated into the four structuring dimensions of the Program - "vision / training / digital educational resources / infrastructure" - to provide for investments that can be allocated to: (i) educational and intangible resources such as infrastructure, equipment, services, knowledge acquisition, training and improvement for professionals and managers of education, tools, methodologies, practices and contents; (ii) governance, planning, social mobilization and management of public education networks and schools; (iii) innovative initiatives for education in the third millennium. Each Brazilian state submitted a joint proposal with two municipalities to the Public Call.

Intervention Start Date

2019-01-01

Intervention End Date

2020-12-31

Primary Outcomes

Primary Outcomes (end points)

Our main goal is to analyze student learning’ outcomes. For this analysis, we will use the publicly available SAEB (Brazilian Basic Education Assessment System) results in Math and Reading for 5th grade students. In addition to SAEB results, we will collect data on the number of years students in the treated schools have attended that school from 2019 to 2021.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

In addition to our main question, we are interested in analyzing the mechanisms of change of the program. We have divided the indicators of mechanisms in three groups. First, the immediate outputs, referring to the availability, use and appropriation of technology by school principals, teachers and students. Second, the intermediate results, referring to the changes in behavior by teachers and school principals. Third, the students’ changes in engagement and motivation.

Secondary Outcomes (explanation)

To calculate the availability of technology in the schools, we will use the Brazilian School Census information. To assess the use and appropriation of technology, we will adopt several measures, including administrative data (SAEB questionnaires) and primary data collection. We will collect data on the use of technology by teachers and school principals using a questionnaire prepared by CIEB (Innovation Center for Brazilian Education - http://www.cieb.net.br/) for this purpose. A baseline of this questionnaire was applied to all municipal and state systems that sent proposals to BNDES to participate in this project.
For the intermediate results, we will look at two transmission channels: The first transmission channel refers to the use of teacher time, and the second refers to efficiency in school administration. We plan to collect information about the teacher use of time in the classroom using the TEACH instrument of classroom observation (http://documents.worldbank.org/curated/en/949541542659103528/Teach-Observer-Manual). We have also proposed the Trial partners to use the World Management Survey to evaluate efficiency in school administration.
The third mechanism of change indicates variations in student behavior. For the changes in student behavior, we will also collect primary data with questionnaires about students’ socioemotional measures with a questionnaire prepared specifically for evaluating students’ engagement and motivation due to technology use, and another questionnaire designed to evaluate the Big Five Personality Traits.

Experimental Design

As the main sponsor, BNDES has compromised to conduct a randomized controlled trial within the municipalities selected to receive the intervention. Our assessment is designed in three levels (municipalities, schools and students) with treatment at level two (schools). Within the municipalities selected to receive support we have randomly selected 50% of the municipal schools to receive the treatment. Within the schools, we will evaluate student outcomes. The randomized evaluation will focus on 5th grade students of municipal schools.
Treated schools are those that receive the four dimensions of the Program – vision, infrastructure, expertise and content – versus the results of schools in the municipalities that do not receive these four dimensions (control schools). Our key hypothesis is that the interaction between the four dimensions is what will lead to the intervention’s impact. The control group may be affected by transversal or ad hoc actions, but will not be supported by a coordinated effort that considers the balance of the four dimensions.

Experimental Design Details

We will use a matched-pair cluster randomization design. For that, we paired the schools within the municipalities according to their baseline prediction of the outcome, and randomly assigned each school in the pair to treatment or control. The results of the lottery were published on December 5, 2018. The randomization strategy was divided in three steps: (1) Definition of clusters; (2) Pairing; and (3) Lottery.
The most common profile of schools in the sample is of schools on the first half of elementary education (with 5th grade). Therefore, we defined the clusters considering the following blocks: (i) municipalities; (ii) availability of SAEB data in the school; (iii) types of schools (having or not the second half of elementary education in the school, that is, from 6th to 9th grades).
Second, to select the variables to pair the schools, we ran an Adalasso model using the variables of the 2015 SAEB (Brazilian Basic Education Assessment System) and the 2015 School Census to explain the 5th grade students’ results in 2017 SAEB test. We selected the smallest model whose mean squared error is within a standard deviation of the model that minimizes the mean squared error. We used the selected model to predict the 2019 results using the 2017 data, and then we paired on the predicted value of 2019 test results in Mathematics and Reading.

Randomization Method

The random selection of schools was done through public lottery. We randomly assigned each school to a group of numbers of the public lottery, which were publicly disclosed by BNDES. The schools were selected in two stages. The first-stage selection was to deal with municipalities with an odd number of schools. In these cases, the public lottery results of December 1st, 2018 have determined one school to be assigned to treatment. Next, we paired the schools according to the predicted 2019 results, and each school in the pair was randomly assigned to be part of group one or two (referring to odd and even numbers, respectively). The public lottery results of December 5th, 2018 have determined the final treatment assignment.

Randomization Unit

School

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

There are 332 schools in the 11 municipalities of the sample, to be randomized to treatment and control. In 2017, 295 schools of these had 5th grade students (which is the focus of our study).

Sample size: planned number of observations

There are 14,328 students in 5th grade of the 295 municipal schools.

Sample size (or number of clusters) by treatment arms

There are 170 treatment schools and 162 control schools in the 11 municipalities. For the focus of our study, there are 150 treatment schools and 145 control schools with 5th grade (in 2017).

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Our MDES is of 0.0301 standard deviations of 5th grade math performance in SAEB.

Supporting Documents and Materials

IRB