An informational campaign prompting the choice of academic-oriented and college-bound secondary track in Hungary – a cluster randomized field experiment

Last registered on December 06, 2018

Pre-Trial

General Information

Title

RCT ID

AEARCTR-0003648

Initial registration date

December 05, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

December 06, 2018, 8:17 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Country

Hungary

Region

Primary Investigator

Name

Tamas Keller

Affiliation

Research Center for Educational and Network Studies, Hungarian Academy of Sciences, Centre for Social Sciences; TÁRKI Social Research Institute, Budapest

Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status

On going

Start date

2018-11-06

End date

2020-09-30

Keywords

Education

Additional Keywords

secondary school track choice, non-economic information campaign, uncertain admission standards, empowerment, cluster randomized field experiment

JEL code(s)

I20, I21, H75

Secondary IDs

Abstract

We pre-register a pair-matched cluster-randomized field experiment involving last (eighth) year primary school students in Hungary. Randomization is done at the classroom level on data gathered in Spring 2018. We designed a non-economic information campaign. Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic oriented high schools, the secondary grammar schools. Students in treated classrooms will receive the intervention while students in control classroom do not receive any treatment. Our primary outcome is the application of students to secondary grammar schools, measured by register data obtained from the Hungarian Educational Authority. We pre-register primary and secondary analyses. In the primary analysis we hypothesize that receiving the treatment about the admission standards will increase the likelihood of applying to grammar schools. In our secondary, exploratory, analysis we hypothesize that the intervention will have (1) larger benefit for those whose parents do not have high-school final examinations; (2) that it will have a greater influence on those who already intended to apply to secondary grammar schools; and (3) that it will have greater effect on those who originally intended to apply but believed that their own admission chances are weak. We submit the pre-registration plan before finishing the treatment. We are collecting baseline data about the students before the treatment in Fall 2018 from teachers’ reports and via students’ questionnaires. These data will be available in Spring 2019. Endline data will be available to researchers in Fall 2019. We pre-register to publish the results after a carrying out a similar intervention in Fall 2019 (so the data of two cohorts will be published). We also pre-register the possible circumstances under which we will not conduct the intervention the subsequent year, which means publishing the data of only one cohort.

External Link(s)

Registration Citation

Citation

Keller, Tamas. 2018. "An informational campaign prompting the choice of academic-oriented and college-bound secondary track in Hungary – a cluster randomized field experiment." AEA RCT Registry. December 06. https://doi.org/10.1257/rct.3648-1.0

Former Citation

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic-oriented high schools, the secondary grammar schools.

Intervention Start Date

2018-11-06

Intervention End Date

2018-12-07

Primary Outcomes

Primary Outcomes (end points)

Our primary outcome is the application of students to secondary grammar schools, measured by register data obtained from the Hungarian Educational Authority.

Primary Outcomes (explanation)

Outcome variables will be provided by the Hungarian Educational Authority (HEA) based on consented students’ ID. Parallel to the data request from HEA we will obtain the same information from primary schools. We will use school reported outcomes if HEA does not provide us the data for whatever reason. We will fill in missing HEA outcomes from the school reported outcomes if HEA is not able to provide information about a student due to the damaged or wrongly reported anonymous student ID. Outcome variables will be available in Fall 2019.

Secondary Outcomes

Secondary Outcomes (end points)

• whether students applied to grammar school (=1) in places 1-3 on their application list.
• whether students are admitted (=1) to grammar school.

Secondary Outcomes (explanation)

Approximately 94% of students are admitted to one of the first three choices (Applied_1_3)

Experimental Design

We designed a non-economic information campaign. Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic-oriented high schools, the secondary grammar schools. Students in treated classrooms will receive the intervention while students in the control classroom do not receive any treatment. We randomized at the classroom level based on eleven individual level baseline variables collected in Spring 2018.

Experimental Design Details

Randomization Method

We randomized at the classroom level based on eleven individual level baseline variables collected in Spring 2018 we extracted the first principal component (blocking score). We calculated the classroom average of this blocking score. We ranked the classes according to the blocking score and identified the most similar class-pairs. We then randomly assigned one class within the pairs to the treated and the other to the control group using a random number generator. In order to optimize balance, we randomized pairs 1000 times, computed p-values for balance on each covariate, and chose the randomization that maximized the minimal p-value across covariates.

Randomization Unit

We randomized classrooms within schools. The randomization resulted in a classification where in three schools (out of the 34) parallel classes were both in the treated and control group. Since our previous study (Keller, Takács, & Elwert, 2018) found no sign for spillover effects within classrooms, we expect no spillover (contamination) across classrooms.

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

We pre-register the sample of eighth-graders in the academic year of 2018/19. This sample contains 34 schools and 40 classes. The sample size in the analytic sample is not yet known.
We pre-register to repeat the same informational campaign in the following academic year (2019/2020) for those who are now in seventh grade and will be in the eighth year next year. This will be done in September 2019. We know, however, that in our database we have 42 seventh-grade classes in 34 schools. We will not do the informational campaign next year if
• more than 6 schools (20%) do not cooperate with us next year for whatever reasons,
• more than 40% of students in at least 6 schools do not have parental consent.
In these cases, the new eighth-grade cohort will not substantially increase the statistical power, therefore there is no point in including them in the analysis. We will also not extend the trial for another year if political or academic circumstances prevent the extension. We will make the final decision about doing the fieldwork in the next academic year in late September 2019.

Sample size: planned number of observations

We plan to have 20 students in each classroom. However, the exact sample size is not yet known.

Sample size (or number of clusters) by treatment arms

In the academic year of 2018/19 we have 20 treated and 20 control classrooms.
For the following academic year (2019/2020) the randomization is not done since we do not know the number of participating schools

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

We have calculated with average 20 students (n) in 2 blocks since in each class-pair (J) there is one treated and one control class, and for 20 classrooms (K) in each class-pair. These parameters give the minimum detectable effect of 0.29. If we can conduct the same experiment next year with an additional 20 classrooms in the treated and control groups, the minimum detectable effect will be 0.2.

Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

IRB