NEW UPDATE: Completed trials may now upload and register supplementary documents (e.g. null results reports, populated pre-analysis plans, or post-trial results reports) in the Post Trial section under Reports, Papers, & Other Materials.
An informational campaign prompting the choice of academic-oriented and college-bound secondary track in Hungary – a cluster randomized field experiment
Initial registration date
December 05, 2018
December 06, 2018 8:17 AM EST
This section is unavailable to the public. Use the button below
to request access to this information.
Research Center for Educational and Network Studies, Hungarian Academy of Sciences, Centre for Social Sciences; TÁRKI Social Research Institute, Budapest
Other Primary Investigator(s)
Additional Trial Information
We pre-register a pair-matched cluster-randomized field experiment involving last (eighth) year primary school students in Hungary. Randomization is done at the classroom level on data gathered in Spring 2018. We designed a non-economic information campaign. Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic oriented high schools, the secondary grammar schools. Students in treated classrooms will receive the intervention while students in control classroom do not receive any treatment. Our primary outcome is the application of students to secondary grammar schools, measured by register data obtained from the Hungarian Educational Authority. We pre-register primary and secondary analyses. In the primary analysis we hypothesize that receiving the treatment about the admission standards will increase the likelihood of applying to grammar schools. In our secondary, exploratory, analysis we hypothesize that the intervention will have (1) larger benefit for those whose parents do not have high-school final examinations; (2) that it will have a greater influence on those who already intended to apply to secondary grammar schools; and (3) that it will have greater effect on those who originally intended to apply but believed that their own admission chances are weak. We submit the pre-registration plan before finishing the treatment. We are collecting baseline data about the students before the treatment in Fall 2018 from teachers’ reports and via students’ questionnaires. These data will be available in Spring 2019. Endline data will be available to researchers in Fall 2019. We pre-register to publish the results after a carrying out a similar intervention in Fall 2019 (so the data of two cohorts will be published). We also pre-register the possible circumstances under which we will not conduct the intervention the subsequent year, which means publishing the data of only one cohort.
Keller, Tamas. 2018. "An informational campaign prompting the choice of academic-oriented and college-bound secondary track in Hungary – a cluster randomized field experiment." AEA RCT Registry. December 06.
Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic-oriented high schools, the secondary grammar schools.
Intervention Start Date
Intervention End Date
Primary Outcomes (end points)
Our primary outcome is the application of students to secondary grammar schools, measured by register data obtained from the Hungarian Educational Authority.
Primary Outcomes (explanation)
Outcome variables will be provided by the Hungarian Educational Authority (HEA) based on consented students’ ID. Parallel to the data request from HEA we will obtain the same information from primary schools. We will use school reported outcomes if HEA does not provide us the data for whatever reason. We will fill in missing HEA outcomes from the school reported outcomes if HEA is not able to provide information about a student due to the damaged or wrongly reported anonymous student ID. Outcome variables will be available in Fall 2019.
Secondary Outcomes (end points)
• whether students applied to grammar school (=1) in places 1-3 on their application list.
• whether students are admitted (=1) to grammar school.
Secondary Outcomes (explanation)
Approximately 94% of students are admitted to one of the first three choices (Applied_1_3)
We designed a non-economic information campaign. Our treatment is communicating otherwise unknown admission standards, and to encourage primary school students to apply to academic-oriented high schools, the secondary grammar schools. Students in treated classrooms will receive the intervention while students in the control classroom do not receive any treatment. We randomized at the classroom level based on eleven individual level baseline variables collected in Spring 2018.
Experimental Design Details
We randomized at the classroom level based on eleven individual level baseline variables collected in Spring 2018 we extracted the first principal component (blocking score). We calculated the classroom average of this blocking score. We ranked the classes according to the blocking score and identified the most similar class-pairs. We then randomly assigned one class within the pairs to the treated and the other to the control group using a random number generator. In order to optimize balance, we randomized pairs 1000 times, computed p-values for balance on each covariate, and chose the randomization that maximized the minimal p-value across covariates.
We randomized classrooms within schools. The randomization resulted in a classification where in three schools (out of the 34) parallel classes were both in the treated and control group. Since our previous study (Keller, Takács, & Elwert, 2018) found no sign for spillover effects within classrooms, we expect no spillover (contamination) across classrooms.
Was the treatment clustered?
Sample size: planned number of clusters
We pre-register the sample of eighth-graders in the academic year of 2018/19. This sample contains 34 schools and 40 classes. The sample size in the analytic sample is not yet known.
We pre-register to repeat the same informational campaign in the following academic year (2019/2020) for those who are now in seventh grade and will be in the eighth year next year. This will be done in September 2019. We know, however, that in our database we have 42 seventh-grade classes in 34 schools. We will not do the informational campaign next year if • more than 6 schools (20%) do not cooperate with us next year for whatever reasons,
• more than 40% of students in at least 6 schools do not have parental consent.
In these cases, the new eighth-grade cohort will not substantially increase the statistical power, therefore there is no point in including them in the analysis. We will also not extend the trial for another year if political or academic circumstances prevent the extension. We will make the final decision about doing the fieldwork in the next academic year in late September 2019.
Sample size: planned number of observations
We plan to have 20 students in each classroom. However, the exact sample size is not yet known.
Sample size (or number of clusters) by treatment arms
In the academic year of 2018/19 we have 20 treated and 20 control classrooms.
For the following academic year (2019/2020) the randomization is not done since we do not know the number of participating schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We have calculated with average 20 students (n) in 2 blocks since in each class-pair (J) there is one treated and one control class, and for 20 classrooms (K) in each class-pair. These parameters give the minimum detectable effect of 0.29. If we can conduct the same experiment next year with an additional 20 classrooms in the treated and control groups, the minimum detectable effect will be 0.2.
INSTITUTIONAL REVIEW BOARDS (IRBs)
Hungarian Academy of Sciences
IRB Approval Date
IRB Approval Number