Do teacher assessments and track recommendations discriminate against Roma minority students? – A randomized experiment among Hungarian primary school teachers
Last registered on June 18, 2021

Pre-Trial

Trial Information
General Information
Title
Do teacher assessments and track recommendations discriminate against Roma minority students? – A randomized experiment among Hungarian primary school teachers
RCT ID
AEARCTR-0007838
Initial registration date
June 18, 2021
Last updated
June 18, 2021 12:57 PM EDT
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
Centre for Social Sciences, Budapest
Other Primary Investigator(s)
PI Affiliation
Centre for Economic and Regional Studies, Institute of Economics,Budapest, Hungary; Centre for Labour Economics, Corvinus University of Budapest, Budapest, Hungary
PI Affiliation
Centre for Economic and Regional Studies, Institute of Economics,Budapest, Hungary; Centre for Social Sciences, Budapest Hungary
Additional Trial Information
Status
On going
Start date
2021-06-18
End date
2021-12-31
Secondary IDs
Abstract
Conducting a randomized experiment, we examine whether discrimination exists among teachers against Roma minority students in Hungary. Our target population consists of teachers who teach Hungarian grammar and literature or mathematics in the 5th-8th grades of Hungarian primary schools. The experiment will be carried out online. We will contact every Hungarian primary school with an invitation letter sent to the schools’ central e-mail address that is publicly available. The first 200 mathematics and 200 Hungarian and grammar school teachers who are willing to participate can take part in the experiment. Teachers will be asked to evaluate six mathematics/Hungarian tests and recommend a secondary school track to the fictive student. Students’ names on the tests will be randomized (Roma male, Roma female, non-Roma male, non-Roma female names). We hypothesize that tests with Roma names receive lower evaluations and track recommendations on average than tests with non-Roma names. We test our hypotheses estimating linear regression models with teacher and test fixed effects. We preregister the analysis plan before having any endline data.
External Link(s)
Registration Citation
Citation
Kisfalusi, Dorottya, Zoltán Hermann and Tamás Keller. 2021. "Do teacher assessments and track recommendations discriminate against Roma minority students? – A randomized experiment among Hungarian primary school teachers." AEA RCT Registry. June 18. https://doi.org/10.1257/rct.7838-2.0.
Experimental Details
Interventions
Intervention(s)
Our experimental manipulation consists of a random manipulation of students’ names (Roma male, Roma female, non-Roma male, non-Roma female names) on mathematics/Hungarian tests evaluated by primary school teachers.
Intervention Start Date
2021-06-18
Intervention End Date
2021-09-30
Primary Outcomes
Primary Outcomes (end points)
1. total points assigned to the test by the teachers 2. track recommendations by the teachers
Primary Outcomes (explanation)
We have two primary outcome variables:
1. The total points assigned to the test, which ranges from 0 to 30 in the case of both subjects. The mathematics test consists of 6 exercises with the following maximum points: 5,4,4,5,6,6. The literature test consists of 5 exercises with the following maximum points: 8,4,4,4,10.
2. The recommended secondary school track (1 - vocational, 2 – secondary vocational, 3 – secondary grammar), recoded into a dummy variable (1 – vocational, 0 – secondary vocational or secondary grammar) for the confirmatory analysis.
Secondary Outcomes
Secondary Outcomes (end points)
grade assigned to the test by the teachers
Secondary Outcomes (explanation)
The grade assigned to the test (integer between 1 and 5)
Experimental Design
Experimental Design
Teachers have four tasks in the experiment:
1. They evaluate six mathematics or Hungarian grammar and literature tests on a 30-point scale first. Students’ names on the tests will be fictive and randomly assigned (Roma male, Roma female, non-Roma male, non-Roma female names).
2. Teachers recommend a grade based on the evaluation of the test on the 5-grade scale used in the Hungarian educational system (1 = fail, 2 = pass, 3 = satisfactory, 4 = good, 5 = excellent).
3. They recommend a secondary school track to the students whose tests they evaluated. (1 – vocational school, 2 – secondary vocational school, 3 – secondary grammar school). Besides the points and grades assigned by the teacher in the given subject, we will provide an additional information: fictive points the student received for their test in the other subject.
4. Teachers are asked to fill out a questionnaire focusing on background information, seating arrangements in the class, and tracking in school.
Experimental Design Details
Not available
Randomization Method
The experimental procedure employs randomization at different levels:
1. Based on the value of a random number, teachers will receive a random test package from the eight different test packages.
2. Within the assigned package, teachers will see the tests in a random order (based on the value of a random number).
3. Before deciding on the track recommendations, we communicate teachers the fictive points that students received for a test in another school subject (e.g., if the teacher evaluated the Hungarian test, we communicate the students’ fictive achievement on the math test, and vice-versa). This procedure is based on randomization in which we randomly choose one out of the six possible functions below with 1/6 probablity:
i. points given by the teacher – 6 point;
ii. points given by the teacher – 4.5 point;
iii. points given by the teacher – 3 point;
iv. points given by the teacher + 3 point;
v. points given by the teacher + 4.5 point;
vi. points given by the teacher + 6 point
Randomization Unit
The experimental procedure employs randomization at different levels:
1. Based on the value of a random number, teachers will receive a random test package from the eight different test packages.
2. Within the assigned package, teachers will see the tests in a random order (based on the value of a random number).
3. Before deciding on the track recommendations, we communicate teachers the fictive points that students received for a test in another school subject (e.g., if the teacher evaluated the Hungarian test, we communicate the students’ fictive achievement on the math test, and vice-versa). This procedure is based on randomization in which we randomly choose one out of the six possible functions below with 1/6 probablity:
i. points given by the teacher – 6 point;
ii. points given by the teacher – 4.5 point;
iii. points given by the teacher – 3 point;
iv. points given by the teacher + 3 point;
v. points given by the teacher + 4.5 point;
vi. points given by the teacher + 6 point
Was the treatment clustered?
No
Experiment Characteristics
Sample size: planned number of clusters
we do not have clusters
Sample size: planned number of observations
Having a single-level trial and expecting 400 teachers, each of them grading six tests, our planned number of observations will be 2,400 tests.
Sample size (or number of clusters) by treatment arms
Students' names will be randomized on the tests. The planned number of observations is 2,400 tests, half of them (1200) is expected to be assigned to Roma names, half of them (1200) is expected to be assigned to non-Roma names.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We calculate the power using the Optimal Design program (Spybrook, 2010). Having a single-level trial and expecting 400 teachers, each of them grading six tests, our analytical sample will contain 2,400 tests. Our most conservative p-value is 0.025. Using a one-sided t-test, the minimum detectable effect size (MDE) obtained by our design is 0.11. Since prior research found an effect size of 0.12 comparing Turkish and native German (Sprietsma, 2013; Wenz & Hoenig, 2020), our design is well-powered to detect a similar-sized treatment effect among Roma and non-Roma Hungarians.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
Research Ethics Committee, Centre for Social Sciences, ELKH
IRB Approval Date
2021-06-17
IRB Approval Number
N/A
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information