Examining Bias in Teacher Evaluations of Students

Last registered on December 04, 2023
View Trial History

Pre-Trial

Trial Information

General Information

Title
Examining Bias in Teacher Evaluations of Students
RCT ID
AEARCTR-0012510
Initial registration date
November 15, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 29, 2023, 9:59 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
December 04, 2023, 9:00 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Rune Borgan Reiling
Affiliation
Nordic Institute for Studies in Innovation, Research and Education (NIFU)
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Ragnar Alne
PI Affiliation
Nordic Institute for Studies in Innovation, Research and Education (NIFU)
Contact Investigator
PI Name
Eyo I. Herstad
PI Affiliation

Additional Trial Information

Status
On going
Start date
2023-11-15
End date
2025-08-31
Keywords
Behavior, Education, Gender
Additional Keywords
ethnicity, bias, discrimination
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We present causal evidence regarding the impact of a student's ethnicity and gender on teacher evaluations in middle school. In this study, we engage teachers to assess written assignments for full classes in two subjects: mathematics and Norwegian language. By randomly assigning names that signal both gender and ethnicity to the assignments within teachers, we investigate whether students with equal ability but different names receive equal grades, on average. While our primary focus is on the average effects of student gender and ethnicity, we also conduct an extensive analysis of heterogeneity, considering factors such as ability, student gender, teacher experience, and the gender of the teacher. This document outlines our analysis plan, including our primary specifications of interest.
External Link(s)

Registration Citation

Citation
Alne , Ragnar, Rune Borgan Reiling and Eyo I. Herstad . 2023. "Examining Bias in Teacher Evaluations of Students." AEA RCT Registry. December 04. https://doi.org/10.1257/rct.12510-2.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Student grades in schools are supposed to be a reflection of the students’ real ability in the subject, and commonly decide which students go to which high school, higher education and job. As such, from a fairness perspective, it is important that student grades reflect the the student’s ability in the subject. To identify whether students are treated fairly, we recruit real teachers to evaluate real student assignments.
Intervention Start Date
2023-11-23
Intervention End Date
2024-08-31

Primary Outcomes

Primary Outcomes (end points)
Our primary outcome is the assessment of students’ ability levels by teachers.
Primary Outcomes (explanation)
We collect this data through our questionnaire, in which teachers evaluate written assignments for an entire class using the standard Norwegian grading scale. The Norwegian grading scale spans from 1 to 6, with 6 representing the highest grade and 1 signifying a failed assignment. We collect student grades for both a Mathematics and a Norwegian language assignment.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The interventions will be randomized across teachers.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer.
Randomization Unit
30 student assignments
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Our goal is to recruit approximately 150 teachers for the project, with an anticipated participation
of over 100 teachers who will each grade approximately 30 assignments. This amounts to a
total of roughly 3,000 graded assignments clustered at the teacher level.
Sample size: planned number of observations
Our goal is to recruit approximately 150 teachers for the project, with an anticipated participation of over 100 teachers who will each grade approximately 30 assignments. This amounts to a total of roughly 3,000 graded assignments.
Sample size (or number of clusters) by treatment arms
Each teacher will grade an entire class of student assignments. This amounts to a sample size equal to the number of teachers per name that is randomly assigned. Our goal is to recruit approximately 150 teachers for the project, with an anticipated participation of over 100 teachers who will each grade approximately 30 assignments. This amounts to a total of roughly 3,000 graded assignments.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
With 3,000 observations, we have a statistical power of 0.8 to detect an effect size of .004 (Cohen’s f2) at a significance level (α) of 0.05.
Supporting Documents and Materials
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information