Unveiling bias: the impact of blinded marking on university examen evaluation

First experiment: First midterm blind

The first experiment investigates whether instructors take into account the past academic history of students when grading exams. This study involves randomly assigning undergraduate classes from the second year or higher at Universidad de San Andres to either blind or non-blind grading. In the blind grading setup, instructors evaluate exams knowing only the student IDs, thus remaining unaware of any additional information about the students. Conversely, in the traditional grading setup, instructors are aware of the students' identities and can access further information, such as previous GPA. The eligible population includes all classes from the second year or higher. By comparing the grading of the first midterm exam outcomes of classes under both blind and traditional grading, we aim to determine the influence of prior student performance on current exam assessments. Consistent with existing literature, this experiment also seeks to establish whether there are disparities in grades between blind and non-blind graded classes based on observable student characteristics such as gender, age, financial aid status (specifically full scholarship recipients), and special educational needs (SEN).

Second experiment: Second midterm exam blind

The second experiment aims to explore the grading dynamics in first-year classes at Universidad de San Andres, focusing on the second midterms. This experiment is designed with the consideration that first-year students have minimal or no prior academic history, allowing for a clear analysis of the influence of performance solely observed from the first midterm. All first-year classes will implement non-blind grading for the first midterm, while for the second one, classes will be randomly assigned to either blind or non-blind grading. As in the first experiment, this analysis will also include an examination of differences in grades based on observable student characteristics such as gender, age, financial aid status (full scholarship), and special educational needs (SEN).

2024-09-30

2024-12-23

First experiment:

Grade; absolute difference between midterm grade and past GPA; share of changed grades (at class level).

Second experiment:

Grade; absolute difference between the second and first midterm exam; share of changed grades (at class level).

First Experiment:

1. Outcome 1: Grades
- Objective: Determine whether blind grading versus non-blind grading results in higher or lower average grades.

2. Outcome 2: Absolute Difference Between Midterm and GPA:
- Objective: Investigate whether the grading method (blind vs. non-blind) influences the discrepancy between midterm grades and students' historic GPA.

3. Share of Grade Changes:
- Objective: Examine if the grading approach (blind vs. non-blind) affects the proportion of midterm grades that are subsequently modified.

Second Experiment:

1. Outcome 1: Grade Analysis for First-Year Classes:
- Objective: Analyze if blind grading versus non-blind grading leads to higher or lower grades on average specifically for first-year classes.

2. Outcome 2: Absolute Difference Between Second Midterm and Midterm:
- Objective: Explore whether the grading method (blind vs. non-blind) impacts the discrepancies between grades of the second midterm and the first midterm.

3. Outcome 3: Share of Grade Changes:
- Objective: Determine if blind grading versus non-blind grading impacts the percentage of grade changes made after the second midterm in first-year classes.

First experiment:

For the first experiment, we will take all second-year classes and above at Universidad de San Andrés and conduct a random assignment without stratification so that half of the classes must grade the blind exam and the other half do not. For this intervention, we will have N=x classes to draw, so x/2 will belong to the treated group and x/2 to the control group.

Second experiment:

For the second experiment, we will take all first-year classes at Universidad de San Andrés and conduct a random assignment without stratification so that half of the classes must grade the blind exam and the other half do not. For this intervention, we will have N=x classes to draw, so x/2 will belong to the treated group and x/2 to the control group.

Both experiments: randomization by a computer.

We will randomize classes without stratification. The analysis will be done at the student level.

Yes

First experiment: We cluster at the class level (X classes).

Second experiment: We cluster at the class level (Y classes).

First experiment: X students Second experiment: Y students

First experiment: X classes, X/2 treated, X/2 controls.

Second experiment: Y classes, Y/2 treated, Y/2 controls.