Do Words Matter - Understanding the Effect of Exam Question Phrasing

Last registered on January 02, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Do Words Matter - Understanding the Effect of Exam Question Phrasing
RCT ID
AEARCTR-0012687
Initial registration date
December 13, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 02, 2024, 10:50 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Name
Nicolas Röver
Affiliation
University of Cologne
Contact Primary Investigator

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2023-12-13
End date
2024-08-01
Keywords
Gender
Additional Keywords
JEL code(s)
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
n Western countries, a gender gap in quantitative subjects in university and STEM-related1
jobs is well documented (e.g. Chen and Luo, 2022 and Ross et al., 2012). There is a multitude
of factors driving the gender gap; in this study we focus on the role of exams. Exam
performance is a major factor in university admission and job allocation. However, the notion
that exams are fair and truly test the abilities they are designed for is more and more called
into question (e.g. Duquennois 2022). There is a growing body of evidence suggesting that
the setting of exams and design of questions can systematically disadvantage socioeconomic
groups.2 Duquennois (2022) shows that students from poor families perform worse if a
question has a financial theme. Building on this result, our study aims to understand if
certain question phrasings systematically disadvantage one gender. By randomly varying
the exam question phrasing, we want to study this.
External Link(s)

Registration Citation

Citation
Röver, Nicolas . 2024. "Do Words Matter - Understanding the Effect of Exam Question Phrasing." AEA RCT Registry. January 02. https://doi.org/10.1257/rct.12687-1.0
Sponsors & Partners
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2023-12-13
Intervention End Date
2024-07-01

Primary Outcomes

Primary Outcomes (end points)
Exam Scores
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will randomly vary question phrasing in math related questions.
Experimental Design Details
Not available
Randomization Method
randomization done in office by a computer
Randomization Unit
individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
500
Sample size: planned number of observations
1500
Sample size (or number of clusters) by treatment arms
1
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan