Inclusive Classrooms and Equitable Student Success: A Faculty Experiment

Last registered on November 27, 2021

View Trial History

Pre-Trial

Trial Information

General Information

Title

Inclusive Classrooms and Equitable Student Success: A Faculty Experiment

RCT ID

AEARCTR-0005505

Initial registration date

February 28, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

March 02, 2020, 4:13 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

November 27, 2021, 9:39 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

United States of America

Region

Virginia

Primary Investigator

Name

Michela Carlana

Affiliation

Harvard Kennedy School

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Lena Shi

PI Affiliation

Harvard University

Contact Investigator

PI Name

David Deming

PI Affiliation

Harvard University

Contact Investigator

Additional Trial Information

Status

In development

Start date

2020-10-07

End date

2021-12-23

Keywords

Education, Labor

Additional Keywords

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

Researchers have documented racial and gender gaps in college enrollment decisions, choice of major, degree attainment, and earnings—despite narrowing gaps in test scores and course-taking in K-12 settings. Implicit racial and gender stereotypes of faculty members may affect their interactions with students and exacerbate these gaps, even without awareness or intent to harm members of underrepresented groups. Yet, there is no causal evidence on the extent to which faculty’s implicit bias contributes to these educational disparities and which types of interventions are cost-effective in mitigating any harmful effects of implicit bias on student achievement gaps.This study aims to address implicit bias of faculty members through the collaboration between psychologists and economists. First, we plan to understand the relationship between faculty’s implicit bias and gaps in student achievement, completion, and economic mobility using a newly constructed dataset with schools’ student-level and faculty-level administrative data, and faculty’s implicit association test (IAT) results. Second, we plan to implement a randomized field experiment to evaluate the effects of faculty implicit bias trainings on students' academic performance. Due to schools’ adjustments to online education in March 2020, we will pilot the study using an online format in Fall 2020 at Reynolds Community College and Fall 2021 at other three community colleges. The second pilot will include videos instead of only in-person interaction with psychologists for the implementation of the implicit bias training.

External Link(s)

Registration Citation

Citation

Carlana, Michela, David Deming and Lena Shi. 2021. "Inclusive Classrooms and Equitable Student Success: A Faculty Experiment." AEA RCT Registry. November 27. https://doi.org/10.1257/rct.5505-3.0

Sponsors & Partners

Interventions

Intervention(s)

Intervention Start Date

2020-10-23

Intervention End Date

2021-12-23

Primary Outcomes

Primary Outcomes (end points)

We intend to evaluate the impact of our intervention on measures of students' academic performance, attainment, attitudes, and mobility.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Secondary Outcomes (explanation)

Experimental Design

The study intends to evaluate how providing implicit bias training to higher education instructors impacts students' outcomes. We will offer an implicit bias training to a randomly selected sample of instructors teaching courses at the Fall 2020 term in Reynolds Community College and Fall 2021 term at other three community colleges in Virginia. The training is designed to expose faculty members to their own implicit biases and provide them with tools to adjust their automatic pattern of thinking with the ultimate goal of mitigating any biased behavior. This treatment--- implemented by psychologists---will be based on scientific evidence and previous research results and it will adopt a non-judgmental approach that focuses on the recipients’ self-interest and organizational interest. The pilot in the Fall 2020 includes only syncronous interaction between psychologists and faculty members. The pilot in the Fall 2021 will test the implementation of professional videos for the implicit bias training together with group discussion mediated by a psychologist. Follow up emails will be sent at most bi-weekly to remind instructors of training content to raise awareness about potentially biased behavior. We will then evaluate the impact of interacting with instructors exposed to training on students' outcomes. Due to schools’ adjustments to online education in March 2020, we will pilot the study using an online format in Fall 2020 at Reynolds Community College, randomizing across instructors from all departments.

Experimental Design Details

Randomization Method

The randomization will be done in office by a computer.

Randomization Unit

We will randomize at the individual---instructor---level.

Was the treatment clustered?

Experiment Characteristics

Sample size: planned number of clusters

For the online pilot at Reynolds community college in Fall 2020, we will stratify based on faculty’s baseline survey completion such that half of the survey completers are assigned to treatment and half are assigned to the control group.

For the online pilot at the three community college in Fall 2021, we will stratify based on faculty’s baseline survey completion such that half of the survey completers are assigned to treatment and half are assigned to the control group.

Sample size: planned number of observations

For the online pilot at Reynolds community college in Fall 2020, we will include 328 instructors across all departments in our randomization. We can observe for these instructors across 1,000 classes and we also have information on over 6,000 students. For the online pilot at the three community college in Fall 2021, we will include 1078 instructors across all departments in our randomization.

Sample size (or number of clusters) by treatment arms

For each experiment, we include 50% of faculty members as treated and 50% as control.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Using administrative information from previous terms, we were able to simulate the Minimum Detectable Effect considering different measures of students’ performance. Our power calculations suggest that we will be able to detect an impact 3 percent of a standard deviation for outcomes measured at the student-class level (e.g., grade) and from 13 to 18 percent of a standard deviation for outcomes measured at the instructor-class level (e.g., black-white grade gap, hispanic-white grade gap). To compute MDEs, we assumed a significance level of 5 percent and an 80 percent power for the overall treatment.

Supporting Documents and Materials

IRB