The Promises and Perils of Diagnostic Labels

Last registered on February 01, 2021

View Trial History

Pre-Trial

Trial Information

General Information

Title

The Promises and Perils of Diagnostic Labels

RCT ID

AEARCTR-0006472

Initial registration date

September 19, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

September 21, 2020, 11:26 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

February 01, 2021, 10:40 PM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Russian Federation

Region

Primary Investigator

Name

James Chu

Affiliation

Stanford University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Elena Kardanova

PI Affiliation

PI Name

Elen Abdurakhmanova

PI Affiliation

PI Name

Prashant Loyalka

PI Affiliation

Stanford University

Contact Investigator

Additional Trial Information

Status

On going

Start date

2019-10-15

End date

2022-02-15

Keywords

Education

Additional Keywords

JEL code(s)

Secondary IDs

Abstract

Diagnostic labels are commonly used in educational and medical fields. Although research has shown that diagnostic labels can have adverse consequences on individual outcomes, there remain fundamental, unresolved questions in this literature: (1) To what extent does the introduction of diagnostic labels affect outcomes for a group on average? (2) To what extent do labels improve average group outcomes at the expense of certain subgroups of individuals? (3) How do the effects of diagnostic labels for groups and individuals depend on the proportion of other individuals who are also labelled in the group? To answer these questions, we propose a large-scale cluster randomized controlled trial among approximately 5,000 first-grade schoolchildren in 195 schools in Russia. Schools were randomly assigned to one of two treatment arms, where teachers received information about the performance of their students (a) without a corresponding diagnostic label for their students’ performance or (b) with a label for students as being “basic” / “proficient” in their academic performance or “developing” / “mature” in their behavioral performance. Results will inform the functions and tradeoffs of diagnostic labels for groups and individuals.

External Link(s)

Registration Citation

Citation

Abdurakhmanova, Elen et al. 2021. "The Promises and Perils of Diagnostic Labels." AEA RCT Registry. February 01. https://doi.org/10.1257/rct.6472-1.1

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

In the control group schools, psychometricians will provide teachers with numeric measures of student cognitive (academic) and non-cognitive (behavioral) performance (on a scale between 0 to 100) on the baseline assessments. This information is provided as part of an ongoing partnership with the regional governments to equip teachers with more information about their students.

In the treatment group schools, psychometricians will not only provide teachers with the numeric measures of student cognitive and non-cognitive performance but also diagnostic labels of student performance. Specifically, students across the sample who scored below 50 in their academic assessment are labeled as “basic” rather than “proficient,” and students in the sample who scored below 50 in their behavioral assessment are labeled as “developing” rather than “mature.” These assessments and labels are designed and pre-tested by Russian academics to be easily interpretable by teachers. For instance, teachers were invited to focus groups to offer feedback on different labels, and these labels were the most easily understood.

Intervention Start Date

2019-11-15

Intervention End Date

2020-09-28

Primary Outcomes

Primary Outcomes (end points)

Students will participate in math, reading, and behavioral assessments (created by a team of trained psychometricians). Performance on the standardized math and reading assessments will be the primary outcome variables in this study.

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

Students will be asked to report on their sense of belonging in the classroom to understand the potentially stigmatizing effects of diagnostic labels. Teachers will also be given a roster of their students and asked to describe, for each student, their perception of their academic and behavioral performance. Moreover, teachers will be asked to estimate the time and effort they directed toward each student. We collect teacher perceptions and self-reported effort toward individual students as important mediators to explore in the study.

Secondary Outcomes (explanation)

Experimental Design

We will conduct a cluster randomized controlled trial (RCT) involving 5,392 first grade schoolchildren (mean age of children = 7.4 years, proportion female = 0.50) across 195 schools in two regions of Russia. The vast majority of the 195 schools had one first grade class and teacher, but some schools had multiple first grade classes, each with its own teacher. Students and teachers from all first-grade classes were requested to participate in the study (for a total of 288 classes and teachers). All teachers participated in both a baseline and follow-up survey questionnaire. All students, in both baseline and follow-up, also participated in standardized cognitive and non-cognitive assessments produced by trained psychometricians.

Schools were randomized to one of two treatment arms in the RCT. Sample strata or blocks were created by placing the four schools with the closest mean math scores in a strata. Altogether, this resulted in 49 strata. Schools were then randomly allocated within these strata to one of two different treatment conditions.

Experimental Design Details

Randomization Method

Randomization done in office by a computer (Stata)

Randomization Unit

School

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

195 schools

Sample size: planned number of observations

5392

Sample size (or number of clusters) by treatment arms

97 schools in control group; 98 schools in treatment group

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Based on our blocked/stratified randomization design, we can estimate minimum detectable effect sizes (MDESs) of 0.08 SDs for math and 0.11 SDs for reading (alpha=0.05, power=0.8, ICC=0.000 for math and 0.064 for reading, R-squared=0.45).

Supporting Documents and Materials

IRB