x

Please fill out this short user survey of only 3 questions in order to help us improve the site. We appreciate your feedback!
What Is It About Growth Mindset?
Last registered on September 11, 2020

Pre-Trial

Trial Information
General Information
Title
What Is It About Growth Mindset?
RCT ID
AEARCTR-0006436
Initial registration date
September 11, 2020
Last updated
September 11, 2020 11:59 AM EDT
Location(s)
Region
Primary Investigator
Affiliation
University of Zurich
Other Primary Investigator(s)
Additional Trial Information
Status
In development
Start date
2020-09-11
End date
2021-03-31
Secondary IDs
Abstract
Recent evidence documents that low-cost interventions aimed at giving adolescents a growth mindset have very sizable and positive effects on their test scores and probability of dropping out of school. However, such interventions are yet to be tested in poorer settings, where resources such as computers and internet are often not available. In partnership with the São Paulo State Secretariat of Education (SEDUC-SP), in Brazil, this project adapts and evaluates the previously evaluated Growth Mindset intervention from its computer-based original format to an SMS-based format, which could be scaled in both developed and developing countries. Above and beyond comparing the computer-based and SMS-based intervention, the project investigates the mechanisms behind the impacts of the growth mindset intervention. To unbundle its message, we randomize students within the SMS intervention into treatment arms that target specifically (i) beliefs about high returns to effort, or (ii) beliefs about low costs of effort, or (iii) salience of school activities, or (iv) risk-taking, or (v) future-orientation. 866,666 students from 6th to 12th grade will participate in the study. We evaluate impacts of the different treatment arms on participation in distance learning activities during the school shutdown caused by the COVID-19 pandemic, and on school attendance, grades and student dropouts when regular classes resume.
External Link(s)
Registration Citation
Citation
Lichand, Guilherme. 2020. "What Is It About Growth Mindset?." AEA RCT Registry. September 11. https://doi.org/10.1257/rct.6436-1.0.
Experimental Details
Interventions
Intervention(s)
We translated and adapted the computer-based growth mindset intervention from English to Portuguese, following the methodology in Bettinger et al. (2018), that translated this intervention from English to Norwegian. Due to limitations on school operations during the COVID-19 pandemic, we deliver the computer-based intervention remotely to each student by means of a text message link, instead of the original in-person delivery at the school. To access this content, students will need a smartphone, tablet or computer with internet access. We also adapted this content to be delivered exclusively through text messages, making it more accessible to vulnerable populations and scalable in both developed and developing countries. This adaptation process considered the methodology for SMS interventions used in Bettinger et al. (2020) that developed a parent engagement SMS intervention called EDUQ+, namely a platform powered by Movva, a Brazilian social impact startup, which allows schools to send messages to parents with information about children’s attendance and grades, and which nudges them with motivating facts and suggested activities to engage them in their children’s school life. The SMS-based version of EDUQ+ has been shown to be effective in Brazil, where communication with parents had large impacts on attendance, test scores and promotion rates (Bettinger et al., 2020). We will evaluate the effectiveness of this intervention in Brazil in its computer- and SMS-based versions separately on outcomes such as likelihood of dropping out, engagement with the remote schooling app provided by SEDUC-SP, and test scores.

Moreover, rather than just comparing average treatment effects of computer vs. SMS messages, this paper investigates why the growth mindset interventions works, unbundling mindsets into its underlying mechanisms. We do so by randomly assigning 800,000 Brazilian students from grades 6 to 12 to different versions of the mindset intervention, varying whether students are exposed to specific components of the mindset message: (i) whether or not messages make school activities salient (without conveying any message related to mindsets), whether or not messages emphasize (ii) high returns to effort or (iii) low costs of effort, and whether or not messages highlight (iv) the value of risk-taking (risk preferences) or (v) that of assigning higher weights to future outcomes (time preferences).
Intervention Start Date
2020-09-14
Intervention End Date
2020-12-31
Primary Outcomes
Primary Outcomes (end points)
We will assess how the different versions of the Mindset intervention impact learning efforts and outcomes for students enrolled in grades 6 to 12 (ranging 10-18 years old students). To do so, we will use administrative records to evaluate treatment effects on
- Daily access to the distance learning platform;
- Daily time online on the distance learning platform;
- Weekly attendance (by school subject) when the time regular classes resume;
- Quarterly grades (by school subject) when regular classes resume;
- Student dropouts.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
We will measure baseline levels of growth mindset, motivation to return to school when regular classes resume, students’ experiences with distance learning, and feelings of isolation during the pandemic (school-grade averages, as of early September, before the intervention starts). We will do so by assigning each student one of four questions randomly, i.e., 25% of students in a given grade-school pair will receive one text message with one of the following questions:

Q1 – On a scale from 1 to 6, how much do you disagree (0) or agree (6) with the following: Your intelligence is something about you that you can’t change very much.
Q2 – Do you plan on going back to school once in-person classes resume?
Q3 – Tell us about your experience of studying from home during this period without in-person classes.
Q4 – Share with us how you’ve been feeling during the past few months without seeing your teachers and classmates.

These questions will be rephrased accordingly when the message recipient is the student’s caregiver and not the student.
In addition, we also have information on various other baseline variables: the history of access to the distance learning platform since May; attendance and grades for the first quarter of the year, provided by administrative records; predicted risk of dropouts at the student-level, on a 0-100 scale.

We are interested in estimating heterogeneous treatment effects of the interventions according to those baseline measures.
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
We undertake 3 experiments to answer the research questions of interest.
Experiment 1 evaluates the impact of the computer-based intervention, adapted to the Brazilian context, to test hypothesis 1. In September, 400,000 students will receive a text message with a link to access the online content typically used in the computer-based intervention, made available through an online platform (Qualtrics). Upon accessing the link, students will be randomized on the spot to one out of 2 different groups, with equal probability:
1- A placebo group, which will receive an online intervention about the brain; or
2- A treatment group, which will receive an online growth mindset intervention.
The placebo intervention will instruct students in this group about the brain’s features and functionalities but will not provide any information about mindset. The treatment intervention will instruct students in the treatment group about growth mindset and how to develop it. Both treatments are based on text, illustrated with a few images (keeping fidelity to prior studies, but adapting to local context). Audio recording of the texts are also made available for students who prefer that option (in case they have enough airtime credit and bandwidth to access it).

Experiment 2 aims at testing hypotheses 2, 3 and 4. Also in September, 400,000 students (with no overlap with Experiment 1) will receive 1 SMS, assigned to 1 out of 6 different groups. There is also a control group, of identical size as each of the other cells, assigned to receive no SMS, as follows:
1- Typical growth mindset intervention group ("your brain is like a muscle, ...");
2- Salience group ("we want you to stay in school");
3- High returns to effort group ("if you put more effort in, you can always improve relative to yourself");
4- Low costs of effort group ("studying is actually more fun than you might think");
5- Risk-taking group ("trying is always worth it, failure does not mean anything about your potential");
6- Future-orientation group ("when thinking about your studies, it is important to think about your future!"); or
7- A pure control group, which does not receive any text message.

Experiment 3 aims at testing hypotheses 5 and 6. From October through December, the plan is to stick to the assignment for a sub-sample of students, potentially with more messages per student per month. We plan on updating the pre-analysis plan before the end of September to specify the final sample and number of messages. That decision depends on other tests being undertaken by SEDUC together with Movva to determine the optimal trade-off between number of students impacts by the nudges and number of nudges per student per month.
If SEDUC decides to stick to 1 message per month, the plan is to repeat the messages with slight variations in framing but keeping both the assignment and the core concept within each group. Alternatively, if SEDUC decides to go with more than 1 message per month (with a smaller sample size), we will randomly draw users within each treatment arm to receive additional messages. That experiment would allow us to estimate persistence and fade-out of treatment effects, as well as saturation effects (contrasting those assigned to receive additional messages to the ones who received messages only in September, and those to the pure control group).

Additionally, we intend to embed a 4th experiment into experiment 3 to evaluate hypothesis 8, or how the computer-based intervention format interacts with the text message format. To do so, we will select a random subsample of subjects in Experiment 3, to be determined and updated here after September, to receive a text message containing the link to the computer-based treatment.
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer
Randomization Unit
- Individual level for Experiments 1, 2 and 4.
- School-grade level for Experiment 3.
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
For experiment 3, ~ 3,500 schools x 7 grades = ~ 24,500 clusters
Sample size: planned number of observations
- 400,000 students for Experiment 1 - 466,667 students for Experiment 2 - TBD for Experiments 3 and 4
Sample size (or number of clusters) by treatment arms
TBD for Experiment 3
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For Experiment 1, we expect to send the SMS question to all students who take-up the computer-based intervention up to a 4.75% of the total sample. As a conservative estimate with a take-up rate of 2% and 2% of response rate to SMS questions, we estimate an MDEs of 44% within a single week, and as low as 11% when accumulated over 15 weeks. At a 4.75% take-up rate and 2% response rate for the SMS question, we estimate the MDEs for a single week to be 28%, and as low as 7.4% when accumulated. For Experiment 2, at a weekly response rate of 2%, we estimate an MDEs of 88% within a single week, and as low as 22% accumulated over 15 weeks. At a weekly response rate of 12%, we estimate an MDEs of 36% within a single week, and as low as 5.8% accumulated over 15 weeks.
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
University of Zurich
IRB Approval Date
2020-09-07
IRB Approval Number
OEC IRB # 2020-057
Analysis Plan
Analysis Plan Documents
Pre-Analysis Plan

MD5: 8ea08738e6b3e51eb206e6cb9f9e09ad

SHA1: f64e1ee7eb1a3eb97f6a43b3f01d27d01730816d

Uploaded At: September 11, 2020