x

We are happy to announce that all trial registrations will now be issued DOIs (digital object identifiers). For more information, see here.
Increase children’s interest in STEM – a field experiment in Austria
Last registered on November 12, 2019

Pre-Trial

Trial Information
General Information
Title
Increase children’s interest in STEM – a field experiment in Austria
RCT ID
AEARCTR-0005014
Initial registration date
November 09, 2019
Last updated
November 12, 2019 11:46 AM EST
Location(s)

This section is unavailable to the public. Use the button below to request access to this information.

Request Information
Primary Investigator
Affiliation
WU Vienna University of Economics and Business
Other Primary Investigator(s)
PI Affiliation
University of Vienna, Institute for Advanced Studies
PI Affiliation
Institute for Advanced Studies
Additional Trial Information
Status
On going
Start date
2019-11-08
End date
2020-12-31
Secondary IDs
Abstract
In this study, we implement a field experiment in which we test the effect of a digital platform on children’s interest in STEM-related fields. Stereotypical beliefs of STEM occupations may not correspond well with the reality of those jobs. Children may believe that men are more skilled in STEM compared to women. Besides, children may have wrong preconceptions such as STEM professionals work on their own rather than in teams and do not work on societal relevant topics. By counteracting false beliefs about STEM jobs, we want to enable boys and girls to develop an interest in STEM. To do so, we have developed a platform that aims at reducing stereotypical thinking about STEM occupations by presenting different STEM jobs and their applications and presents potential female and male role-models working in STEM. In addition, it incorporates mini games that are related to STEM fields focusing on active learning by using gamification tools such as instant feedback and goal rewards in the form of badges (“science badge,” “math badge”, …). Our intervention focuses on the children themselves (and not on parents or teachers) since children’s preferences may be more malleable at a young age than adults’ preferences. However, we test the impact of a parents’ brochure. In the brochure, we want to raise parents’ awareness about the effects of stereotypical thinking on children’s’ interests as well as on their math capacity. We give advice on how to increase children’s interest in math and STEM in general.
External Link(s)
Registration Citation
Citation
Grosch, Kerstin, Simone Haeckl and Martin Kocher. 2019. "Increase children’s interest in STEM – a field experiment in Austria ." AEA RCT Registry. November 12. https://doi.org/10.1257/rct.5014-1.0.
Experimental Details
Interventions
Intervention(s)
The intervention is a digital platform that addresses identified behavioral drivers that reduce the interest in STEM such as stereotypical beliefs, a fixed mindset, and a lack of confidence. The intervention aims at influencing these drivers positively to eventually increase interest in STEM. Additionally, the intervention is designed to make learning about STEM subjects intriguing and entertaining. We expect that by simply spending time learning about applications of STEM, children’s interest in STEM may increase.

Additional to the web platform, we test the effect of a STEM brochure on parents’ awareness on stereotypes and relevance of STEM, particularly on math, and, ultimately, on their children’s interest in STEM.
Intervention Start Date
2019-11-18
Intervention End Date
2019-12-20
Primary Outcomes
Primary Outcomes (end points)
Short term effects:
• Job_interest: a aggregated measure of children’s answers on how interested they are in a variety of STEM jobs
• Book_choice: a dummy for whether the children chose a book with a STEM content as present after the final survey
Long term effects:
• Track_choice: a dummy with value 1 if the children choose a school with a specialization in STEM after primary school
Outcome variable for 2nd treatment (brochures):
• Workshop_choice: a dummy for whether the parents chose a STEM summer workshop for their children
Primary Outcomes (explanation)
Job interest: use variable "job_*_interest" from kids questionnaire:
if job interest = ing/social/comp/lang/math/art : -2 = "nein gar nicht" to 2 ="ja sehr"
Generate aggregated measure of relative interest in MINT Jobs: sum job_ interest (ing+comp+math)/sum job_interest


Secondary Outcomes
Secondary Outcomes (end points)
• self-efficacy (primarily measured as “Self_efficacy_general”)
• math confidence (oc-Mathe)
• STEM confidence (jobability)
• Competitiveness (2 if comp_math + comp_german==1, 1 if only one of the variable ==1, 0 if both 0)
• stereotypical beliefs (1st measure explicit stereotypes (BELIEFS) – primary measure; 2nd measure less explicit: RANKING, secondary measure; 3rd measure explicit stereotypes (IAT) – least likely to be affected, but still observed)
Secondary Outcomes (explanation)
• Self_efficacy_general= mean(self-efficacy5-7): Index based on 3 questions from Bettinger et al. (2018).
Self_efficacy 5&6: -2= “nein stimmt nicht”- 2 = “ja, stimmt“
Self-efficacy 7: -2= “ja,stimmt”- 2 = “nein, stimmt nicht“
Likert scales are aggregated across questions and divided by the number of questions.

• Math confidence: guess_math-performance (split in two variables):
Overest_math=guess_math-math_performance if guess_math>=math_performance
underest_math=-(guess_math-math_performance) if guess_math<=math_performance

• Stem confidence: measured using variable job ability:
-2 = nein gar nicht / 2 =ja sehr

Generate aggregated measure of relative belief in ability in MINT Jobs: sum job_ ability(ing+comp+math)/sum job_ability*

• stereotypical beliefs:
Beliefs: 3 questions on stereotypical beliefs who is more skilled in math and 3 questions on who is more skilled in German
Q1,Q3,Q5: “Burschen mehr” -2 –“Mädchen mehr” 2
Q2, Q4, Q6: “Mädchen mehr” -2 –“Burschen mehr” 2
Likert scales are aggregated across questions and divided by the number of questions.

Ranking: : (number of boys chosen for math ranking – number of boys in top 3) + (number of girls chosen in German ranking- number of girls in top 3)

IAT: : (number of boys chosen for math ranking – number of boys in top 3) + (number of girls chosen in German ranking- number of girls in top 3)






Experimental Design
Experimental Design
Schools are randomly selected to get access to the web platform designed by us or to a control group, which gets access to a different learning platform without a focus on STEM. The intervention lasts for four weeks. In addition, 50% of the parents in the treatment and 50% of the parents in the control group receive information brochures explaining the importance of STEM.

We will collect children’s data twice, just before the intervention starts and about 3 weeks after the intervention. Additionally, we will collect data on parents’ and teachers’ preferences and attitudes, and on school characteristics.

The effect of the main intervention will be evaluated with survey instruments, laboratory experiments, and an IAT.
Experimental Design Details
Not available
Randomization Method
We recruit participants via schools. The ministry of education, as well as the educational administrations for Upper Austria and Vienna, support the study and encourage schools to participate in the study by providing a recommendation letter.
We aim to recruit 20 schools in Vienna and 20 schools in Upper Austria with a mode of 2 classes per school in Vienna and one class per school in Upper Austria since Upper Austria is more rural and schools much smaller than in Vienna. For organizational reasons, the randomization strategy differed between Vienna and Upper Austria. In Upper Austria, we randomly picked 8 schools in the region Mühlviertel and 2 schools in Linz. After that we recruited the closest neighboring schools. This way, we have two schools in a very similar neighborhood in terms of families’ sociodemographic characteristics. When a school is not willing to participate in our study, we randomly pick another school from the same area to replace the drop-out. One school of each of these pairs will be randomly selected as a treatment school and the other one will automatically become a control school. We use Windows Excel to do this and generate a random number 0 or 1 for each school whereas 1 means treatment group and 0 means control group. In Vienna, for logistical reasons and higher homogeneity of participants, we only include schools in districts 2 to 9 which are the central districts in the capital. We randomly chose 10 schools and stratified by share of schools in each district. For instance, if 50% of the schools were in district 5, we randomly chose 5 schools in this district. In a second step, we recruited the closest neighboring school for each of the 10 schools. After contacting the schools, we learned, that eleven schools in Vienna already use the control web-platform. We do not assign the control treatment to schools with a large share of teachers using this control platform but assign them to the treatment group. Consequently, neighboring schools of treatment schools are assigned to the control group. To test if previous usage of the control web-platform moderates treatment effects, we will compare treatment effects between these two groups using the same specifications as in the main analysis discussed below.
The parents’ information brochure is handed out at 50 percent of the schools (half of the control group and half of the treatment group) at the last day of the main intervention.
Randomization Unit
schools
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
The expected sample size is about 40 schools (20 schools in Vienna and 20 schools in Upper Austria).
Sample size: planned number of observations
In each school in Vienna, we expect to find 2 classes on average with about 22 children each. In the more rural area in Upper Austria, we expect to find 1 class on average. In total, this results in a sample size of 20 schools x 2 classes x 22 = 880 and 20 x 1class x 18 =360 which sums up to about 1240 children.
Sample size (or number of clusters) by treatment arms
20 schools treatment and 20 schools control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
IHS Kommission zur Behandlung von Fragen der Ethik und wissenschaftlichen Integrität
IRB Approval Date
2019-10-18
IRB Approval Number
CASE_02_MINT_2020000_IA
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information