The effect of teacher professional development on financial literacy education. Evidence from a randomized experiment

Last registered on July 10, 2019

Pre-Trial

Trial Information

General Information

Title
The effect of teacher professional development on financial literacy education. Evidence from a randomized experiment
RCT ID
AEARCTR-0003501
Initial registration date
October 29, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 29, 2018, 5:05 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
July 10, 2019, 5:36 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Universiteit Antwerpen

Other Primary Investigator(s)

PI Affiliation
KU Leuven
PI Affiliation
University of Antwerp
PI Affiliation
KU Leuven

Additional Trial Information

Status
Completed
Start date
2017-09-01
End date
2019-01-31
Secondary IDs
Abstract
We study the impact of professional development of teachers in secondary education in the Flemish region of Belgium. We investigate whether an online training module affects financial knowledge and behavior of both teachers and students. To identify a causal effect, we make use of a randomized controlled trial in which 48 schools and approximately 2000 students participate. Schools are randomly assigned to one of the treatment conditions. We compare financial literacy test scores in schools in which teachers follow a training module to test scores in schools in which teachers do not follow this module.
External Link(s)

Registration Citation

Citation
Compen, Boukje et al. 2019. "The effect of teacher professional development on financial literacy education. Evidence from a randomized experiment." AEA RCT Registry. July 10. https://doi.org/10.1257/rct.3501-4.0
Former Citation
Compen, Boukje et al. 2019. "The effect of teacher professional development on financial literacy education. Evidence from a randomized experiment." AEA RCT Registry. July 10. https://www.socialscienceregistry.org/trials/3501/history/49643
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Schools are assigned to the following four experimental conditions:
1) Control group: Students do not follow a course on financial education.
2) Treatment group 1: Students follow a course on financial education. Teachers do not follow an online training module.
3) Treatment group 2a: Students follow a course on financial education. Teachers can follow an online training module.
4) Treatment group 2b: Students follow a course on financial education. Teachers can follow an online training module and are activated to participate in this training by a facilitator.
Intervention Start Date
2018-01-12
Intervention End Date
2018-05-31

Primary Outcomes

Primary Outcomes (end points)
We measure financial literacy of students by a test that contains 9 questions. This test consists of questions that measures both financial knowledge and financial behavior. We measure the financial literacy of teachers by a separate questionnaire. We are able to measure the intensity of treatment because we observe the activity of the teachers in the online training module (for example the number of clicks on each item).
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Schools were randomized to the aforementioned four experimental conditions. We measured the level of financial literacy before and after students followed the course. Students who were assigned to the control group completed the same tests at the same time as students in the several treatment groups. The experiment was conducted in two waves. In the first wave, students in the second year of secondary education (age 13-14) participated in the experiment. All teachers who could follow the online training module were assigned to treatment group 2a. In the second wave, students in the third year of secondary education (age 14-15) followed exactly the same course and completed the same questionnaires. All teachers who could follow the online training module in this wave were assigned to treatment group 2b.
Experimental Design Details
Before the intervention, all students (control group + treatment groups) filled in an online questionnaire to test their financial literacy. Students also reported some personal characteristics like gender, study results, and socio-economic background. Teachers completed a separate questionnaire. Once students and teachers filled in the questionnaire, teachers in the treatment groups received the study material, including instructions on how to implement the course.

Approximately one month after filling in this test, students in the treatment groups followed a 4 hour course on financial literacy that focused on means of payment. The course material was designed as an escape room and students had to solve questions on the topic in pairs. Students could solve the escape room only if they found the correct answer to all questions. Students could find information to solve the questions in a booklet that was designed for the course. We incentivized students to perform well during the course as there was a small present for the winning team. The study material was developed by high school teachers according to the age and the ability of the students.

In treatment group 2a, teachers could follow an online training module. Here, teachers could find more information about the contents of the course and how to take heterogeneity between students into account when teaching financial education. Teachers could also share their experiences with other teachers through online discussion boards. In treatment group 2b, teachers could follow a similar online training module, but teachers were more intensively activated to participate in the treatment by a facilitator. This facilitator regularly contacted teachers by e-mail to encourage them to read the information on the platform and to be active in discussion boards. In this way, treatment was more intense in this treatment condition.

At the end of this course, students and teachers filled in a test to measure financial knowledge and financial behavior. At the same time, students and teachers in the control group also filled in this test without having followed the course. Both tests contained similar questions as the tests students and teachers had to take before the intervention.
Randomization Method
Schools were randomly assigned to the different experimental conditions by a random number generator in Stata.
Randomization Unit
In both waves, we randomized at the school level. All students and teachers in the same school were assigned to the same condition. In this way, all teachers in the same school received the same teaching material and instructions. This minimizes the possibility of spill-over effects and contamination of the different treatment conditions.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
Because we already executed the experiment, and collected the data for both the pretest and the posttest, we report the actual number of students and schools for which we have observations for both the pretest and the posttest. This will be the final sample for our analysis. We also use this sample size for the calculation of the minimal detectable effect size.
Wave 1: 35
Wave 2: 13
Sample size: planned number of observations
Wave 1: 1628 Wave 2: 321
Sample size (or number of clusters) by treatment arms
Wave 1: second year of high school
Control group = 739 students, 13 schools
Treatment group 1 = 408 students, 12 schools
Treatment group 2a = 481 students, 10 schools

Wave 2: third year of high school
Control group = 111 students, 5 schools
Treatment group 1 = 91 students, 4 schools
Treatment group 2b = 119 students, 4 schools

Total sample
Control group = 850 students, 18 schools
Treatment group 1 = 499 students, 16 schools
Treatment group 2 = 600 students, 14 schools

Average number of schools per condition = 16
Average number of students per condition = 649.7
Average number of students per school = 40.6

Please note that as teacher characteristics were included as control variables in the final analyses, this resulted in missing values for a selection of students. Consequently, the final sample reported in the article is somewhat smaller:

Wave 1: second year of high school
Control group = 675 students, 13 schools
Treatment group 1 = 401 students, 11 schools
Treatment group 2a = 481 students, 10 schools

Wave 2: third year of high school
Control group = 111 students, 5 schools
Treatment group 1 = 91 students, 4 schools
Treatment group 2b = 86 students, 3 schools

Total sample
Control group = 786 students, 18 schools
Treatment group 1 = 492 students, 15 schools
Treatment group 2 = 567 students, 13 schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We base our computation on List et al. (2011) and account for intracluster correlation in the calculation of the minimal detectable effect size. Please note that this computation was based on the sample sizes before the inclusion of teacher characteristics resulting in missing values. In our experimental setting, there on average 16 schools in each experimental condition. Each school on average contains 40.6 students. We computed an intracluster correlation of 0.12. In the analysis, we can control for characteristics of schools and students, which would decrease the intracluster correlation. With the conventional power of 0.8 and a significance level of 0.05, we are able to detect a treatment effect of 0.37 standard deviations or larger. Details of calculation: To calculate the minimum detectable effect size, we follow List et al. (2011). They show that in a clustered design, the minimum number of observations in each experimental group can be computed as follows: n=2(t_(α/2)+t_β)²(σ/δ)²(1+(m-1)ρ) This implies that the minimum detectable effect size is equal to: δ=σ/√(n/(2(t_(α/2)+t_β)²(1+(m-1)ρ))) Or the minimum detectable effect size expressed as a fraction of a standard deviation is equal to: δ/σ=1/√(n/(2(t_(α/2)+t_β)²(1+(m-1)ρ))) δ/σ=1/√(649.7/(2(1.96+0.84)²(1+(40.6-1)0.12)))=0.37 Reference List, J., Sadoff, S. and Wagner, M. (2011), So you want to run an experiment, now what? Some simple rules of thumb for optimal experimental design, Experimental Economics 14, 439-457
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
January 31, 2019, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
January 31, 2019, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
46 schools
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
1845 students
Final Sample Size (or Number of Clusters) by Treatment Arms
Control group: 18 schools Treatment 1 group: 15 schools Treatment 2 group (combined): 13 schools
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials