WILL-TO - A matched savings program to increase households' investments in children's education

The program (WILL-TORINO) is implemented by Fondazione Ufficio Pio, a philanthropic organization in Turin, Northwestern Italy since 2020.

The program targets 5th graders from low-income families and offers them and their families the opportunity to save small amounts of money (between 5 and 30 euros per month up to a maximum of 1,500 euros) over a 6-year period in a dedicated digital wallet, linked to a bank account opened in the name of the child. Household deposits are multiplied by four if the money was spent on proven educational expenses (i.e., computers/internet; culture, book purchases; various school expenses, language or computer courses, sports, transportation). In addition to the savings account, beneficiaries are offered a financial education program, educational support and guidance.

The program is opt-in. Every year, an information campaign is launched and interested households residing in Turin and with an income below a given threshold (below €15.000 of family equivalent annual income (ISEE index, Indicatore di Situazione Economica Equivalente) can apply.

2021-12-15

2030-02-12

The primary outcome is regular course of study in upper secondary education, which is measured at two different points in time:
A) Upper secondary school attendance, 4 years after randomization
B) Upper secondary school completion, 9 years after randomization

A) Upper secondary school attendance, 4 years after randomization

Modalities
1 Yes, regularly enrolled in the 10th grade
2 Yes, enrolled in a lower grade
3 No, dropped out

In the analysis, modalities 2 and 3 could be merged into one single category.

Because the study involves three different cohorts of program applicants, which were randomized in November 2021, 2022, and 2023, the data collection is scheduled for school years 2025/2026, 2026/2027, 2027/2028.


B) Upper secondary school completion, 9 years after randomization

Modalities:
1 Yes, graduated on time
2 No, still enrolled in upper secondary education
3 No, dropped out

In the analysis, modalities 2 and 3 could be merged into one single category.

Because the study involves three different cohorts of program applicants, which were randomized in November 2021, 2022, and 2023, the data collection is scheduled for 2029/2030, 2030/2031, 2031/2032.


Source: Administrative data, or survey administered to students by an external agency – if administrative data are not accessible.

Several secondary outcomes will be collected at different points in time.

Two and four years after randomization, the following outcomes will be observed:
- Household’s savings;
- Household’s school and extra-scholastic;
- Educational aspirations and expectations;
- Parental involvement;
- School attendance;
- Grade retention;
- School grades in Italian language and mathematics;
- School tests (if available through administrative archives);
- Extracurricular activities.

The first post-intervention data collection (2 years after randomization) is planned to start on March 12, 2024.

Four years after randomization, the following outcomes will be observed:
- Lower Secondary school grade in Italian language and mathematics;
- Upper secondary school type choice.

Nine years after randomization, the following outcomes will be observed:
- Type of Upper secondary school diploma achieved;
- Upper secondary school’s Final exam Grade.

Two and four years after randomization:

- Household’s savings (whether the household saves, whether it plans to save in the next two years, and for what reasons);
- Household’s school and extra-scholastic expenses (whether the household spends for child’s extra-scholastic activities, whether the household think they will be able to sustain these expenditures in the next year);
- Educational aspirations and expectations (parents’ and children’s aspired and expected highest educational title achieved by the child);
- Parental involvement (whether parents take part in school-related activities, such as talking to teachers);
- School attendance (whether the child skips days or classes);
- Grade retention (whether the child has retained a school year);
- School grades in Italian language and mathematics;
- School tests (if available through administrative archives);
- Extracurricular activities (the frequency at which the child participates in extra-scholastic activities).

The first post-intervention data collection (2 years after randomization) is planned to start on March 12, 2024.

Four years after randomization:
- Lower Secondary school grade in Italian language and mathematics (min 6, max 10);
- Upper secondary school type choice (Vocational, Technical, General).

Nine years after randomization:
- Type of Upper secondary school diploma achieved (Vocational, Technical, General);
- Upper secondary school’s Final exam Grade (min 60, max 100).

School grades and type of school attended and completed will be retrieved via administrative archives. Because access to administrative data is not granted, we plan to administer (via an external agency) surveys to parents and children (2 years after randomization) and to children (4 years after randomization).

Applications are collected yearly in November. The study focuses on the 2021, 2022 and 2023 cohorts, cohorts in which applicants were randomly assigned to the treatment or the control condition.

The number of valid and eligible applications is 1,141. 502 are assigned to the treatment condition and 639 to the control condition.
The distribution of participants in the two groups across the three cohorts is as follows.
2021: 79 (Treatment), 98 (Control)
2022: 274 (Treatment), 200 (Control)
2023: 286 (Treatment), 204 (Control)

The overall treatment allocation ratio is .44, although it varies across experimental cohorts, being .55 in the first cohort, and .42 in cohorts 2 and 3.

Randomization was performed within strata, which were identified by a dummy variable identifying the residential neighborhood (only in the first experimental cohort), parents’ possession of a high school diploma or above (dummy variable), parent who applied born abroad (dummy variable).

Due to the implementation over three school years and the possibility for families to submit multiple applications for different children, there are 51 households with more than one child in the study and 29 of them with children belonging to different experimental groups.

The first post-intervention data collection (2 years after randomization) is planned to start on March 12, 2024 for the first cohort.

Not available

Randomization is done using the statistical package Stata. To improve statistical precision, a stratified randomization design is implemented. Randomization was performed within strata, which were identified by a dummy variable identifying the residential neighborhood (only in the first experimental cohort), parents’ possession of a high school diploma or above (dummy variable), applicant parent born abroad (dummy variable).

The unit of randomization is the individual, although in the second and third cohort twins were assigned to the same experimental group.

No

1141

1141

Treatment: 502; Control: 639

Power analysis is calculated using “3ie Sample size and minimum detectable effect calculator” (Djimeu & Houndolo 2016) and PowerUp! (Dong and Maynard 2013). The assumptions made to calculate the minimum detectable effects of the experiment are: a) statistical significance level (p-value) = .05; b) statistical power: 80%; c) proportion of randomization units assigned to treatment: 44%. Because we don’t have a reliable measure of the variance of primary outcome variable for a comparable population, we prefer to opt for a conservative scenario, i.e. a probability of .80, which is the observed percentage of upper secondary school graduates out of lower secondary school graduates in the region Piemonte. The resulting MDE for the primary outcomes is .059 percentage points. The above estimates assume zero attrition. Attrition would increase the MDE estimates. Now, it is not possible to predict how large it will be: if we assume a response rate as low as 75% the resulting MDE is .069. Also, the above estimates do not account for the fact that the statistical analysis of the treatment effects will be based on regression-adjusted estimation of the impact including randomization strata and covariates. Beyond estimating the treatment effects on primary and secondary outcomes, the analysis will also explore heterogeneity of the effects by key factors such as parental education, family income and migrant background. References: Djimeu, E. W., & Houndolo, D. G. (2016). Power calculation for causal inference in social science: sample size and minimum detectable effect determination. Journal of Development Effectiveness, 8(4), 508-527. Dong, N. and Maynard, R. A. (2013). PowerUp!: A tool for calculating minimum detectable effect sizes and sample size requirements for experimental and quasi-experimental designs. Journal of Research on Educational Effectiveness, 6(1), 24-67. doi: 10.1080/19345747.2012.673143