Reducing the digital divide for marginalized households

The intervention focuses on low-income families with school-age children, aiming to enhance their digital access and skills through a program that provides tablets, a year of free internet, and digital training for one adult per family. In collaboration with an Italian ONG, we implemented a RCT allocating over 850 applicants into a control and two treatment groups.

All treated individuals receive tablets and internet access for a year. The two treatment groups differ in duration and content of the training. In fact, all courses cover basic digital skills, and the second treatment includes additional specialized topics. Applicants filled the baseline survey between September and November 2023, the application outcomes were disclosed in November 2023, followed by information sessions in December for the treated. Courses began in January 2024, running once a week for two hours, concluding in April for one treatment harm and May for the other. Throughout the program, participation is monitored, culminating in a final test during the last class.

A follow-up in October 2024, approximately one year after the baseline survey, will revisit applicants with the same questions from the baseline, along with inquiries about changes in digital competencies, work attitude, and digital activities conducted with their children.

The “DigitAll” program, implemented by the Ufficio Pio foundation (Compagnia di San Paolo), focuses on low-income families with school-age children in Turin, Italy. It provides a tablet, one year of free internet, and digital skills training for one adult per family. In collaboration with the foundation, we conducted the third edition of the program as a randomized controlled trial.

Applications were accepted during a two-month period from September 5 to November 5, 2023. Applicants responded to baseline questions covering digital literacy, social inclusion, work status and preferences, and engagement with their children's education. They also specified up to two preferred locations and times.

We received a total of 859 valid applications and allocated participants into groups: 337 in the control group (C), 261 in the first treatment group (T1), and 261 in the second treatment group (T2). Allocation considered treatment type and logistical preferences. When unable to fulfill stated preferences, participants were assigned to the nearest center with availability.

All treated applicants receive a tablet with a year of internet access. T1 and T2 differ in training duration and content. T1 involves an 18-hour basic digital skills course, covering for instance web browsing, Google tools, and internet safety. T2 offers a 30-hour course, including T1 content and additional topics like Google sheets, e-commerce, guiding children in social network use, and utilizing the internet for job searches.

Application outcomes were disclosed in November 2023, with treated individuals invited to December information sessions. Courses commenced in January 2024, occurring once per week for two hours. T1 courses conclude in April, and T2 courses finish in May. Participation is monitored, and a final test is administered during the last class.

We will recontact all applicants in October 2024, approximately one year after the baseline survey, and ask the same questions as before, along with additional inquiries about their digital competencies, work attitude, and digital activities they have done with their children.

2024-01-22

2024-05-31

Our primary outcomes are the variables targeted by the training. These outcomes are evaluated in October/November 2024, roughly 6 months after the training, in comparison to baseline measurements collected in October/November 2023, 12 months prior. We categorize our primary outcomes as "short-term." They are divided into five sets of categories:

1) Adoption: This category focuses on assessing whether participants are adopting the provided device and internet connection.
2) Digital literacy: This category aims to evaluate the effectiveness of the training program in enhancing participants' digital literacy skills. We use an index derived from Eurostat, comprising five sub-indices.
3) Social inclusion: This category focuses on assessing whether participants have taken steps towards improving their socio-economic status, particularly whether participants searched for or requested income measures against poverty.
4) Human capital transmission: This is assessed through a series of questions, partially derived from PISA, focusing on parents' engagement in activities related to their children's education.
5) Labor market: This category assesses participants' readiness for and engagement with the labor market, focusing on their attitudes, motivations, and efforts related to employment.

1) Adoption:

This category delves into evaluating the extent to which participants embrace and integrate the provided device and internet connection into their daily lives. It encompasses not only the initial acceptance and utilization of the equipment and connectivity but also ongoing usage patterns and integration into various aspects of their digital activities. This assessment examines factors such as frequency and duration of device usage, the range of tasks performed using the device, proficiency in navigating digital interfaces, and the extent to which participants leverage internet access for educational, professional, or personal purposes. Additionally, it considers any barriers or challenges encountered in adopting the device and internet connection, as well as strategies employed to overcome them. Overall, the adoption variable provides insight into the practical implementation and integration of the provided digital resources within participants' households and routines.

2) Digital literacy:

We have an aggregate index based on sub-indices, and we will use both the aggregate index and the sub-indices to clarify the most relevant dimensions. These indices measure different dimensions of digital literacy, such as proficiency in using digital tools, understanding of digital concepts, ability to navigate online platforms, and awareness of digital safety and security practices. By utilizing these indices, the training program can quantitatively assess the extent to which participants' digital literacy skills have improved as a result of the training.

3) Social inclusion:

This category focuses on assessing whether participants have taken steps towards improving their socio-economic status, particularly in relation to poverty alleviation measures. The variables are constructed based on whether participants actively searched for or requested income measures aimed at addressing poverty. This assessment provides insights into participants' engagement with social support systems and their efforts to access resources and opportunities to improve their financial well-being.

4) Human capital transmission:

This category evaluates participants' engagement in activities related to their children's education, specifically focusing on their ability to effectively contribute to their children's human capital accumulation. For instance, we will measure the frequency of tasks such as assisting with homework and maintaining communication with the school.

5) Labor market:

The variables are constructed based on participants' beliefs, willingness to work, and job search effort. Beliefs may include attitudes towards work, career aspirations, and perceptions of job opportunities. Willingness to work reflects participants' eagerness and commitment to securing employment and advancing their careers. Job search effort encompasses the actions taken by participants to actively seek and apply for job opportunities, such as networking, submitting applications, attending interviews, and participating in job training programs.

Our secondary outcomes are the variables targeted by the training beyond the 10-month period. We categorize them as "long-term." They are "secondary" for us because it is less likely that we have the power to observe the program's effects on them directly and because we need to request access to Italian administrative data to study them. These outcomes are divided into three sets of categories:

1) Social inclusion: The variables are constructed based on whether participants obtained and benefited from income measures targeted at reducing poverty.
2) Human capital transmission: This category focuses on evaluating the long-term effect of the intervention on the educational outcomes of the participants' children.
3) Labor market: This category evaluates the long-term impact of the training program on participants' labor market outcomes, including their employment status, wages, and engagement with employment services.

1) Social inclusion:

This assessment might involve examining participants' access to and utilization of various social welfare programs, subsidies, or financial assistance aimed at improving their socio-economic well-being over an extended period beyond the initial 10-month training period.

2) Human capital transmission:

The variables included in this category aim to measure children's educational achievement and attainment. In particular, we plan to explore their school participation (number of absences) and performance (grades, graduation, results in standardized tests...) over time. This examination aims to discern whether the participation of their parents in the program has influenced the development of their human capital and educational outcomes.

3) Labor market:

The variables included in this category are participants' employment status (e.g., employed, unemployed), wages earned, and registration with employment centers. By examining participants' labor market outcomes beyond the initial 10-month period, the training program can assess whether participants experience sustained improvements in their employment prospects, earnings potential, and access to employment support services as a result of their participation in the program.

We received approximately 859 valid applications, each indicating participant's preferred class locations and time slots. Our experimental design is based on block randomization (stratification) at the individual level. We randomize participants to one of three treatment statuses - Control (C), Treatment 1 (T1), and Treatment 2 (T2). 522 applicants are allocated to treatment (261 to each treatment group) and the remaining 337 to control. We use five dichotomous variables for stratification, which means that we have 2 x 2 x 2 x 2 x 2 = 32 strata. We group participants into classes based on class location and time slot preferences.

Below we provide the details of our experimental design and randomization algorithm.

-- We employ block randomization (stratification) to assign individuals to one of three groups: Control (C), Treatment 1 (T1), and Treatment 2 (T2). Stratification variables are selected based on their high predictability (R2 sense) and consist of:
- Digital literacy score (continuous variable dichotomized as above/below the median level)
- Education (no diploma, yes diploma)
- Nationality (Italian, non-Italian)
- Place of residency (living in the city center, not living in the city center)
- Presence of preschool-age children (at least one child below age 6, no child below age 6).

Each of the variables used for stratification is dichotomous. This means that we have 2 x 2 x 2 x 2 x 2 = 32 strata.

-- Following this process, we ensure a similar size of eligible participants in each of the C, T1, and T2 groups. Importantly, the stratification variables are balanced across these groups. For instance, 70% of the applicants are foreigners, which implies that each group will have 70% foreign participants.

-- For each treatment group (T1 and T2), we allocate between 7 and 13 individuals (with a median of 10) to a class based on their preferences and treatment type. There are 50 classes offering the shorter training (T1) and 51 classes offering the longer training (T2). Specifically, the allocation of participants to these classes is determined through random sorting, prioritizing preferred choices, and assigning participants to the nearest available class if their preferred option is oversubscribed. This process ensures a fair and balanced distribution of participants across the treatment groups.

-- Applicants in the control groups, who do not attend classes, are allocated to the 51 available classes using a similar algorithm. Consequently, they are grouped into 51 clusters, each containing between 5 and 9 applicants (with a median of 7). Therefore, in total, we have 102 clusters.

-- Balancing tests are conducted focusing on the highly predictive variables identified in step 1.

-- The above steps are repeated 100 times, and the resulting p-values are ranked using a predefined decision rule. The allocation with the largest set of significant p-values is selected.

Randomization done in office using Stata

participant

Yes

102 classes

859

- 25 classes (261 eligible individuals) in treatment 1
- 26 classes (261 eligible individuals) in treatment 2
- 51 classes (337 eligible individuals) in control

We conducted power calculation only for the primary outcome "2) Digital literacy" because it is the only one for which we have reliable data for the specific population of analysis. Our power analysis on this outcome is based on four key factors: 1) Effect size of the treatment: This is calculated from the pilot conducted 1 year ago. In the pilot, the effect sizes on the digital literacy index ranged from 0.40 to 0.60 standard deviations (SD), which is relatively large. 2) Intra-cluster correlation coefficient (ICC): We assess how variable the output is between classes. We calculated this from the baseline survey we collected using the command loneway predicted_index class_assigned. It's quite low at 0.001, which is favorable. 3) Predictive power of strata fixed effects: We employ a stratified random assignment using five variables expected to predict the output variable. While a precise calculation could be based on the pilot study, we lack some variables there. We can only make a rough estimate using a subset of the stratification variables (education, nationality). Based on the pilot results, a conservative estimate is that strata fixed effects explain 20-30% of the variation in the output variable. However, this estimate may be too conservative given that we collect the output variable at baseline, which aids in predicting future outcomes. 4) Number of clusters: We have 25 clusters in T1, 26 T2, and 51 clusters in C. Based on this information, we can conduct a rough power analysis. Presently, the experiment is powered to capture a treatment effect that is 25% of a standard deviation, which aligns with standard practices in the literature.