Experimental Evidence on the Effects of a Coursera Program on Labour Outcomes in Costa Rica

Last registered on June 18, 2024
View Trial History

Pre-Trial

Trial Information

General Information

Title
Experimental Evidence on the Effects of a Coursera Program on Labour Outcomes in Costa Rica
RCT ID
AEARCTR-0011483
Initial registration date
May 26, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 06, 2023, 3:25 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
June 18, 2024, 6:30 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country
Costa Rica
Region
Latin America and the Caribbean

Primary Investigator

Name
Richard Freund
Affiliation
University of Cape Town
Contact Primary Investigator

Other Primary Investigator(s)

PI Name
Rafael Novella
PI Affiliation
University College London
Contact Investigator
PI Name
David Rosas-Shady
PI Affiliation
Inter American Development Bank
Contact Investigator

Additional Trial Information

Status
Completed
Start date
2020-08-01
End date
2023-09-30
Keywords
Education, Labor
Additional Keywords
JEL code(s)
E24, J01, J24, J62, I26
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Massive open online courses (MOOCs) are frequently viewed as a tool for democratizing education and job training. However, their effectiveness is largely untested. The existing evidence of the value of online courses is generally based on observational data where the selection of unobservable characteristics is a major concern (see e.g., Castaño-Muñoz and Rodrigues, 2021; Hadavand, 2018).

To address this issue, we carry out a randomized controlled experiment in Costa Rica in which the treatment group received cost-free access to Coursera MOOCs and certificates. Additionally, treated individuals were assigned to one of two behavioural intervention arms, where they either received light-touch reminders, or personalised reminders.

We are primarily interested in investigating three main questions: (i) What is the profile of the individuals who complete a course?; (ii) Is a more comprehensive, personalized reminder more effective in increasing course completion than a standardized email reminder?; and (iii) What are the labour market effects of completing a Coursera MOOC? To better understand the third question, we also analyse the heterogeneity of the treatment effects and study the impact of treatment on educational outcomes.
External Link(s)

Registration Citation

Citation
Freund, Richard, Rafael Novella and David Rosas-Shady. 2024. "Experimental Evidence on the Effects of a Coursera Program on Labour Outcomes in Costa Rica." AEA RCT Registry. June 18. https://doi.org/10.1257/rct.11483-1.1
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
In this intervention, randomly selected applicants whose jobs were affected by the pandemic were offered cost-free access to Coursera online courses and certificates. Treated individuals were also assigned to to one of two behavioural intervention arms - a light-touch reminder group, or a more comprehensive personalised reminder group.
Intervention (Hidden)
Intervention Start Date
2020-08-01
Intervention End Date
2021-02-28

Primary Outcomes

Primary Outcomes (end points)
First, we are interested in assessing treatment take-up. The key outcome variables for this are where individuals: i) Activated their Coursera license ii) Enrolled in at least one online course, iii) Completed at least one online course, iv) The number of courses completed.

Second, we are interested in assessing the labour market effects of the treatment. For this, the key outcome variables are: i) Current employment status, ii) Monthly income, iii) Formal job.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
As secondary outcomes, we are interested in the effect of treatment on other labour market outcomes. These include whether individuals are active in labour market, engaged in work related to what they applied to in Coursera, self-employed, have a tenured job, and are working full-time.

We are also interested in the effect of treatment on educational outcomes. For this, we are interested in whether individuals are currently studying, studying full-time (i.e., not working or looking for work), and are engaged in online education.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The design followed an oversubscription model, whereby the program sought to attract a pool of eligible applicants that was at least twice the capacity the vacancies reserved for evaluation. From this, eligible applicants were randomly assigned to receive cost-free access to Coursera courses or not. After eligible applicants were assigned to receive treatment or not, a second round of randomization was implemented to assign treatment individuals to one of two behavioural intervention arms.
Experimental Design Details
Randomization Method
Randomization done in office by a computer
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
8543 individuals
Sample size: planned number of observations
8543 individuals
Sample size (or number of clusters) by treatment arms
4,271 individuals control, 4,272 individuals treatment. Within treatment, 2,136 individuals assigned to either behavioural intervention arm.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
IRB approval letter
Document Type
irb_protocol
Document Description
Attached is the IRB approval letter from the Human Subjects Committee for Innovations for Poverty Action IRB-USA.
File
IRB approval letter

MD5: 88e56f13fb49369d6359dff5a80de15a

SHA1: 49b7b440a3fe5c2a53a8626fc87c2fafe3b5a4e5

Uploaded At: May 25, 2023

IRB

Institutional Review Boards (IRBs)

IRB Name
Human Subjects Committee for Innovations for Poverty Action IRB-USA
IRB Approval Date
2022-12-20
IRB Approval Number
N/A
Analysis Plan

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Massive open online courses (MOOCs) are frequently viewed as a tool for democratizing education and job training. However, their effectiveness is largely untested. We report on the first randomized evaluation of a job training program offering cost-free access to curated, short Coursera MOOCs and certificates. We find low course completion rates (10%), with males and wealthier individuals more likely to complete a course. Personalized reminders did not increase treatment take-up over a simple, standardized email reminder. Treatment has no significant effect on labour market outcomes roughly two years after the program. However, we find marginally significant evidence that treatment increases post-secondary education enrolment by 11%. Evidence on mechanisms suggests that this may be operating partially through the program motivating individuals who lacked sufficient skills to pursue further specialization.
Citation
Novella, R., Rosas-Shady, D. and Freund, R. (2024). Is online job training for all? Experimental evidence on the effects of a Coursera program in Costa Rica. Journal of Development Economics 169, 103285.
URL
https://doi.org/10.1016/j.jdeveco.2024.103285

Reports & Other Materials