Back to History Current Version

The Effects of a Tech-Based Tutoring Program in Ukraine

Last registered on December 16, 2022

Pre-Trial

Trial Information

General Information

Title
The Effects of a Tech-Based Tutoring Program in Ukraine
RCT ID
AEARCTR-0010634
Initial registration date
December 13, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
December 16, 2022, 4:03 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
The World Bank

Other Primary Investigator(s)

PI Affiliation
The World Bank
PI Affiliation
The World Bank
PI Affiliation
The World Bank

Additional Trial Information

Status
In development
Start date
2023-01-02
End date
2023-09-29
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Evidence suggests that traumatic events, such as pandemics and wars, can impact children’s learning, socio-emotional development, and sense of protection (Quintana-Domeque and Ródenas-Serrano, 2017; Almond et al. 2018). Ukraine’s education system faces critical constraints in providing high-quality education to its students on their path toward recovery after disruptions to schooling and learning due to years of pandemic-related school closures. While children in many countries have gone back to school, the return to in-person education has been hindered by a lack of security, significant student and teacher displacement, and school damages posed by 6 months of the Russian invasion. Currently, learning losses in Ukraine are estimated to be over one year (Angrist et al, 2022), with learning outcomes falling below the lowest-performing countries in Europe which will have substantial impacts on human capital development in the country. To mitigate the impact of these traumatic events on children, the Ukrainian education system must find new strategies for supporting learning recovery and increasing learning equity while children are not able to return to in-person schooling.

High dosage small group instructional tutoring can be a powerful strategy to improve learning outcomes and cognitive and socioemotional skills, as it offers students a massive increase in personalized instruction, enabling teaching at the right level (Banerjee et al., 2015). To test the effectiveness of these programs in a conflict-affected setting, we study a tutoring program offering supplemental learning in math and Ukrainian language and psychosocial support. The target population of the tutoring program is Ukrainian students in grades 5 to 10 who are seeking supplemental support beyond the standard online schooling schedule. Initially, students are placed in groups of 3 and will receive 3 hours of tutoring per week for 6 weeks by paid-for tutors through the Edmodo online platform. The implementing partner is Teach for Ukraine.

The impact evaluation will include three waves, through which we will test for the effectiveness of varying certain attributes, including program length, group size, content allocation, group composition by ability, and tutors as role models. Students will be recruited in each wave. All waves will have one control and one treatment group. Waves 2 and 3 will have an additional group composed of students who participated in the control group in the previous wave who will receive the treatment. In this group, we will either test the effect of group composition or pilot an attribute to be tested in the following wave.

The effects of the tutoring program will be measured in math and Ukrainian language test scores and socioemotional skills. As secondary outcomes, we will measure the effects of the intervention on expectations, mental health, time use, attendance to the tutoring activities, and attitudes towards tutoring.
External Link(s)

Registration Citation

Citation
Dinarte, Lelys et al. 2022. "The Effects of a Tech-Based Tutoring Program in Ukraine." AEA RCT Registry. December 16. https://doi.org/10.1257/rct.10634-1.0
Experimental Details

Interventions

Intervention(s)
The intervention under evaluation consists of a tutoring program offering supplemental learning in math and Ukrainian language and psychosocial support. The learning component includes subject-specific academic content in accordance with Ukrainian educational programs. The socio-emotional part includes activities to support student’s social and emotional well-being. The total scheduled time for psychosocial support activities is 30 mins per week. The target population of the tutoring program is Ukrainian students in grades 5 to 10 who seek supplemental support beyond the standard online schooling schedule. Students are placed in groups of 3 and will receive 3 hours of tutoring per week for 6 weeks by paid-for tutors through the Edmodo online platform. The implementing partner is Teach for Ukraine.
Intervention (Hidden)
The intervention under evaluation consists of a tutoring program offering supplemental learning in math and Ukrainian language and psychosocial support. The learning component includes subject-specific academic content in accordance with Ukrainian educational programs. The socio-emotional part includes activities to support student’s social and emotional well-being. The total scheduled time for psychosocial support activities is 30 mins per week. The target population of the tutoring program is Ukrainian students in grades 5 to 10 who seek supplemental support beyond the standard online schooling schedule. Students are placed in groups of 3 and will receive 3 hours of tutoring per week for 6 weeks by paid-for tutors through the Edmodo online platform. The implementing partner is Teach for Ukraine.
Intervention Start Date
2023-01-23
Intervention End Date
2023-06-30

Primary Outcomes

Primary Outcomes (end points)
Math test score
Ukrainian language test score
Socioemotional skills (locus of control, GRIT/perseverance)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Personal and Academic Expectations
Mental health (stress and anxiety)
Time use
Attendance of the tutoring activities
Attitudes towards tutoring
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The impact evaluation will be conducted in three waves of tutoring. In each wave, we will test for the effectiveness of varying certain attributes, including program length, group size, group composition by ability, tutor feedback, and tutor value-added. The exact attribute to be tested in each wave will depend on the results from the previous wave.

Students will be recruited in each wave. The tutoring program will be advertised on social network platforms and disseminated by the Ministry to schools around the country, influencers in the Education Sector, and TFU. Students interested in taking part in the tutoring program will register online, provide responses to a questionnaire, and provide their own consent as well as parental consent. Students who submit this information will become part of the study sample and, for all the 3 waves, will be randomly assigned to the treatment or the control group.

The randomization procedure for each wave will consist of stratified household-level randomization. The stratification variables for each wave will be student grade, preferred schedule (Mon-Tue-Wed or Wed-Thu-Fri), and if they were in or outside Ukraine. Then, students randomly allocated to a treatment status will be randomly assigned to each of the tutors. All treatment groups will receive a variation of the tutoring program in groups of 3 students.

All waves will have one control and one treatment group. For waves 2 and 3, we will add an additional treatment group that will consist of students in the control group in the previous wave. For example, in wave 2, we will work with three groups. We randomly assign enrolled students to treatment or control, and we will also treat students in the control group from wave 1. These students will be randomly assigned to a specific attribute, such as providing teachers feedback or estimating tutor value-added.
Experimental Design Details
Randomization Method
Randomization will be done in the office using a code developed by the PIs
Randomization Unit
The unit of randomization will be the household. Yet, based on pilot data, the average number of students enrolled per household is 1.2. Therefore, we expect our randomization to be almost at the individual level.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
8,000 households
Sample size: planned number of observations
10,000 students
Sample size (or number of clusters) by treatment arms
Wave 1:
Treatment 1 --> 1,200
Control --> 2,000

Wave 2.
Treatment 2 --> 2,000
Treatment 3 (Controls from wave 2) --> 1,400
Control --> 2,000

Wave 3.
Treatment 4 --> 2,000
Treatment 5 (Controls from wave 2) --> 1,400
Control --> 2,000

Using pilot data, we are assuming attrition rates of 30% for the control group across waves.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Assuming attrition of 30%, take up of 52%, a power of 80%, and a type I error rate of 0.05, for a sample size of 3,000, we estimate an MDE of 0.13SD.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials