Decision-focused evaluation of an NGO-led TaRL model in Morocco

Last registered on May 30, 2024

Pre-Trial

Trial Information

General Information

Title
Decision-focused evaluation of an NGO-led TaRL model in Morocco
RCT ID
AEARCTR-0013669
Initial registration date
May 23, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 30, 2024, 3:17 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
IDinsight

Other Primary Investigator(s)

PI Affiliation
IDinsight
PI Affiliation
IDinsight
PI Affiliation
IDinsight
PI Affiliation
IDinsight

Additional Trial Information

Status
On going
Start date
2023-10-23
End date
2024-06-02
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
To achieve the goal of ensuring that over 90% of students acquire basic academic skills by the end of primary school, the Government of Morocco is revisiting its remedial education policy. This study evaluates the impact of the Teaching at the Right-Level (TaRL) approach compared to the standard remedial education program implemented by an NGO, focusing on four key outcomes.

We are conducting a Randomized Control Trial (RCT) to assess the effects of TaRL on: (i) student performance on an extended ASER test, (ii) gender dynamics during remedial education sessions, and (iii) teachers' gender stereotypes towards students, measured through an Implicit Association Test (IAT). We will also evaluate if the increase in students' learning outcomes is proportionally higher than the increase in program costs. To the best of our knowledge, the gender-related impacts of TaRL have not yet been studied in the literature, making this an important area of investigation.

By leveraging administrative data and a difference-in-difference methodology, we will examine the spillover effects of TaRL on the academic performance of non-beneficiary students. Our access to extensive administrative data enables us to use schools without a remedial education program as a control group.
External Link(s)

Registration Citation

Citation
Plat, Estelle et al. 2024. "Decision-focused evaluation of an NGO-led TaRL model in Morocco." AEA RCT Registry. May 30. https://doi.org/10.1257/rct.13669-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Teaching at the Right-Level (TaRL) is a remedial education approach aimed at enhancing students' basic literacy and numeracy skills. This method involves grouping students by proficiency level and subject rather than by age, as is typically done in the national education system. TaRL incorporates a set of activities and a participatory pedagogical approach, with students attending remedial education sessions outside regular school hours.

The implementing foundation will administer Annual Status of Education Report (ASER) tests, adapted to the Moroccan context, to students in grades 3 to 6. Based on their performance in these tests, students will either receive TaRL support (beneficiaries) or no additional support (non-beneficiaries). ASER is a simple, standardized test that estimates children's educational status and basic learning levels.

The remedial education sessions will be organized as follows:

- 100 hours of TaRL activities (4 hours per week) over 25 weeks (approximately 6 months, or one full semester).
- Each week, there will be 1 session of French (2 hours), 1 session of Arabic (1 hour), and 1 session of Mathematics (1 hour).
- This schedule will provide students with 50 hours of French sessions, 25 hours of Arabic, and 25 hours of Mathematics.

Educators were trained in the TaRL methodology during the second semester of 2023. Concurrently, mentors were trained to supervise and support educators in implementing TaRL activities. The pilot phase began in October 2023 and will conclude, including national school holidays, around June 2024.
Intervention (Hidden)
Intervention Start Date
2023-11-06
Intervention End Date
2024-06-02

Primary Outcomes

Primary Outcomes (end points)
Scores at the extended ASER test we developed with our partners
Primary Outcomes (explanation)
Learning outcomes in French, Arabic, and Mathematics will be measured using an extended ASER test, which was constructed with support from the implementing NGO and Pratham. This test is distinct from the one used by the implementing NGO to group students by level.

Secondary Outcomes

Secondary Outcomes (end points)
1) Difference in the implementation cost between the control and treatment interventions.
2) Difference in gender dynamics during remedial education sessions between TaRL and the control intervention.
3) Difference in teachers' implicit gender stereotypes towards students in treatment and control schools.
4) Difference in the trend of average grades given by teachers in treatment and control schools.
Secondary Outcomes (explanation)
2) Gender Dynamics: Gender dynamics will be measured by the frequency of boys' and girls' interventions during the remedial education sessions. An intervention includes raising a hand to speak, speaking when called on by the teacher, speaking without permission, chatting with other students (same or opposite sex), and interrupting (or being interrupted by) another student (same or opposite sex).

3) Teachers' Implicit Gender Stereotypes: We will use two Implicit Association Tests (IATs) to measure teachers' implicit gender stereotypes towards students. The first IAT assesses the stereotype that boys excel in science while girls excel in liberal arts. The second IAT measures the stereotype that girls are more disciplined than boys.

4) Average Grades Given by Teachers: This outcome will be derived from administrative data. For each teacher in our sample schools, we will collect grade data from the past five school years. Using a difference-in-difference method, we will determine if the trend in average grades given by teachers changes due to the implementation of TaRL.

Experimental Design

Experimental Design
We selected a sample from a pool of 129 schools collaborating with the implementing partner. However, not all schools could participate in the study.

39 schools that implemented TaRL last year as part of a pre-test were excluded from the evaluation. Due to logistical constraints, the implementing partner chose not to implement its usual program in the provinces of Inezgane and Khmisset. Only one school in Sidi Slimane remained eligible for inclusion in the sample, as it was not part of the pre-test list submitted by the implementing partner.

To conduct impact analyses by province, a mix of schools implementing TaRL and the traditional remedial education program was necessary. Unfortunately, this arrangement was not feasible in Inezgane, Khmisset, and Sidi Slimane, leading to their exclusion from the sampling frame. Consequently, the final sample frame consisted of 89 schools.

From these 89 schools, a subset of 60 schools was selected for the impact evaluation. Each selected school was randomly allocated to either the TaRL group or the usual remediation group.

Schools were selected based on a stratified randomization procedure to ensure balance along outcomes of interest and representation of subgroups. Stratification was done by province, and within each school, students were randomly drawn in each stratum (gender and school grade) proportionately to the Sanady student population.

Due to implementation issues in the province of Taroudant, this province was excluded from our sample (8 schools). An additional school was removed because the implementing partner was unable to work there. The final sample comprises 51 schools with an average of 22 students per school. All teachers conducting remedial education in each school will take the Implicit Association Test, with an expected average of 3 teachers per school.
Experimental Design Details
Randomization Method
Randomization was conducted in office by a computer.
Randomization Unit
Two-stage cluster random sampling:
First stage is schools ;
Second stage is students
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
51 schools
Sample size: planned number of observations
1,140 pupils Approximately 153 teachers for the Implicit Association Tests
Sample size (or number of clusters) by treatment arms
26 schools control, 25 TaRL schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
To calculate power we use 80% power and 5% alpha. For ASER score, we assume an ICC of 0.19 based on baseline data and a correlation between baseline and endline scores of 0.6 which leads to a MDE of 0.1 s.d.
IRB

Institutional Review Boards (IRBs)

IRB Name
IDinsight internal IRB
IRB Approval Date
2024-09-29
IRB Approval Number
N/A

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials