After School Tutoring: A randomised controlled trial

Last registered on January 02, 2022

Pre-Trial

Trial Information

General Information

Title
After School Tutoring: A randomised controlled trial
RCT ID
AEARCTR-0008736
Initial registration date
January 01, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 02, 2022, 10:11 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Monash University

Other Primary Investigator(s)

PI Affiliation
Monash University
PI Affiliation
Monash University &CEPR
PI Affiliation
Monash Univeristy

Additional Trial Information

Status
Completed
Start date
2013-11-01
End date
2017-02-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
Tutoring has become widespread, particularly in developing countries. A two-stage randomized controlled trial was implemented of an after-school free tutoring program conducted in 254 schools over two years from year 2013 to year 2015 in rural Bangladesh. The program offered students in primary schools group-based private tutoring and/or a weekly one-to-one home visit. In the PT-only treatment, tutors provided two hours of teaching English and Math to students in a group setting. In the HPT treatment, in addition to receiving tutoring in a group setting, students received one-hour one-on-one instruction from a tutor every week. The purpose of the home visit was to provide individualized sessions, teaching at the students’ level to identify the individual learning gap and meet their learning needs.

The RCT involved a two-stage design where, in the first stage, all schools were randomly assigned as either treatment or control schools. The second stage involved randomly selecting subgroups of students from each grade within each treatment school and offering them the intervention.

The intervention lasted for two years from year 2013 to year 2015, and various dimensions of data collected at different period of time. Examples include test scores and non-cognitive skills of students, parent, teacher and school information. Successful randomization is guaranteed based on the fact that student-level characteristics and school-level variables are balanced across groups. Then the effect of the program on educational outcomes of students was estimated.
External Link(s)

Registration Citation

Citation
Islam, Asad et al. 2022. "After School Tutoring: A randomised controlled trial." AEA RCT Registry. January 02. https://doi.org/10.1257/rct.8736
Experimental Details

Interventions

Intervention(s)
The main objective of the tutoring program is to improve the learning outcomes of rural children. Global Development Research Initiative (GDRI), an NGO, implemented the intervention in collaboration with the Directorate of Primary Education in Bangladesh. The intervention offered free outside school hours tutoring for two years from 2013 to 2015. The intervention took place in public schools located in two south-western districts of Bangladesh, Khulna and Shatkhira.

In year 1, the tutoring program started in March 2014 among students in grade 3 and grade 4 and lasted for nine months until November 2014 when the students sat for the end-of-year exam. In year 2, the tutoring started in late February with the same cohort of students who progressed to grade 4 and grade 5, respectively. The program ended in November 2015.

We implemented a two-stage RCT design. The first stage involved assigning schools in these districts into treatment or control groups. In the second stage, students in each grade were randomly selected for free private tutoring. Thus, some students from the same grade did not receive free tuition from the program. Schools were randomly assigned to one of the three types of treatments- free small group tutoring to random groups of students(PT, treatment 1); free tutoring plus an additional weekly one-to-one session to random groups of students (HPT, treatment 2) ; free small group tutoring to most central students in each classroom (PT-central, treatment 3).

In PT treatment (treatment 1), students received tutoring for about 2 hours a day in a small group/batch, three days a week. The tutors taught mathematics and English, the two subjects that most Bangladeshi students find difficult. The tutors helped the students complete homework, prepare for exams, and understand the relevant basic concepts to develop more foundational skills.

In the HPT treatment (treatment 2), in addition to private tutoring as described above, the tutors also made home visits once a week to provide individual tutoring sessions for each student. The purpose of adding an individual session through home visit was to make sure that students were taught in accordance with their abilities and individual needs were better assessed and addressed. Tutors were also instructed to focus on the students' individual needs, so each student gets the opportunity to learn something that is difficult to cover in a group setting. During the home visit, tutors spent about an hour at students’ homes.

In PT-central treatment (treatment 3), treated students receive private tutoring which is same as treatment 1, but selected based on students' centrality, only central students in each classroom was treated. Centrality is calculated based on baseline social network survey in which each student was asked to nominate who are their best friends in his/her classroom, then students were ranked from high to low in terms of their centrality. Only students with high centrality in each classroom were selected to receive treatment.
Intervention Start Date
2014-02-01
Intervention End Date
2015-11-01

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are represented by cognitive skills and non-cognitive skills of students. Cognitive skills are represented by English and Math test scores of students, collected at different points of time. The tests were conducted among all students, especially for evaluating the intervention, and hence they are different from school-administered final exams or other exams. The tests focused on actual learning to understand how much students really have learned.

Our second group of outcome variables consist of measures of the noncognitive skills of the students. To obtain these measures, we designed several questions to explore three main domains of noncognitive skills of each student: self-control, social skills, and motivation. School teachers (not the private tutors) provided an evaluation of a student in these dimensions. The questions collected to measure each skill are listed in the detail section.

Primary Outcomes (explanation)
The test items consist of both multiple- choice and short questions. The test is intended to assess problem-solving capacities in mathematics (e.g., geometric skills and complex worded problems), and English comprehension requiring students verbal, language, and reasoning skills. Local school teachers and educators were consulted to ensure that the tests are appropriate for the grade level. In addition, test scores were collected in the range of 0 to 100. In order to make it comparable across groups, we normalize these test scores of each student by subtracting the average test score of the control group in the respective year and dividing by the standard deviation.

In terms of non-cognitive skills, they are measured by three dimensions including self control, social skills and motivation to learn of a student. The questions used to measure each category are listed below.

Self control
(1) Is the student regular in class? 1=Yes, 0=No
(2) Does the student regularly do his homework? 1=Yes, 0=No
(3) Does the student regularly do his homework? 1=Yes, 0=No

Social skills
(1) How adaptive is the student in making friends? 1=Moderate, 2=Much, 3=Very much
(2) How adaptive is the student in sustaining relationships? 1=Moderate, 2=Much, 3=Very much
(3) How does the student behave with classmates? 1=Competitive, 2=Irreconcilable 3=Friendly

Motivation
(1) How much attentive is the student in class? 1=Moderate, 2=Much, 3=Very much
(2) How desirous of study is the student? 1=Moderate, 2=Much, 3=Very much
(3) How eager is the student in learning lessons? 1=Moderate, 2=Much, 3=Very much
(4) Involvement in taking challenging job 1=Moderate, 2=Much, 3=Very much
(5) How much does the student enjoy doing challenging jobs 1=Moderate, 2=Much, 3=Very much
(6) Does the student take challenging work? 1=Sometimes, 2=Often, 3=Very often

The standardization of non-cognitive skills are as follows. After answers are collected for each skill, we aggregate the corresponding questions into one variable by calculating the average across the responses (all responses are numerical), so all measures take values in [0, 1]. We then normalize the measures by subtracting the mean and then dividing by the standard deviation of the control group. We also construct an Index, which is the average of these three measures and it represents the overall non-cognitive ability of a student.

Secondary Outcomes

Secondary Outcomes (end points)
Apart from primary outcomes, we collected other outcomes in order to identify the possible channels to explain why private tutoring intervention matters, these outcomes are represented by different surveys conducted during and after the completion of the intervention.

Firstly, we collected student survey at the end of 2015 just before the completion of the intervention, the survey gathered information on the distribution of out-of-school time between studies, leisure, self-esteem, ambition, confidence, etc. The answers were obtained from survey responses of parents about their children, and students’ self-reporting their behaviors. The purpose of student survey is to measure how much effort students have spent on study in order to evaluate whether private tutoring intervention shaped students' behavior.

Next, we collected parent survey one year after the intervention finished in 2017. The key variables measure parent behavior and investment towards a child’s education. The purpose of parent survey is to understand whether the program increase or decrease parent effort devoted on their children's education. By examining parents’ time and resource allocation on their children, we are able to understand whether private tutoring increased or decreased parent effort and whether there are persistent effect on parents after intervention finished.

In addition, we collected teacher survey in August, 2015 which contains information on teaching methods and teachers’ level of effort in the class. These data were collected by random visits to the classrooms by the enumerators during the intervention and the teachers were not pre-informed about the visits. The objective of teacher survey is to assess whether tutoring program would change the behavior of class teachers. For example, they may be more engaged in class activities as the program could a threat to their position ; Or they could be less engaged in class activities given students already obtained help from private tutors.

Finally, we also collected cognitive skills of students one year after the intervention finished in order to assess whether private tutoring generated long-term effects that persist after the intervention finished. The cognitive skills are represented by PSC exam scores. PSC exams are externally administered public exam for the grade 5 students, called the primary school certificate (PSC) exam. A student needs to pass the PSC exam to move to secondary school, and the results for this PSC exam are taken seriously by teachers, parents, and students.
Secondary Outcomes (explanation)
As summarised above, the secondary outcomes mainly come from three aspects of surveys including student survey, parent survey and teacher survey.

The student survey consists of questions that describe student effort devoted on study, represented by the amount of time they spent on out-of-school activities. Outcomes include how much time spent on personal study, household chores, watching TV, sports participation, and study with tutors. The amount of time are measured by hours spent in a day regarding above activities.

The parent survey focused on parents' behavior towards a child’s education including time spent helping them study, money spent on study materials and tutoring, as well as whether they monitor their child’s performance. Some variables are continuous, For example, the number of hours spent by parents to study with children, the amount of money invested on children education and tutoring. Some variables are categorical values such as 1 to 5 where 1 is not at all and 5 is very much. In order to make interpretation of results easy, we converted categorical values into 0 or 1 where 0 stands for no and 1 stands for yes.

The teacher survey records their effort devoted in class. We considered several aspects of teachers’ effort within the classroom, such as whether the teacher arrived on time to class, gave homework or other assignments, checked the homework, administered exams, used any teaching material (such as syllabus, lesson plans etc.) or tools (such as maps, charts, posters etc.) to deliver their lessons, etc. Some questions are recorded as a Yes or No, while some questions are recorded in different scales such as 1 to 5 where 1 is not at all and 5 is very much. Similar as parent survey, we converted categorical values into 0 or 1 where 0 stands for no and 1 stands for yes.

Experimental Design

Experimental Design
We use a two-stage randomization design where we first assign each school to one of the four experimental groups described above in the intervention section, and then assign students within treatment schools (PT, HPT, and PT-central) to receive treatment. In total, 254 primary (public) schools participated in our study randomly selected from two rural districts in Bangladesh, Satkhira and Khulna.

These 254 primary schools were randomly allocated into treatment and control groups, resulting in 100 schools allocated to the control group and the other 154 schools allocated to one of the treatment groups: 58 schools were randomly assigned to Treatment 1 (PT: private tutoring only ), 58 schools were randomly assigned to Treatment 2 (HPT: private tutoring and home visits), and 38 schools were randomly assigned to Treatment 3 (T3: centrality-based private tutoring). The school-level randomization was conducted based on school-level characteristics collected in the baseline survey such as facility, transportation, number of rooms, teacher characteristics and so on. Successful randomization is guaranteed given school-level characteristics are balanced across PT, HPT and PT-central groups.

After school-level randomization, we implemented student-level randomization. In details, within PT and HPT schools, a subgroup of students in each classroom were randomly chosen to receive the treatment. We used baseline test scores of students to randomize and guarantee that baseline test scores of students are balanced across groups and across treated and untreated students. In the 38

Treatment 3 (PT-central) schools, treated students were not chosen at random but selection was based on their eigenvector centrality. Eigenvector centrality is calculated according to network survey implemented in the baseline survey. In fact, in each school, only students with the highest centrality were treated and these students only received private tutoring, i.e., the same intervention as treated students in PT treatment (treatment 1). All students in each classroom were asked to nominate and rank ten of their closest friends among their classmates. To be more precise, we asked each student to pick their ten best friends and rank them from 1 (very close friends) to 10 (least close friends). Then centrality measures the degree of influence a student is among his classmates.
Experimental Design Details
Randomization Method
Randomization is done by a computer based on student-level and school-level characteristics collected in baseline surveys
Randomization Unit
Two levels of randomization, one is school-level randomization (randomize schools into treatment and control groups), the second one is student-level randomization (randomize treated and untreated students within treatment schools)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
254 schools
Sample size: planned number of observations
14,216 students
Sample size (or number of clusters) by treatment arms
58 schools PT treatment: private tutoring only, 58 schools HPT treatment: private tutoring and home visit, 38 schools PT-central treatment: private tutoring to central students, 100 control school
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Monash University Human Research Ethics Committee (MUHREC)
IRB Approval Date
2013-08-19
IRB Approval Number
CF13/2102 – 2013001096

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 01, 2015, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
November 01, 2015, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
254 schools
Was attrition correlated with treatment status?
Yes
Final Sample Size: Total Number of Observations
14,216 students
Final Sample Size (or Number of Clusters) by Treatment Arms
58 schools in PT treatment (treatment 1), 58 schools in HPT treatment (Treatment 2), 38 schools in PT-central treatment (treatment 3), 100 schools in control group
Data Publication

Data Publication

Is public data available?
No

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials