Targeted instruction and foundational academic skills: Evaluating the impacts of the FastTrack program in Nigerian IDP camps

The FastTrack program combines three proven, independent teaching and learning approaches: Mavis Talking Book and Pen (an offline digital pen with audio capabilities), Teaching at the Right Level (TaRL); and mothertongue-based literacy acquisition model: a dual language approach which involves using Hausa as abridge to learn English.

The program is implemented in three broad stages: Selection, Learning Delivery, and Validation. At each stage, the learner's experience is Facilitator-guided, framed around the Teaching-at-the-Right-Level (TaRL) approach and tailored to the learners' context. Following assessment of their individual learning levels, learners are placed in groups for foundational learning in literacy and numeracy. This component of the program runs for 6
months. Learning delivery is multimodal, comprising direct facilitation based on TaRL approach, use of the talking pen, peer interaction, and formative assessments. The program runs 4 times weekly for 2 hours each meeting (8 hours per week total).

2024-11-04

2025-05-04

Outcome Construct 1: Child Functional Literacy Skills
Outcome Construct 2: Child Functional Numeracy Skills

Outcome 1: Child language and literacy skills will be assessed using several different literacy and language measures in order to capture various levels of reading ability, mother-tongue language and reading skills, and oral language. The Annual Status on Reading Report (ASER; Pratham, 2005) will be used initially to level students and place them at the right level. ASER includes letter-word identification, short sentences, and passage comprehension. Literacy is further operationalized as an index of scores on five subscales of emergent literacy adapted from the Early Grade Reading Assessment (EGRA; RTI International, 2012) and the International Development and Early Learning Assessment (IDELA; Pisani, L., Borisova, I., & Dowd, A. J., 2015), which include phonological awareness (awareness of and ability to manipulate the sound units of language), recognizing letters of the alphabet, reading simple words, understanding sentences and paragraphs, and listening with comprehension. IDELA captures pre-literacy skills,specifically first letter sounds, allowing for a wider-range of skill levels to be captured. The Hausa-language version of EGRA (RTI International, 2014) and Kanuri-language version (Mathematica Policy Research, 2015) will capture children’s first language and literacy skills. Lastly, oral language skills will be assessed using both the WIAT receptive vocabulary picture-based task (Wechsler Individual Achievement Test; Wechsler, 1992) and Woodcock Johnson IV synonym and antonym generation tasks (Woodcock-Johnson III NU Complete; Woodcock, McGrew, & Mather, 2001).

Outcome 2: Children’s numeracy skills are assessed first using ASER (Pratham, 2005) through single and double digit recognition, basic subtraction, and division. Children are leveled using ASER and placed into the corresponding numeracy instruction that is appropriate. A broader set of children’s numeracy skills are also assessed using the Early Grade Mathematics Assessment (EGMA; RTI International, 2012) and IDELA (Pisani, Borisova, & Dowd, 2015) and include six subtasks: number identification, reasoning about magnitude, recognition of number patterns, addition and subtraction, and word problems. IDELA collects pre-numeracy skills and allows for a wider-range of numeracy skills to be captured.

Secondary Outcome 1: Socio-emotional skills
Secondary Outcome 2: Psychosocial well-being
Secondary Outcome 3: Schooling
Secondary Outcome 4: Cognition
Secondary Outcome 5: Program participation
Secondary Outcome 6: Family wellbeing
Secondary Outcome 7: Family investments in education

1. Socio-emotional skills.
International Social and Emotional Learning Assessment (ISELA; D'Sa & Krupar, 2021).
Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001): The SDQ assesses well-being (emotional-behavioral regulation and prosocial behavior) in children 3-17 years. It consists of 25 items in five subscales: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems and prosocial behavior. The SDQ has been extensively evaluated and applied worldwide (Woerner et al., 2004). Children ages 11 and older complete a self-report, for children 10 and under, parents complete a report.
The Holistic Assessment of Learning and Development Outcomes (HALDO) for situational analysis in conflict and crisis settings. HALDO measures the literacy, numeracy, and social and emotional learning of children ages 4 to 12 years affected by conflict and crisis. (Jones et al., 2020)

2. Psychosocial well-being
International Survey on Children’s Well-Being (ISCWeB; Casas & Rees, 2015) This measure targets multiple factors relating to children’s well-being, satisfaction with various aspects of their life, living conditions, material possessions, time use and experiences of daily life. The international survey was conducted with children ages 8, 10, and 12 years in multiple countries (including Ethiopia and S. Africa),and is appropriate for the age range and location of the FastTrack beneficiaries. ISCWeB includes measures of living situation, home and family relationships, money and economic circumstances, friends and other relationships, local area, school, time use, self overall subjective well-being, and children’s rights. The survey incorporates (1) measures of overall cognitive subjective well-being from a short modified version of The Student Life Satisfaction Scale (Huebner & Gilman, 2002), (2) the seven-item Personal Well-being Index - School Children (Cummins & Lau 2005), (3) a modified Brief Multidimensional Student Life Satisfaction Scale (Huebner et al., 2006), (4) a set of six items on positive affect which were influenced by Russell‘s Core Affect scale (Russell & Barrett, 1999), and (5) a set of six items designed to represent various aspects of psychological well-being.
Child & Youth Resilience Measure-Revised (CYRM-R; Jefferies et al., 2018). This self-report measure of social-ecological resilience used by researchers and practitioners worldwide.

3. Schooling
Enrollment and school progression. Whether the child enrolls in school after completing the FastTrack program and the child’s longer term progression in school (e.g. did the child complete the grade, grade repetition, primary school completion, secondary school enrollment).
Educational aspirations and expectations (Gottlieb et al., 2009) Two items including: “How far would you like to go in school?”, “Realistically, how far do you expect to go in school?”

4. Cognition
Working Memory (Diamond et al., 2007). Visual: Children see some sequences of numbers displayed on the screen of the tablet for a few seconds. Child is asked to read the sequence, try to memorize it, and write the sequence to the experimenter.

5. Child program participation: Attendance using administrative records.

6. Family wellbeing: Measures on parental mental health, household wealth, migration history
Household assets.The Poverty Probability Index (PPI; Nigerian version) and Multidimensional Poverty index (MPI) will be completed by parents.
MacArthur Scale of Subjective Social Status (Cundiff et al., 2013). A single-item measure that assesses a person’s perceived rank relative to others in their group.
The Patient Health Questionnaire (PHQ-9) is a measure for mental health, used for measuring depression.

7. Family investments in education.
We use scales and items to understand parent involvement in a child's educational activities, parents’ perceptions of educational value and opportunity, parents’ expectations and aspirations about children’s schooling. Parental questionnaire, two items include “How far would you like your child to go in school?”, “Realistically, how far do you expect your child to go in school?”
Parents’ social, economic, and demographic background (i.e. languages spoken, education, SES, family composition, migration history) will be measured. Language Environment Questionnaire (ALEQ-4; Paradis 2011): Parent questionnaire that yields information on children’s language development history,parent education and fluency in English and a mother tongue, home and school language use, and literacy activities. This questionnaire is used to establish a child's home language background.

Intervention Factor 1: Full FastTrack program with targeted instruction plus the Mavis Pen
Intervention Factor 2: FastTrack program with targeted instruction only.
Control Group: Waitlisted

Randomization done in office by a computer

Child-level

No

1,750 children

1,750 children

NA

Our design is a person-randomized trial within IDP camps. We use Optimal Design software and the following standard assumptions: 80% power, and R-squared=0.2 (given child-level baseline data). The target sample size is 1,750 children and assumes 15% attrition (a total of 500 per treatment arm for analysis). This allows for an MDES of 0.16 per treatment contrast (i.e., FastTrack versus control, FastTrack without Mavis Pen versus FastTrack). The same assumptions using 90% power lead to an MDES of 0.18. For child-level subgroup analyses (i.e., sex, baseline ability, relative disadvantage), we are powered to detect an MDES of 0.23 and 0.26 for 80% and 90% power, respectively. Previous research with targeted instruction programs has shown effect sizes of around 0.1 SD, with school-level randomization and a counterfactual of “business as usual” school attendance compared to targeted instruction within schools. In our study, the control receives no formal schooling, and thus effect sizes are likely to be larger than the previous literature. Similarly, a recent meta-analysis of targeted instruction programs showed much larger effect sizes when programs were implemented by volunteers versus teachers (Angrist & Meager, 2023), attributing this to motivation of the volunteers and higher implementation fidelity. The Fast Track program is implemented by full-time volunteer facilitators (as opposed to teachers who implement the program in addition to other responsibilities). We anticipate implementation to be similar to programs that have evaluated volunteer-implemented targeted instruction programs. Impacts of such programs, and improvement in foundational literacy and numeracy skills on child well-being have been very under-studied in these populations and contexts. Thus, we view impacts on our secondary outcomes as exploratory and do not have a body of evidence to draw on to establish a reasonable effect size. We hope these analyses will be hypothesis-generating for future research on how education impacts vulnerable children’s holistic development in the short- and longer-term.