Matching Children with Level-Appropriate Books and Engaging Families

Last registered on February 27, 2017

Pre-Trial

Trial Information

General Information

Title
Matching Children with Level-Appropriate Books and Engaging Families
RCT ID
AEARCTR-0001034
Initial registration date
February 27, 2017

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
February 27, 2017, 12:33 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
ITAM

Other Primary Investigator(s)

PI Affiliation
QFD

Additional Trial Information

Status
On going
Start date
2015-12-14
End date
2017-06-15
Secondary IDs
Abstract
Reading skills are an important predictor of educational attainment and, consequently, of other long-term outcomes. School curriculum often follows the “one size fits all” rule, which means that every student within a classroom is assigned the same reading materials, regardless of their reading skills. This practice potentially leads to adverse effects for students in the extremes of the reading abilities’ distribution, which is particularly negative for those falling behind. In this project, we test an alternative system to select reading material for early grade readers: we build a web-based platform that gives book recommendations tailored to readers’ abilities and interests; participants can borrow books for free at the community library. We also test whether parental involvement could complement this strategy. We implement a randomized control trial to assess the potential benefits of this intervention on students’ reading proficiency and habits.

Registration Citation

Citation
Aguilar, Arturo and Maria Elena Ortega. 2017. "Matching Children with Level-Appropriate Books and Engaging Families." AEA RCT Registry. February 27. https://doi.org/10.1257/rct.1034-1.0
Former Citation
Aguilar, Arturo and Maria Elena Ortega. 2017. "Matching Children with Level-Appropriate Books and Engaging Families." AEA RCT Registry. February 27. https://www.socialscienceregistry.org/trials/1034/history/14431
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The presented intervention, hereon "Mundo de Libros" (MdL), program aims to improve reading skills and habits of primary students enrolled in Grades 1 to 3 in Spanish-speaking countries. This free program is independent of their school curriculum and runs in community libraries. The intervention has three core components:
1. Access to children’s books at the library and that can be taken home. Children have a program passport – similar to a library card – that allows them to borrow books and keep track of due-dates. Each library is equipped with a stock of 720 children’s books. The book collection is diverse in terms of difficulty and topics, thus ensuring that every child will have a wide spectrum of choices. All books in the project are new at the time of the start of the intervention.
2. Access to an individual profile (with login and password) through the web-based platform (www.mundodelibros.mx). Each child’s profile on the platform gives them personalized book recommendations according to their assessed level of vocabulary and reading skills. These recommendations are determined by QfD’s MATCH algorithm that takes into account the reading level of each child and the characteristics of each book. The reading level of the child is determined with an initial test that includes the EGRA and Peabody Picture Vocabulary standardized tests. After logging into the site, participants see recommendations, are able to filter titles according to interests, and search for specific titles, authors or words. The website also allows users to rate the books (on a scale of one to five) after returning them, which is later used to give recommendations based on their interest.
3. Workshops and materials for parents or caregivers. The main objectives of these workshops and materials is to: (i) promote parental engagement, (ii) provide information and strategies to scaffold their children’s reading practices, and (iii) advise how to create a rich literacy environment at home. Workshops for parents or caregivers take place every two months.
Intervention Start Date
2016-01-11
Intervention End Date
2017-04-15

Primary Outcomes

Primary Outcomes (end points)
To measure the impact of the program we use three different instruments, to be collected at beseline and endline: EGRA, TVIP, and a Reading Habits and Attitudes survey.

a) EGRA. To measure different reading skills, from phonemic awareness to reading comprehension, the following EGRA standard subtasks, adapted into Spanish, were used: (1) Letter-sound Knowledge, (2) Initial Sound Identification, (3) Familiar Word Reading, (4) Non-word Reading, (5) Oral Reading Fluency (ORF), (6) Reading Comprehension.
In addition, we determined it would be valuable to make two modifications. The first was to add two inferential questions to the subtask associated with the timed ORF subtask, which normally contains five Reading Comprehension questions. The second modification was to include two additional untimed subtasks: Adaptive Oral Reading Fluency (AORF) and Adaptive Reading Comprehension. These two subtasks help to better differentiate the reading comprehension level of the sample population. The adaptation of EGRA to the linguistic context and the pilot testing of the AORF and Adaptive Reading Comprehension subtasks were conducted by MetCuantus, a psychometrics consulting firm. The adapted EGRA was piloted by QFD with 225 children in June 2015 in one preschool and three public primary schools in the State of Mexico with children enrolled in the last grade of preschool through Grade 3.

b) TVIP. To measure receptive (hearing) vocabulary acquisition, QFD administered the Pearson’s Spanish version of the Peabody Picture Vocabulary Test. Known as the Test de Vocabulario en Imagenes Peabody (TVIP) in Spanish, the test is applied in the following manner: an assessor says a stimulus word, and the child responds by pointing to one of the full-color pictures displayed on the test easel. A benefit of using the TVIP is that it can be individually administered from the age of two because it does not require reading or verbal or written responses. The TVIP used during the baseline assessment contains 125 stimulus words specific to Spanish vocabulary and norms in Mexico.

c) SURVEY. To measure reading habits and attitudes of children, QFD designed and piloted a short Reading Habits and Attitudes survey. The survey includes yes/no questions and four-Likert scale questions focused on personal and family literacy activities and behaviors. For the endline, this survey was complemented with questions focused on the implementation of the program and basic context questions.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
To evaluate the effect of the components of the “Mundo de Libros” program, the implementation sites (libraries) were divided in two groups based on parental workshops. Five libraries were selected to receive bi-monthly workshops for parents/guardians and five were selected to act as control libraries regarding the parental workshop component. Then, within each library, half of the participating children were randomly assigned to have access to the adequate-level book recommendations (using the matching algorithm) through the Internet-based platform. The other also had access to the platform but were selected to receive random recommendations (i.e. not matched to their abilities and interests). All the libraries and students have access to the same stock of 721 books and to the same web-based platform, including the loan history and book search tools. To summarize, 4 experimental groups are:
[A] Parent workshop & adequate-level book recommendations
[B] Parent workshop & random book recommendations
[C] No parent workshop & adequate-level book recommendations
[D] No Parent workshop & random book recommendations

Using this design, the effect of receiving adequate-level book recommendations will be assessed through a weighted average of the comparisons [A] to [B] and [C] to [D] groups. Similarly, the workshop component (research question 2) could be evaluated through a weighted average of the comparisons between [A] - [C] and [B] - [D]. Assignment in this case is not individual since the likelihood of contamination is higher. If complementarities between the parent workshop and the MATCH algorithm exist, it is likely that the effect of comparing [A] to [B] will be higher than that comparing [C] to [D].

Qualitative analysis and fidelity of implementation (FOI) will complement the quantitative assessment and will be used as the main strategy to evaluate potential benefits of the different components of the program.
Experimental Design Details
Randomization Method
Randomization of libraries to parental workshop condition was done in office using a computer random number generator. Within each library, children were randomly assigned to treatment (adequate-level recommendations) or control in office using coarsened exact matching. Children were matched based on two variables: (i) sex and (ii) mother's level of education. Pairs of children were formed and within each pair, one of the children was assigned treatment and the other control. Unassigned children were randomly assigned to treatment or control.
Randomization Unit
We conducted randomization at 2 different levels:
For the parental workshop treatment, the unit of randomization were libraries (cluster).
For the adequate-level book recommendations treatment, the unit of randomization within each library were children (individual).
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
10 libraries
Sample size: planned number of observations
786 children in two registration rounds* * Preliminary numbers. This will be modified in a subsequent round where we estimate a last round of close to 80 children to be added.
Sample size (or number of clusters) by treatment arms
Parent Workshop: 5 treatment libraries, 5 control libraries.
By treatment arm*:
[A] Parent workshop & adequate-level book recommendations: 235 children
[B] Parent workshop & random book recommendations: 241 children
[C] No parent workshop & adequate-level book recommendations: 158 children
[D] No Parent workshop & random book recommendations: 152 children

* Preliminary numbers. This will be modified in a subsequent round where we estimate a last round of close to 80 children to be added.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Solutions
IRB Approval Date
2015-12-13
IRB Approval Number
2015/12/27

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials