Investigating cognitive, non-cognitive and behavioral mediators for building literacy through edutainment in Kenya

The Nuzo and Namia show, developed by Ubongo, targeted 6-9-year-old children with the aim of improving their literacy, social-emotional learning, and gender attitudes. The plot revolves around 2 protagonists, Nuzo and Namia, as they meet with different African characters at different moments throughout history. By interacting with the protagonists, who appear just in time to help them solve a problem that they are facing, viewers see and hear the protagonists model key strategies in early reading comprehension, such as predicting, questioning, clarifying, and analyzing information. For example, a character may predict a book's content based on the story's cover and title. The audience can then later see them refine their predictions halfway through the story or when they start reading. They may also compare texts they have read in different parts of the book and then self-question their comprehension.

Furthermore, seeing protagonists who are just like the audience (i.e., relatable role models) reading and using stories/books to practise their reading aims to give children agency, confidence, and motivation to improve and practise their own reading skills. In interactions with the historical figures in the show, along with their family and friends, viewers witness the protagonists identify and model positive social-emotional learning outcomes and gender attitudes. The protagonists are taken on adventures where they can try things that earlier may have seemed unsuitable or impossible beforehand. Through the open-mindedness and perseverance of the protagonists, viewers learn that by trying different things and sticking to them for a while, they can find activities that truly suit them and their skills, even if societal norms or early difficulties don’t encourage it. The series aims to both “show” these character traits (like confidence and curiosity) and explicitly state how these helped the protagonists succeed (e.g., a protagonist saying, “I am really glad I didn’t give up when XY happened!”). The programme consists of 13 22-minute episodes.

Overview:
We design a lab-in-the-field RCT in 3 communities with public schools in Nairobi. We have 3 groups in the study: placebo/ control and 2 treatment groups that receive variations of the intervention. Each group or treatment arm within the group will be seated in a room with a smart TV, where the show will be broadcast. We envision six-episode viewings over six weeks.

Treatment Groups:
Group 1: Placebo Control
This group will watch a placebo show content and not be exposed to the Nuzo and Namia treatment. At this moment, we propose showing the children a wildlife documentary to ensure no similarities in programming in literacy, SEL, or gender attitudes.
This group represents a control group against which we can compare results from the treatment groups to extract the impact on the sub-skills developed by the show.
Group 2: Receives the N&N show in Kiswahili without caregiver intervention
No iterations. This group watches the Kiswahili version of the Nuzo and Namia show.
This group helps us decode the cross-linguistic transfer component compared to the other treatment arms. It would be interesting to see if these children demonstrate skills transfer from Kiswahili to English or their mother tongue and whether we should expect that to happen to the original study sample.
Group 3: Receives N&N in Kiswahili) with salient caregiver engagement nudges
This group watches in Kiswahili and receives some variations in the treatment, with caregiver engagement as the primary component.
Narration, a recap at the end of the episode, and a short demonstration of building conversational literacy in the home while watching the show can drive caregiver engagement.
Any other mechanism-related design will be incorporated during the research conceptualization phase.
We can decode the aspect of cross-linguistic transfer from this group by using the relative literacy scores between languages but controlling for absolute level differences between groups. We can also compare the results of having received caregiver intervention (absolute level differences).


Caregiver Intervention:
The following aspects of caregiver intervention will be tested in the third treatment arm:
Engaging in meaningful conversations: Research shows that engaging in meaningful conversations with a caregiver leads to language and literacy development, especially in the earlier stages of life. We aim to test whether this element is salient in the Nuzo and Namia show as well. We plan to include an element of a “recap” where the characters demonstrate doing a recall exercise with their parents on the show, at the end of each episode, and encourage children and caregivers watching the show to also talk to each other and recapitulate what the child did that day.
Teaching caregivers simple and easy-to-implement literacy development techniques: We would like to test whether equipping a caregiver with the skills to engage in simple literacy building activities with their child lead to better reading behaviors amongst children. An example of this would be to encourage them to read or tell a bedtime story with their children, demonstrate ways in which you can plan for richer conversations with your child on a busy day, and incorporate object-word association games with their children, amongst others.

Book and Print Recognition:
Developmental literature suggests that reading habits are improved when a child is aware of the functions of a book and how to navigate reading experiences. In Group 3 of the treatment group, we would like to place different types of books in the lab room, including picture books, books with heavy texts, books of different languages, and some supplementary reading material aligned to the Nuzo and Namia show, to measure whether the children show curiosity towards different books and are able to describe the functions of the books.

2024-10-01

2024-11-17

We explore the following primary research questions:
1. Does the show facilitate cross linguistic transfer (CLT) of reading skills? Since the program exists in a language agnostic space, are we on track to build literacy in the long-term by anticipating a transfer of skills across languages?
2. How does caregiver engagement through targeted dual-generation programming modulate variations in programme impact?
3. How closely do curiosity and executive functioning skills relate to literacy development, following the Active View of Reading Model (Duke and Cartwright, 2021)?

1. Reading ability
To assess one primary summary measure for latent reading and listening comprehension skills, which are the primary teaching objective of Nuzo & Namia, we perform Item Response Theory (IRT) analysis on all items included in:
i. EGRA listening and reading comprehension sections
ii. EGRA adapted subtasks on object word association and spatial recognition

IRT models the relationship between individuals' latent trait (e.g., ability) and their performance on observed items. We compute this separately for each of the three languages tested in the survey: English, Kiswahili, and Luyo.

2. Confidence and Curiosity score (developed by RTI)-
Combined score: 8 items coded from 0 to 2. We follow Anderson (2008) to create an index which has a mean of zero and standard deviation of one, and is standardized relative to the control group. Given that the combined score mixes two constructs, we will furthermore conduct a principal component analysis as a more data-driven approach and as a robustness check.
Confidence score: 4 sub-items coded from 0 to 2. Similar to the combined score we will create an index following Anderson (2008).
Curiosity score: 4 sub-items coded from 0 to 2. Similar to the combined score we will create an index following Anderson (2008).

3. Summary measure: Behavioral index
As primary measure in this family, we will conduct principal component analysis (PCA) on the following likert-scale measures, after having standardized each item.
IB1: How often do you read outside of school, i.e. for enjoyment?
IB2: How often do you look for or ask for books for enjoyment and not school work ?
IB3: How often does your mum, dad or carer read books or stories to you?

4. Information and Skill (Book descriptives and Executive Function)

In this family, we construct 6 primary measures.

Primary measures:
I. Executive Function (comprising of):
a. Inhibitory Control
b. Working Memory
c. Cognitive Flexibility & Creativity- Alternative uses task measure: Sum of eligible alternative uses the child could think of
Functions of a book

II. Book and Print Recognition measure: Sum of correctly identified book elements and functions

5. Social Interactions
I. Social interaction index: We construct an index using principal component analysis (PCA) on the following items, measured on a 4-point likert scale. We standardize all items before conducting PCA:
a. "How often do you meet with friends to read (for enjoyment, not school books)?
b. "How often do you ask friends for help if you have a problem to solve (problem related to studies or school)?
c. "How often do you ask your parents for help if you have a problem to solve (related to studies or school)?
d. "How often do you talk to your friends about your feelings?
e. "How often do you talk to your parents about your feelings?
II. Parent conversational interaction: We construct an index using principal component analysis (PCA) on the following items
a. Talk to you while doing an activity w/ you around
b. Describe things to you when walking
c. Pointed, named object and asked you to repeat
d. Read/ looked at a book with you
e. Told you a story
III. General adult conversational interaction: We construct an index using principal component analysis (PCA) on the following items
a. Sing to you
b. Play with you
c. Read/ looked at a book with you
d. Told you a story
e. Described things to you

Overview:
We design a lab-in-the-field RCT in 3 communities with public schools in Nairobi. We have 3 groups in the study: placebo/ control and 2 treatment groups that receive variations of the intervention. Each group or treatment arm within the group will be seated in a room with a smart TV, where the show will be broadcast. We envision six-episode viewings over six weeks.

Treatment Groups:
Group 1: Placebo Control
This group will watch a placebo show content and not be exposed to the Nuzo and Namia treatment. At this moment, we propose showing the children a wildlife documentary to ensure no similarities in programming in literacy, SEL, or gender attitudes.
This group represents a control group against which we can compare results from the treatment groups to extract the impact on the sub-skills developed by the show.
Group 2: Receives the N&N show in Kiswahili without caregiver intervention
No iterations. This group watches the Kiswahili version of the Nuzo and Namia show.
This group helps us decode the cross-linguistic transfer component compared to the other treatment arms. It would be interesting to see if these children demonstrate skills transfer from Kiswahili to English or their mother tongue and whether we should expect that to happen to the original study sample.
Group 3: Receives N&N in Kiswahili) with salient caregiver engagement nudges
This group watches in Kiswahili and receives some variations in the treatment, with caregiver engagement as the primary component.
Narration, a recap at the end of the episode, and a short demonstration of building conversational literacy in the home while watching the show can drive caregiver engagement.
Any other mechanism-related design will be incorporated during the research conceptualization phase.
We can decode the aspect of cross-linguistic transfer from this group by using the relative literacy scores between languages but controlling for absolute level differences between groups. We can also compare the results of having received caregiver intervention (absolute level differences).

Randomized in office by a computer using R as the statistical software.

Randomized at the household level

No

150 households.

300 observations with 150 children and 150 caregivers.

50 households in control, 50 households in treatment arm 1 with exposure only to the Nuzo and Namia show, 50 households in treatment arm 2 with exposure to Nuzo and Namia and caregiver intervention.

Pre-Analysis Plan (PAP): Investigating cognitive, non-cognitive and behavioral mediators for building literacy through edutainment in Kenya