Using Phone-Based Assessments (PBA) to Provide Actionable Learning Outcomes Data to Policymakers

The main focus of building foundational literacy and numeracy is during the early grades of primary schooling, i.e., up to Grade 3. These grades build the “foundational” skills for later grades. Research suggests that after Grade 3, the gaps in learning outcomes start widening, making even remediation difficult later as these gaps lead to multi-level classrooms in subsequent grades. Therefore, the state government of Uttar Pradesh wants to use phone-based assessments to achieve universal foundational literacy and numeracy by ensuring that students meet grade-level requirements by Grade 3.

The state government of Uttar Pradesh will provide a sufficiently large sample of the phone number of students, without any other personal identification information. The call center agency hired by Central Square Foundation will call 3000 phone numbers out of that sample to collect some basic demographic information and gather consent from parents for phone assessments. Finally, 500 students who make a representative sample with respect to state census data will be assessed on phone-based assessments.

These Grade 3 students will be surveyed on the phone where they will be answering questions related to literacy (in their mother tongue, Hindi) and numeracy

2023-05-01

2023-06-01

Literacy and Numeracy Scores of Phone-Based Assessments (in comparison to in-person assessments) across three modes (WhatsApp, SMS, Textbook)
Representativeness of student population and attrition

A representative sample of students selected randomly from the state student population will be assessed on phone and in-person assessments. They will also be compared to outcomes reported by assessments conducted at the local district and block levels.

The outcomes and the comparison with existing learning outcomes data will be reported to state and district policymakers. The response of these local bureaucrats will be recorded, especially if the local assessment outcomes are inflated in comparison to assessments conducted over the phone. The goal is to understand if reporting from PBA will encourage more reliable reporting from local administrators and schools.

Randomization is done using Stata Statistical Software

Student (individual randomization)

Yes

227 strata established using DISE data on school infrastructure so that students from all kinds of schools (for example - one with electricity and no electricity or pucca wall or no pucca wall) are well-represented. This is to ensure that the outcomes reported are representative at the state-level.

500

500 students, 2 students per strata