Using Phone-Based Assessments (PBA) to Provide Actionable Learning Outcomes Data to Policymakers

Last registered on May 16, 2023

Pre-Trial

Trial Information

General Information

Title
Using Phone-Based Assessments (PBA) to Provide Actionable Learning Outcomes Data to Policymakers
RCT ID
AEARCTR-0011368
Initial registration date
May 05, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 16, 2023, 12:44 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Columbia University

Other Primary Investigator(s)

Additional Trial Information

Status
In development
Start date
2023-05-01
End date
2023-06-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This research project aims to conduct a subsequent pilot of phone-based assessments (PBA) in one district of Uttar Pradesh. These PBA will be conducted with a sample of 500 students in the district of Rae Bareli.
External Link(s)

Registration Citation

Citation
Gupta, Saloni. 2023. "Using Phone-Based Assessments (PBA) to Provide Actionable Learning Outcomes Data to Policymakers ." AEA RCT Registry. May 16. https://doi.org/10.1257/rct.11368-1.0
Experimental Details

Interventions

Intervention(s)
The main focus of building foundational literacy and numeracy is during the early grades of primary schooling, i.e., up to Grade 3. These grades build the “foundational” skills for later grades. Research suggests that after Grade 3, the gaps in learning outcomes start widening, making even remediation difficult later as these gaps lead to multi-level classrooms in subsequent grades. Therefore, the state government of Uttar Pradesh wants to use phone-based assessments to achieve universal foundational literacy and numeracy by ensuring that students meet grade-level requirements by Grade 3.

The state government of Uttar Pradesh will provide a sufficiently large sample of the phone number of students, without any other personal identification information. The call center agency hired by Central Square Foundation will call 3000 phone numbers out of that sample to collect some basic demographic information and gather consent from parents for phone assessments. Finally, 500 students who make a representative sample with respect to state census data will be assessed on phone-based assessments.

These Grade 3 students will be surveyed on the phone where they will be answering questions related to literacy (in their mother tongue, Hindi) and numeracy
Intervention (Hidden)
Intervention Start Date
2023-05-01
Intervention End Date
2023-06-01

Primary Outcomes

Primary Outcomes (end points)
Literacy and Numeracy Scores of Phone-Based Assessments (in comparison to in-person assessments) across three modes (WhatsApp, SMS, Textbook)
Representativeness of student population and attrition
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
A representative sample of students selected randomly from the state student population will be assessed on phone and in-person assessments. They will also be compared to outcomes reported by assessments conducted at the local district and block levels.
Experimental Design Details
The outcomes and the comparison with existing learning outcomes data will be reported to state and district policymakers. The response of these local bureaucrats will be recorded, especially if the local assessment outcomes are inflated in comparison to assessments conducted over the phone. The goal is to understand if reporting from PBA will encourage more reliable reporting from local administrators and schools.
Randomization Method
Randomization is done using Stata Statistical Software
Randomization Unit
Student (individual randomization)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
227 strata established using DISE data on school infrastructure so that students from all kinds of schools (for example - one with electricity and no electricity or pucca wall or no pucca wall) are well-represented. This is to ensure that the outcomes reported are representative at the state-level.
Sample size: planned number of observations
500
Sample size (or number of clusters) by treatment arms
500 students, 2 students per strata
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Teachers College Columbia University Institutional Review Board
IRB Approval Date
2023-04-28
IRB Approval Number
23-295

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials