Carrot or Stick: How to improve technology adoption in Education?

Last registered on October 25, 2021

Pre-Trial

Trial Information

General Information

Title
Carrot or Stick: How to improve technology adoption in Education?
RCT ID
AEARCTR-0008047
Initial registration date
October 24, 2021
Last updated
October 25, 2021, 5:47 PM EDT

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
Fordham University

Other Primary Investigator(s)

PI Affiliation
The World Bank

Additional Trial Information

Status
In development
Start date
2021-09-01
End date
2022-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study seeks to improve demand estimation of free government textbooks in Nagaland, India by encouraging digitization of student records at the school level through the introduction of a new technology. This paper studies the impact of non monetary rewards and the threat of punishment on technology adoption. We conduct a randomized control trial to understand how public servants, here teachers, react to non-monetary incentives that seek to encourage new technology adoption and use. In the first treatment, higher authority govt. officials will provide certificates of excellence to high performing schools and teachers. In the second treatment, higher authorities contact bottom/low performers and reprimand the bottom performers for their poor work. The third arm is the control group where they are requested to digitize student records but no additional incentives are provided for doing so.
External Link(s)

Registration Citation

Citation
Mani, Subha and Meghna Sharma. 2021. "Carrot or Stick: How to improve technology adoption in Education?." AEA RCT Registry. October 25. https://doi.org/10.1257/rct.8047-1.0
Sponsors & Partners

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
Experimental Details

Interventions

Intervention(s)
The govt. of Nagaland introduced the Pocket School mobile based application in September 2021 among 474 government schools in Nagaland for digitizing student records in grades 1-5. The application allows schools to enter basic details of students, such as name, gender, date of birth, parents’ details and PCR number, within each class by a school representative, usually a teacher. An important and unique ability of this app is that it allows back-end verification of student details. PCR booklets assign a unique PCR number to each student enrolled in a government school and provides basic details of the child - child's name, date of birth, gender and parents’ names. The details in the admission register and PCR booklets available with the school can be cross verified with the digitized student records. In this way, the application aims to fix the missing step in the enrollment data collection process, that is, identifying genuine students studying in each school and grade necessary for facilitating textbook demand estimation calculations.

The PCR booklet works well for verification purposes as it is both unique and constant. The PCR number is supposed to remain the same even if the student transfers to another school within Nagaland. Therefore, these booklets serve as the gold standard for verifying the presence of a genuine student attending school.

The Pocket School app was developed by Quicksand, along with the World Bank for use by teachers. The application’s interface has been designed for easy access, usage and is readily available from the android official app store for download. This application works well on low cost android mobile devices that are widely used by teachers in Nagaland.

In this experiment, our goal is to understand the impact of non-monetary incentives in the form of rewards and punishments and their impacts on the uptake of the pocket school app, and the quantity and quality of digitization of student records.
For the experiment, we had 3 groups - one control and two treatment arms. In the control group, the Pocket school application was introduced and the schools were asked to digitize the student records and no incentives (neither rewards nor punishments) are attached to doing so.
In the two treatment arms, however, additional non-monetary incentives are tied to participation/effort levels of the schools. The intention is to understand how schools react to new technology when incentives are tied to their performance. In one treatment group, here on referred to as the rewards group, good/high performing schools will be rewarded. The best performing schools will be commended for their performance through both a letter and a personal call from high ranking officials at the Department of School Education (DSE), Nagaland, the premier government organization dealing with primary, secondary and higher education in the state. The letter will also be circulated amongst officials in Kohima and with the DEO, SDEO, AEO, JEO, EBRC team, and other officers. They will also be handed certificates of achievement and congratulatory posters, for display at schools, by officials at the smallest administrative office- Sub-divisional education office (SDEO). The teachers have close and frequent interactions with officers at the SDEO level.

Schools in the second treatment arm, here on referred to as the inquiry and action group, the non-monetary incentives work in the opposite fashion. Low performing schools will be contacted and asked to provide a reason for their poor performance. Therefore, these incentives take the form of inquiry calls and letters from the same set of officials as in the rewards group.

Our study seeks to answer which incentive program works well in urging teachers into action - performance based rewards (from high ranking officials in the government) or a threat of punishment (that takes the form of inquiry calls and letter of inquiry from high ranking officials in the government).
Intervention Start Date
2021-09-01
Intervention End Date
2021-10-28

Primary Outcomes

Primary Outcomes (end points)
Primary outcomes for impact assessment: (1) Uptake of the school pocketbook app, (2) Quantity and quality of digitization as measured by real-time enrollment figures and verifiable student records during the experiment.
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes for impact assessment: (1) Uptake of the school pocketbook app, (2) Quantity and quality of digitization as measured by real-time enrollment figures during the post-experiment period.
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
Our sample consists of 474 schools. The sample included primary, upper primary and secondary schools. Stratified randomization was performed; 3 strata groups were identified based on school categorization established by Unified District Information System for Education (UDISE) which collects real-time and credible data from all recognized and unrecognized schools in India. Within each stratum, equal number of schools were randomly allocated to each intervention arm with misfits randomly allocated to any one of the arms.
Primary research question (Q1): What is the impact of the different non-monetary incentive mechanisms (rewards and punishment) on technology adoption and the quantity and quality of digitization?
Secondary research question (Q2): What is the impact of these different incentive mechanisms in phase 2 of student digitization, post-experiment? How do schools respond to requests on completing student digitization in the post-experiment period? Do the effects of the incentives persist or fade away?
Experimental Design Details
Not available
Randomization Method
Randomization done in office by a computer in Stata
Randomization Unit
School level randomization
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
NA
Sample size: planned number of observations
474
Sample size (or number of clusters) by treatment arms
158 schools in Control,
159 schools in Treatment 1 - rewards arm
157 schools in Treatment 2 - punishments/monitoring arm
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
We can detect at least a 10 percent increase in the uptake of digital technology in the treatment group with 80 percent power assuming the following: a type I error rate of 5 percent, sample size of 300 (N=150 per treatment arm) where the uptake rate in the control group is 70 percent and standard deviation is 30 percent. Furthermore, the minimum detectable effect size for a binary outcome (with mean between 0.10 and 0.80; sd of 0.30) with 300 observations split equally between the treatment and control is about 10 percent assuming a 80 percent power and Type I error of 5 percent. Therefore as long as the intervention is successful in improving take up rates by at least 10 percent, our sample size will be sufficiently powered to detect this difference. Next, we also compute the minimum detectable effect size for the quality of data entered in the pocketbook app. We can detect at least a 13 percent increase in the percentage of high quality data entered by the treatment group with 80 percent power assuming the following: a type I error rate of 5 percent, sample size of 300 (N=150 per treatment arm) where the average high quality of data entered by the control group is 80 percent and standard deviation is 40 percent. If the standard deviation of the binary outcome is smaller, that is, 30 percent (or 20 percent) then we will be able to detect at least 10 percent (or 6 percent) difference between the treatment and the control as well.
IRB

Institutional Review Boards (IRBs)

IRB Name
Fordham University
IRB Approval Date
2021-09-24
IRB Approval Number
Protocol #1935