Back to History Current Version

Raskin Welfare Reform: Transition to Electronic Distributions

Last registered on April 02, 2021

Pre-Trial

Trial Information

General Information

Title
Raskin Welfare Reform: Transition to Electronic Distributions
RCT ID
AEARCTR-0004675
Initial registration date
September 13, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 17, 2019, 9:43 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
April 02, 2021, 9:19 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
MIT

Other Primary Investigator(s)

PI Affiliation
Harvard
PI Affiliation
MIT
PI Affiliation
TNP2K
PI Affiliation
TNP2K

Additional Trial Information

Status
On going
Start date
2017-03-01
End date
2021-12-31
Secondary IDs
7200AA19FA00002
Abstract
Raskin is a national program by the Government of Indonesia (GoI) that provides subsidized rice to the poor. It is the largest social assistance program in Indonesia and the first designated to transition from in-kind delivery to electronic distribution. The study aims to test two aspects of this transition: (1) whether the transition to electronic distribution improves program effectiveness and (2) how many electronic distribution agents are needed to provide effective program delivery to beneficiaries. The project is being conducted at the district level and comprises two components, involving a total of 278 districts. The first component is a randomized impact evaluation of the transition to electronic distribution among 105 districts deemed ready for transition in 2018. The second is a randomized control trial in 216 post-transition districts, testing two different sets of criteria for establishing the minimum level of electronic distribution agent coverage available to beneficiaries.
External Link(s)

Registration Citation

Citation
Banerjee, Abhijit et al. 2021. "Raskin Welfare Reform: Transition to Electronic Distributions." AEA RCT Registry. April 02. https://doi.org/10.1257/rct.4675-2.2
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The interventions for the two project components are as follows:
(1) Impact evaluation: Transitioning the delivery of social assistance from in-kind rice assistance (i.e. Raskin) to electronic methods (i.e. BPNT)
(2) Agent ratio experiment: Varying the coverage requirements for electronic distribution agents that banks participating in BPNT delivery are asked to follow
Intervention Start Date
2018-04-01
Intervention End Date
2021-03-01

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes for the two project components are as follows:
(1) Impact evaluation: Total value of subsidy received, quality of rice, food security indicators
(2) Agent ratio experiment: Prices received at agent, quality of rice
Primary Outcomes (explanation)
Our outcome data will come from SUSENAS, Indonesia's National Socioeconomic Survey.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
(1) Impact evaluation: We worked with the Government of Indonesia to randomly select districts that would receive BPNT as part of the 2018 rollout. The rollout took place in four phases, with our treatment districts undergoing the transition in phases 2-4.

(2) Agent ratio experiment: We worked with the Government of Indonesia to randomly assign districts that had already transitioned to BPNT (including treatment group districts in the impact evaluation experiment) into two experimental groups of equal size (Group A and Group B). In Group A, the Government of Indonesia is requiring banks to fulfill a stricter set of agent coverage requirements than in Group B.
Experimental Design Details
(1) Impact evaluation:
To assess the welfare impact of the reform from in-kind to electronic delivery of food assistance benefits, we are working with the GoI to conduct a randomized evaluation to compare the delivery of social assistance using electronic methods (i.e. BPNT) to the status quo of in-kind rice assistance (i.e. Raskin). This randomized evaluation is only possible due to the government’s plan to roll out BPNT in phases, transitioning only some districts in 2018. To facilitate this evaluation, the Indonesian government asked the research team for assistance to randomly select districts to be treated in the 2018 expansion of BPNT from a list of districts that were potentially ready for the reform, under the condition that the total number of beneficiaries in the selected districts amounted to around 8.3 million (roughly equivalent to the budgetary allocation for 2018 for the reform process for this experimental sample).

The experimental sample, thus, involves a total of 105 districts, with 42 districts receiving BPNT serving as our treatment group and 63 receiving districts receiving Raskin serving as our control group. Due to logistical constraints, the GoI additionally decided to implement this expansion in four phases. We additionally randomized which districts were assigned to which phase within our experimental sample.

Note that in addition to our experimental districts, there are two other types of non-experimental reform districts. First, there are the original 44 cities that were part of the GoI’s 2017 pilot. Second, the GoI also selected certain districts to receive the reforms in 2018 regardless of the experimental design of this study. While these districts are not part of our experimental sample, we will collect data on them when possible and will descriptively understand the reform process in these districts.

(2) Agent ratio experiment:
In this study, we will test whether changes to the agent to beneficiary ratio change the effectiveness of the BPNT program. For example, we will assess whether increasing agent coverage in a district makes it easier for households to access the BPNT benefit, lowers the price of the rice beneficiaries receive, improves rice quality, etc.

As with the first experimental component, our unit of analysis is the district. Working with the Government of Indonesia, we identified 216 districts where the BPNT reforms have been carried out. These 216 districts include both districts in our experimental sample for Experiment 1 and districts the non-experimental sample (i.e. the 44 pilot cities from 2017, districts designated to reform in 2018 regardless of experimental design, etc.). We then worked with the GoI to randomly assign the districts into two experimental groups of equal size (Group A and Group B). In Group A, the Government of Indonesia is requiring banks to fulfill a stricter set of agent coverage requirements that are specified by the government. These Group A requirements are: 1) an agent to beneficiary ratio of at least 1: 250 and 2) a minimum of 2 agents per village. In the remaining districts (Group B), we loosened the agent coverage criteria. The banks in Group B are required to fulfill the first criterion only, which is the agent to beneficiary ratio of at least 1: 250.

By varying the requirements that the banks need to fulfill in different districts, we will be able to (1) answer the policy question posed by the government of whether the additional criterion of a minimum 2 agents per village is necessary and (2) shed light on the optimal ratio that should be considered by the government as a future standard to ensure policy success. The banks have agreed to participate in the study, and we are closely coordinating with them on the study implementation process, particularly on the sharing of updated administrative data on agents.

To introduce the agent coverage criteria for this experiment, our GoI partners convened initial discussions with the banks. Through close coordination with the GoI, we have also sent enforcement letters highlighting the agent coverage requirements to the 216 districts in the agent ratio experiment. Furthermore, we have conducted a round of phone surveys with local government representatives and local bank officials in both experimental areas (278 districts across Experiments 1 and 2). These phone surveys serve the dual purpose of reinforcing the agent coverage requirements in Experiment 2 (through survey scripts that reference the applicable criteria) and providing valuable feedback on the implementation of both experiments, as well as the on-the-ground challenges associated with this historic reform.
Randomization Method
Computer random number generator
Randomization Unit
District
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
(1) Impact evaluation: 105 districts
(2) Agent ratio experiment: 216 districts
Combined: 278 districts
Sample size: planned number of observations
(1) Impact evaluation: 65,000 households (approx.) (2) Agent ratio experiment: 147,000 households (approx.) Combined: 180,000 households (approx.)
Sample size (or number of clusters) by treatment arms
(1) Impact evaluation: 42 districts BPNT (treatment), 63 districts Raskin (control)
(2) Agent ratio experiment: 108 districts Group A, 108 districts Group B
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
MIT COUHES
IRB Approval Date
2017-02-06
IRB Approval Number
1701819226
Analysis Plan

Analysis Plan Documents

Raskin Transition Program Evaluation Analysis Plan

MD5: 01a6e888d008dec80e13882b66975303

SHA1: 18c940b7ac0767596b3061b8c13cfd8033fdc6a6

Uploaded At: March 20, 2020

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials