Applying Behavioral Science to Grant Evaluation in Local Governments

Last registered on January 28, 2022

Pre-Trial

Trial Information

General Information

Title
Applying Behavioral Science to Grant Evaluation in Local Governments
RCT ID
AEARCTR-0008826
Initial registration date
January 16, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 18, 2022, 6:33 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 28, 2022, 1:39 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Primary Investigator

Affiliation
Tohoku Gakuin University

Other Primary Investigator(s)

PI Affiliation
Yokohama City University

Additional Trial Information

Status
On going
Start date
2022-01-28
End date
2023-03-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
We examine how to nudge administrators in local governments to evaluate grant applications from nonprofit organizations from long-term perspectives, not short-term ones. Prior studies show that there are two effective ways for nonprofits to win grants: A) providing information about outcomes of service programs such as short-term and long-term outcomes, and B) providing information on service efforts such as accounting efficiency. However, evaluators are more likely to highly evaluate short-term outcomes than long-term outcomes because of their high time-discounting rate in behavioral economics.

We conduct a field-based survey experiment toward 1,916 local governments in Japan. We randomly divide the local governments into four groups. We set in the survey a hypothetical question, where the survey respondents are asked to evaluate the contents and quality of grant applications from two non-profit organizations: one with good short-term outcomes and the other with good long-term outcomes. The four groups consist of three treatment groups and one control group. The three treatment groups add to the latter nonprofit organization the information on A) accounting efficiency, B) social comparison nudge, and C) both. After the hypothetical question, we ask administrators in the local governments to rate the grant applications from the two nonprofit organizations and make a decision on which application is more suitable for acceptance. We also investigate whether the effects are heterogeneous by local governments’ and administrators’ characteristics.
External Link(s)

Registration Citation

Citation
Sasaki, Shusaku and Makoto Kuroki. 2022. "Applying Behavioral Science to Grant Evaluation in Local Governments." AEA RCT Registry. January 28. https://doi.org/10.1257/rct.8826
Experimental Details

Interventions

Intervention(s)
We randomly assign local governments to either of one control group and three treatment groups. The treatment group A provides for one nonprofit organization with good long-term outcomes the information of accounting efficiency on its service programs (accounting). The treatment group B provides the information explaining that many other local governments have already accepted similar applications which would create long-term outcomes (social comparison). The treatment group C provides both information, and the group D is the control group.
Intervention Start Date
2022-01-28
Intervention End Date
2022-02-28

Primary Outcomes

Primary Outcomes (end points)
Rating the grant applications from the two nonprofit organizations
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
1. Making a decision on which application is more suitable for acceptance
2. Subjective evaluation for outcomes and other information in the applications
3. How to refer outcome and other information in making the decision
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We randomly assign local governments to either of one control group and three treatment groups. The treatment group A provides for one nonprofit organization with good long-term outcomes the information of accounting efficiency on its service programs (accounting). The treatment group B provides the information explaining that many other local governments have already accepted similar applications which would create long-term outcomes (social comparison). The treatment group C provides both information, and the group D is the control group.
Experimental Design Details
Not available
Randomization Method
Stratified randomization based on 8 types of Japanese local governments (Prefectures; wards; ordinance-designated cities; central core cities; cities; towns; villages; ordinance-designated city wards)
Randomization Unit
Sections in charge of supporting civic activities at local governments
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
1,916 sections in charge of supporting civic activities in local governments
Sample size: planned number of observations
1,916 administrators at the sections in charge of supporting civic activities in local governments
Sample size (or number of clusters) by treatment arms
479 local governments in the control group, 479 local governments in the treatment group A, 479 local governments in the treatment group B, and 479 local governments in the treatment group C.

These are the number of mailings to randomly assigned local governments, and the sample size finally collected will be smaller than the number of these mailings.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Hakkei Campus Research Ethics Committees, Yokohama City University IRB
IRB Approval Date
2021-12-24
IRB Approval Number
Hachi2021-9