Applying Behavioral Science to Grant Evaluation in Local Governments

Last registered on January 28, 2022

View Trial History

Pre-Trial

Trial Information

General Information

Title

Applying Behavioral Science to Grant Evaluation in Local Governments

RCT ID

AEARCTR-0008826

Initial registration date

January 16, 2022

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published

January 18, 2022, 6:33 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated

January 28, 2022, 1:39 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Country

Japan

Region

Primary Investigator

Name

Shusaku Sasaki

Affiliation

Osaka University

Contact Primary Investigator

Other Primary Investigator(s)

PI Name

Makoto Kuroki

PI Affiliation

Yokohama City University

Contact Investigator

Additional Trial Information

Status

On going

Start date

2022-01-28

End date

2023-03-31

Keywords

Agriculture, Behavior, Labor, Welfare

Additional Keywords

Nonprofit organizations, Charities, Grant Evaluation, Local Government, Accounting Information, Nudge, Social Comparison, Behavioral Economics, Survey Experiment, Japan

JEL code(s)

Secondary IDs

Prior work

This trial does not extend or rely on any prior RCTs.

Abstract

We examine how to nudge administrators in local governments to evaluate grant applications from nonprofit organizations from long-term perspectives, not short-term ones. Prior studies show that there are two effective ways for nonprofits to win grants: A) providing information about outcomes of service programs such as short-term and long-term outcomes, and B) providing information on service efforts such as accounting efficiency. However, evaluators are more likely to highly evaluate short-term outcomes than long-term outcomes because of their high time-discounting rate in behavioral economics.

We conduct a field-based survey experiment toward 1,916 local governments in Japan. We randomly divide the local governments into four groups. We set in the survey a hypothetical question, where the survey respondents are asked to evaluate the contents and quality of grant applications from two non-profit organizations: one with good short-term outcomes and the other with good long-term outcomes. The four groups consist of three treatment groups and one control group. The three treatment groups add to the latter nonprofit organization the information on A) accounting efficiency, B) social comparison nudge, and C) both. After the hypothetical question, we ask administrators in the local governments to rate the grant applications from the two nonprofit organizations and make a decision on which application is more suitable for acceptance. We also investigate whether the effects are heterogeneous by local governments’ and administrators’ characteristics.

External Link(s)

Registration Citation

Citation

Sasaki, Shusaku and Makoto Kuroki. 2022. "Applying Behavioral Science to Grant Evaluation in Local Governments." AEA RCT Registry. January 28. https://doi.org/10.1257/rct.8826-1.1

Sponsors & Partners

Experimental Details

Interventions

Intervention(s)

We randomly assign local governments to either of one control group and three treatment groups. The treatment group A provides for one nonprofit organization with good long-term outcomes the information of accounting efficiency on its service programs (accounting). The treatment group B provides the information explaining that many other local governments have already accepted similar applications which would create long-term outcomes (social comparison). The treatment group C provides both information, and the group D is the control group.

Intervention Start Date

2022-01-28

Intervention End Date

2022-02-28

Primary Outcomes

Primary Outcomes (end points)

Rating the grant applications from the two nonprofit organizations

Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)

1. Making a decision on which application is more suitable for acceptance
2. Subjective evaluation for outcomes and other information in the applications
3. How to refer outcome and other information in making the decision

Secondary Outcomes (explanation)

Experimental Design

Experimental Design Details

In January and February of 2022, we conduct a survey experiment with the following details: We send a questionnaire survey including the experimental part to 8 types of 1,916 local governments (Prefectures; wards; ordinance-designated cities; central core cities; cities; towns; villages; ordinance-designated city wards). We randomly divide the local governments into one control group and three treatment groups within each type. In the survey, we set a hypothetical question, where the survey respondents are asked to evaluate the contents and quality of grant applications from two non-profit organizations: one with good short-term outcomes and the other with good long-term ones. In addition to the common explanations about the two nonprofit organizations, the three treatment groups add to the latter nonprofit organization the following messages: “In previous activities, the organization efficiently used the acquired budget, by spending almost all the budget directly to main program services and minimizing administrative costs.” (Group A), “Many other local governments, including some similar governments in the neighborhood, have already adopted the program services and achieved better outcomes.” (Group B), and both information (Group C). We also set several questions to capture the local governments’ and administrators’ characteristics.

Randomization Method

Stratified randomization based on 8 types of Japanese local governments (Prefectures; wards; ordinance-designated cities; central core cities; cities; towns; villages; ordinance-designated city wards)

Randomization Unit

Sections in charge of supporting civic activities at local governments

Was the treatment clustered?

Yes

Experiment Characteristics

Sample size: planned number of clusters

1,916 sections in charge of supporting civic activities in local governments

Sample size: planned number of observations

1,916 administrators at the sections in charge of supporting civic activities in local governments

Sample size (or number of clusters) by treatment arms

479 local governments in the control group, 479 local governments in the treatment group A, 479 local governments in the treatment group B, and 479 local governments in the treatment group C.

These are the number of mailings to randomly assigned local governments, and the sample size finally collected will be smaller than the number of these mailings.

Minimum detectable effect size for main outcomes (accounting for sample design and clustering)

Supporting Documents and Materials

IRB