Third party evaluation in conservation auctions

Last registered on September 08, 2025

Pre-Trial

Trial Information

General Information

Title
Third party evaluation in conservation auctions
RCT ID
AEARCTR-0016584
Initial registration date
September 02, 2025

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 08, 2025, 7:37 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
University of Cambridge

Other Primary Investigator(s)

PI Affiliation
Purdue University
PI Affiliation
Xi'an Jiaotong University
PI Affiliation
University of Glasgow

Additional Trial Information

Status
In development
Start date
2025-09-10
End date
2026-03-10
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
The past few decades have seen extensive amounts of budgets spent on the procurement of land-based environmental goods, as in public schemes such as the American Conservation Reserve Program and the Chinese counterpart, the Sloping Land Conversion Programme, as well as in private schemes such as the UK’s biodiversity offsetting market. These schemes typically pay private landholders for the provision of environmental benefits through changing land use practices such as restoring natural vegetation on rural land. The payments are often conditional on a third party’s evaluation of certain indicators or proxies of the environmental goods being delivered, where the third party’s position could be influenced if they are commissioned by private landholders seeking a favourable evaluation outcome to outcompete other suppliers of the environmental goods. Through a framed field experiment using Chinese farmer subjects, this study will for the first time investigate whether such environmental procurement may have different levels of conservation efficacy depending on the commission fees for third-party evaluators and the competition among them. Each experimental session will consist of four farmer subjects as ‘bidders’, each of whom will propose a rural land greening project in competition for a reward, and another two farmer participants as ‘referees’ evaluating the proposed projects at an evaluation fee paid by each bidder out of an endowment allocated to them by an experimenter. We will have two treatments that differ in the bidders’ flexibility in deciding the amount of the evaluation fee, which will allow us to examine the implications for auction performance. Moreover, we will have another treatment that mitigates the competition between the two referees, to assess whether the impacts of evaluation fees are less pronounced when the referees become less competitive. This study will make a unique contribution to the literature on conservation auctions and more generally to the literature on bribery and corruption in competitive procurement.
External Link(s)

Registration Citation

Citation
Cason, Timothy N. et al. 2025. "Third party evaluation in conservation auctions." AEA RCT Registry. September 08. https://doi.org/10.1257/rct.16584-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2025-09-10
Intervention End Date
2026-03-10

Primary Outcomes

Primary Outcomes (end points)
The total environmental benefit index of rewarded projects
Primary Outcomes (explanation)
The environmental benefits of a greening project will be evaluated using three criteria (which will be announced to all the referees and bidders). Other conditions being equal, a greening project has higher environmental benefits if:

1) it increases vegetation cover to a greater extent;

2) it has richer vegetation species; or

3) the slope of the land is higher (assuming that vegetation cover on sloped land is more helpful in reducing soil erosion, relative to vegetation cover on flatter land).

The pictures representing the greening projects are designed to be evidently discernible in whether each of the three criteria is met or not. We will define three binary variables with pre-determined values that indicate whether a project meets each of the three criteria or not. The sum of the three binary variables of the two rewarded projects constitutes the total environmental benefit index mentioned above.

We will test the following hypotheses about the primary outcome:

- Hypothesis 1a. Rewarded projects have a lower total environmental benefit index when different bidders in one auction are allowed to pay different amounts of money to the same referee, compared to when different bidders are restricted to pay the same amount of money to the same referee.

- Hypothesis 2a. Rewarded projects have a lower total environmental benefit index when each bidder is allowed to select their own referee (where the referees compete for more bidders and hence more evaluation fees), compared to when each bidder is randomly assigned a referee (where the referees do not have the scope to compete for more bidders).

- Hypothesis 3a. Rewarded projects have a lower total environmental benefit index when bidders with lower-quality greening projects are better endowed to pay higher evaluation fees, compared to when bidders with higher-quality greening projects are better endowed.

We will test these hypotheses using the data from the last four auction periods only.

Secondary Outcomes

Secondary Outcomes (end points)
Total evaluation fees
Secondary Outcomes (explanation)
This outcome variable refers to the total evaluation fees paid by the four bidders to the two referees, which captures an auction’s distributional performance.

We will test the following hypotheses about the secondary outcome:

- Hypothesis 1b. Bidders pay higher total evaluation fees when different bidders in one auction are allowed to pay different amounts of money to the same referee, compared to when different bidders are restricted to pay the same amount of money to the same referee.

- Hypothesis 2b. Allowing each bidder to select their own referee (where the referees compete for more bidders and hence more evaluation fees) impacts the total evaluation fees, compared to randomly assigning a referee to each bidder (where the referees do not have the scope to compete for more bidders).

- Hypothesis 3b. When bidders with lower-quality greening projects are better endowed to pay higher evaluation fees, the total evaluation fees are different from when bidders with higher-quality greening projects are better endowed.

We will test these hypotheses using the data from the last four auction periods only.

Experimental Design

Experimental Design
This is a hypothetical auction game which will be conducted in framed-field-experiment settings with Chinese farmer subjects recruited in Shaanxi province. The results of the auction game will determine participating farmers’ real payoffs from attending the game but have no implications for their real land use practices. Each auction group will consist of four farmer subjects as ‘bidders’, each of whom will propose a rural land greening project in competition for a fixed-size reward (RMB 20), and another two farmer subjects as ‘referees’ evaluating the proposed projects at an evaluation fee paid by each bidder out of an endowment allocated to them by an experimenter. The two projects with the best evaluation results will be rewarded. Each auction group will undertake a total of six auction periods, where each period is an independent one-shot auction that re-assigns the bidders’ endowments and greening projects (represented by pictures showing land cover before and after a project), and determines which projects are rewarded and all the participants’ net payoffs in this period. Each auction group will be randomly assigned to one of the three between-group treatments mentioned above in the Intervention Section, which will allow us to assess whether auction performance is impacted if different bidders are allowed to pay different amounts of evaluation fees to the same referee, and whether such impacts depend on the competitiveness between the two referees. Moreover, each auction period will be randomly assigned to one of the three within-group treatments, which differ in the correlation between the environmental benefits of a greening project and whether a bidder is better positioned to pay a higher evaluation fee.
Experimental Design Details
Not available
Randomization Method
Each auction group will be randomly assigned to one of the three between-group treatments (T1–T3). The randomisation will be performed in advance using a random number generator on a computer, making sure that the three treatments have the same number of auction groups.

Each auction group will undertake a total of six auction periods (six independent auctions), and each period will be randomly assigned to one of the three within-group treatments (W1–W3), where each within-group treatment has two sets of greening pictures and endowments which will be assigned to two auction periods (and so in total there are six sets of greening pictures and endowments). This is equivalent to randomly reordering the six sets of greening pictures and endowments for different auction groups, which will be performed in advance using a random number generator on a computer. This randomisation procedure will be performed only once for the auction groups in one between-group treatment, and the same assignment will be adopted by the other two between-group treatments. Another condition of this randomisation procedure is to ensure that within one between-group treatment, the last four auction periods in total (over all the auction groups) have a balanced assignment of W1, W2 and W3, because each of the three treatments has a specific type of correlation between the environmental benefits of a greening project and whether a bidder is better positioned to pay a higher evaluation fee, and we will test the main hypotheses using the data from the last four auction periods only.
Randomization Unit
Auction group
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
54 auction groups in total
Note: the between-group treatments are not clustered – both the randomisation and the main data analyses will be conducted at the auction group level.
Sample size: planned number of observations
54 auction groups in total
Sample size (or number of clusters) by treatment arms
18 groups in each between-group treatment (T1, T2 or T3)
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
The effect sizes and standard deviations were obtained from our pilot sessions under treatments T1 and T2. For the primary outcome (the total environmental benefit index of rewarded projects): mean: 4.42 (T1), 3.83 (T2); standard deviation: 0.14 (T1), 0.52 (T2). The statistical power of the planned sample size is 89% (rank-sum test) and 92% (t-test) when testing an effect size of 10% of the mean of T1, using the standard deviations of T1 and T2, respectively, at a two-tailed significance level of 5%. For the secondary outcome (total evaluation fees): mean: 15.42 (T1), 21.25 (T2); standard deviation: 0.52 (T1), 3.93 (T2). The statistical power of the planned sample size is 84% (rank-sum test) and 90% (t-test) when testing an effect size of 20% of the mean of T1, using the standard deviations of T1 and T2, respectively, at a two-tailed significance level of 5%. These powers were simulated in Stata with 1,000 replications as per Campos-Mercade (2024).
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethics Committee, Xi’an Jiaotong University
IRB Approval Date
2025-07-24
IRB Approval Number
2025-2918
IRB Name
Research Committee, Department of Land Economy, University of Cambridge
IRB Approval Date
2025-08-29
IRB Approval Number
N/A