Impediments to yardstick competition in federal systems: Experimental survey evidence from German politicians

Last registered on May 26, 2020

Pre-Trial

Trial Information

General Information

Title
Impediments to yardstick competition in federal systems: Experimental survey evidence from German politicians
RCT ID
AEARCTR-0005753
Initial registration date
May 24, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
May 26, 2020, 5:02 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
Technical University of Munich

Other Primary Investigator(s)

PI Affiliation
ZEW Mannheim
PI Affiliation
ifo Institute
PI Affiliation
ZEW Mannheim

Additional Trial Information

Status
In development
Start date
2020-05-29
End date
2021-12-31
Secondary IDs
Abstract
A key argument for policy competition between sub-national entities is that direct comparison of policy outcomes yields efficient public service provision. However, the necessary condition to reap these efficiency gains – that voters can observe policy outcomes – is often not fulfilled. A case in point is the education system of Germany, where the legislative and executive power over public education is vested in the federal states, but regular cross-state comparative student assessment data is largely lacking. To study the underlying political-economy reasons for lacking comparability across states, we conduct parallel surveys among German state parliamentarians and German citizens.

To identify possible impediments to yardstick competition in the area of sub-national education policy, we analyze how important politicians consider the comparability of educational outcomes across states in comparison with citizens' opinion. Possible asymmetries in the answers are further studied by asking whether politicians undererestimate how important their electorate considers comparability. In a next step, we expect to show that politicians misperceive the relative performance of their state’s education system. Providing politicians with real performance information of their home state as compared to other states in Germany should then significantly increase the likelihood of legislators supporting the introduction of regular comparative student assessments across German states.

We elicit politicians’ preferences for comparability in a online survey targeted to all German state parlamentarians who are randomly assigned to one of two experimental groups. Respondents in group 1 state their support for comparability-enhancing policies without receiving any information. Respondents in group 2 answer the same questions after being informed about their state’s educational performance relative to the other German states, i.e. whether they are among the upper or lower half of the distribution of educational performance among all German states. Respondents in both experimental groups also can elicit whether they want to receive information on the public opinion on the matter of comparability-enhancing policies which we retrieve from a representative population survey fielded by the authors. The experimental results will shed light on the determinants of support among political legislators for enhanced comparative testing of education outcomes, which is a necessary condition to foster education-policy competition. We implemented a second, independently randomized, experiment when eliciting valuing the importance of comparability of educational outcomes across states. While a control group was not only asked about their own opinion related to this, the treatment group was asked about the average support for this from German citizens. This allows us to assess whether politicians consider voter support in their elicited support for comparability of education outcomes across German states.
External Link(s)

Registration Citation

Citation
Blesse, Sebastian et al. 2020. "Impediments to yardstick competition in federal systems: Experimental survey evidence from German politicians." AEA RCT Registry. May 26. https://doi.org/10.1257/rct.5753-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
1. We ask politicians how important they consider the comparability of educational outcomes (school level) between states. We then randomly split the sample into a treatment and control group of equal size. Politicians in the treatment group are asked how they think citizens in their state assess this issue to estimate the causal effect considering voters' opinion. The control group does not get asked about citizens' opinion.

2. We ask politicians about their preference regarding the following reform proposal: "Germany-wide standardized tests in math and German are introduced for all school types and are required every two years, starting in the 5th grade. Average results by state are published to compare student outcomes between states." Afterwards, we ask all politicans how they think citizens in their state assess this reform proposal. This is the last question for politicians using the pen&paper option for the survey (group 1). Politicians who answer the survey online are randomly assigned into a treatment (group 2) or control group (group 3) when logging in. The control group (group 3) is asked again about their preference regarding the reform porposal (after having been asked about citizens' opinion). The treatment group (group 2) is asked the same question but is given the following information treatment: "In a recent study on student performance in math, students in your state scored below (above) the German average." Confronting legislators with this state-level information for their home-state, allows us to estimate the causal effect of providing real student performance data (and considering voters' opinion). Politician respondents using the online-mode also answer the following question in order to test effects of student performance information on the willingness of information aquisition regarding public opinion on the topic: "Corresponding to our state parliament survey, we are currently surveying the German population on the same topic. In the following, we offer you that after completing the surveys you will find out what other respondents think of the reform proposal to introduce uniform student comparison tests across Germany. We will send you the selected information by email. Which of the following information would you like to receive? (Please select one of the following options) The average support of .... : all surveyed citizens in Germany, all surveyed citizens in your state, all surveyed voters of your party in Germany, all surveyed voters of your party in your state, I do not want to receive information."
Intervention Start Date
2020-05-29
Intervention End Date
2020-08-31

Primary Outcomes

Primary Outcomes (end points)
1. Legislator support for comparability of educational outcomes across states (with/without considering voters opinion).

2. Legislator support for the policy proposal to enhance comparative educational testing (with/without information on student performance).

3. Legislator demand for information about public opinion wrt comparative educational testing
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
See abstract and intervention.
Experimental Design Details
Randomization Method
Randomization done in office by a computer
Randomization Unit
To estimate the causal effect of considering voter opinions: politicians in a given state (stratified randomization).

To estimate the causal effect of providing real-world student performance data: politicians (simple randomization).
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
~650 politicians.

This number is based on an expected response rate of approx. 35% of German state parliamentarians.

The expectation is based on prior surveys of the authors among German state parliamentarians.

Sample size: planned number of observations
~650 politicians (about 35% of German state parliamentarians)
Sample size (or number of clusters) by treatment arms
Considering voter opinions: treatment and control group each 50% of the sample (~325 politicians)

Providing real-world student performance data: treatment and control group are allocated to ~33% of the sample
(the other 33% are expected to use the pen&paper version and do not get ask the respective question).


Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials