Back to History Current Version

Testing for ethnic discrimination within the Czech Social Security system

Last registered on January 04, 2022

Pre-Trial

Trial Information

General Information

Title
Testing for ethnic discrimination within the Czech Social Security system
RCT ID
AEARCTR-0004873
Initial registration date
November 08, 2019

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 08, 2019, 10:13 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
January 04, 2022, 8:45 AM EST

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region

Primary Investigator

Affiliation
Department of Economics, Faculty of Law, Charles University

Other Primary Investigator(s)

PI Affiliation
Department of Economics, Masaryk University

Additional Trial Information

Status
Completed
Start date
2019-11-11
End date
2020-01-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This correspondence experiment tests for potential discriminatory treatment by employees of the Social Security Service in the Czech Republic, specifically the public servants dealing with unemployment benefits. Our focus is on potential differential treatment of Roma minority versus the Czech majority. Roma people are one of the largest ethnic minorities in the Czech Republic (1.5-3%) as well as in the EU. According to the European Commission (2010), a significant part of 10-12 million Roma in Europe live in extreme marginalization. Differential treatment by public officials, if any, may reinforce the repercussions of existing discrimination of Roma in private markets (see e.g. Bartoš et al. 2016). At the same time, the experiment will test for potentially positive discrimination of minorities by public servants (a commonly held belief in the Czech Republic).
External Link(s)

Registration Citation

Citation
Mikula, Stepan and Josef Montag. 2022. "Testing for ethnic discrimination within the Czech Social Security system." AEA RCT Registry. January 04. https://doi.org/10.1257/rct.4873
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2019-11-11
Intervention End Date
2019-12-20

Primary Outcomes

Primary Outcomes (end points)
The primary outcome will be a binary variable (Yes, No) for receiving a response to the query. Second, we will measure the time to respond. Third, we will measure the courteousness and informativeness of the responses.
Primary Outcomes (explanation)
Courteousness and informativeness will be measured using two independent evaluators. The scale will have three levels (-1, 0, 1), zero for neutral, one for courteous/informative, minus one for non-informative/non-courteous. If the evaluators disagree on how to code a particular response, this will be flagged and evaluated by the researchers.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This is a correspondence experiment.
Experimental Design Details
Because of legal constraints, neither the Census nor any administrative dataset containing ethnicity and names is available in the Czech Republic. In order to obtain names signaling ethnicity, we use a sample of surnames extracted from a convenience survey of poor families in Brno (second largest city in the Czech Republic). From these, we have selected a sample of 20 potentially Roma and typical Czech-sounding names. We have tested the ethnicity signal associated with these names at the end of a lab experiment (unrelated to this project) in which we asked the participants (mostly students of Masaryk University in Brno) to assign one of four ethnicities (Czech, Slovak, Roma, or Hungarian) to each name. For the two names most strongly associated with Roma ethnicity (Lakatoš and Gaži), 70 percent of our participants believed they belong to Roma (2-5 percent thought they are Czech and between 10 to 15 percent stated they are Slovak or Hungarian). For the two names most strongly associated with the Czech majority (Svoboda, Pospíšil), over 95 stated they belong to the Czech majority. We use these four names to signal ethnicity in our emails.

Between one to three employees in each local branch will receive our emails. Given the limited sample size (457 public servants, 2.3 from each local branch on average), our design involves between as well as within-subject variation. Each recipient will receive three emails with at least 10 days gap between them. Both ethnicity and education signals will vary between subjects in each batch of emails we send out. Within-subject, two initial emails each official will receive will differ in the ethnic signal, keeping education signal constant, the third will vary the educational signal. We will be sending our emails twice a week (on Tuesdays and Thursdays) for a total of five weeks so that each branch receives at most one email in any given day from us.

The contact information to unemployment benefit specialists was retrieved from the website of the Czech Social Security Service in July 2019 (portal.mpsv.cz, the Service recently introduced a new website, www.uradprace.cz, which also contains the contacts to individual employees). For each employee, the website states their name, rank, specialization, and contact information. We focus on employees specializing in unemployment benefits and advising the unemployed.
Randomization Method
Computer (R version 3.6.1. functions "sample", package base, and "sample_n", package dply, set.seed(42)).
Randomization Unit
All offices with a single unemployment benefits specialist will be included. Within each local branch of the Czech Social Security Service with more than one unemployment benefits specialists, one-half of unemployment benefits specialists, but no more than three in each branch, will be randomly selected to receive our emails.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
457 employees from 198 local offices.
Sample size: planned number of observations
457 subjects * 3 emails, giving 1371 observations.
Sample size (or number of clusters) by treatment arms
Each unemployment benefits specialist included in this study (457 public servants from 198 local branches) will receive three emails from us. Four types of emails will be sent, with two different signals of ethnicity (Roma, Czech) and two signals of educational attainment (low education, normal education). In the first emails, each type will have a sampling probability of 0.25. The second email will differ (for each subject) in the ethnicity signal (the education signal will be kept constant). In the third email, we will change the education level (for each subject), the ethnicity signal will vary randomly between subjects.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
For each subject (unemployment specialist) we have two observations with varying ethnicity (and constant literacy) and two observations with varying literacy (and constant ethnicity). McNemar’s test (paired binomial test) is therefore the relevant non-parametric test for our data. Denote p11, p10, p10, p00 the sampled probabilities that a subject responds to both Czech and Roma, only Czech, only Roma, and neither of the two ethnicities, respectively. We have p11 + p10 + p01 + p00 = 1. Let pC = p11 + p10 and pR = p11 + p01 be the overall response probabilities of receiving a response for the putative Czech and Roma senders, respectively. Finally, let \delta = pC - pR be the response differential between the two ethnicities (the discrimination effect), which after substituting yields \delta = p10 - p01. Let n be the number of subjects (paired observations), then McNemar’s test statistic is s = (p10 n - p01 n)^2 / (p10 n + p01 n) = (\delta^2 n) / (p10 + p01), which under H0:=0 asymptotically follows a chi-squared distribution with one degree of freedom. Fagerland, Lydersen, and Laake (2013) investigate Type I error frequencies and the power of alternative methods to compute the p-values. Under a wide range of parameter scenarios, the Exact unconditional McNemar test, and McNemar mid-p test, the Type I errors frequency never exceeds five percent and are almost as powerful as the asymptotic McNemar test. We, therefore, base our power calculations on the Exact unconditional McNemar test (Suissa and Shuster 1991). In our notation, the power of the test depends on three parameters, n, \delta, p01. In our case n=457 and we consider =0.05 a substantively significant discrimination coefficient (Giulietti, Tonin, and Vlassopoulos, 2019, found four percentage points differential between whites and blacks). In order to gauge p01, the baseline response rate in Giulietti et al. (2019) was 70 percent, setting our expectation for pC = 0.7 and implying a constraint p01 = 0.3 - p00. One now has to make a judgment about the actual size of p01. Responses to only Roma senders may happen for two main reasons: positive discrimination in favor of Roma enquirers by some subjects, and the fact that some subjects may respond to emails randomly. We believe that positive discrimination of Roma is not likely very frequent, but random responses may be. If we set p01 = 0.05 (randomness in the response occurs with the same frequency as discrimination), \delta = 0.05 implies p10 = 0.1. The power for the one-sided Exact unconditional McNemar test with the rejection criterion \alpha = 0.05 under the stated parameters is 0.85. If we set p01 = 0.06, the corresponding power is 0.80.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
January 31, 2020, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
January 31, 2020, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
As planed: 457 subjects.
Was attrition correlated with treatment status?
Final Sample Size: Total Number of Observations
As planed: 457 subjects x 3 emails, giving 1371 observations.
Final Sample Size (or Number of Clusters) by Treatment Arms
High literacy Czech: 347 emails. Low literacy Czech: 348 emails. High literacy Roma: 351 emails. Low literacy Czech: 325 emails.
Data Publication

Data Publication

Is public data available?
Yes

Program Files

Program Files
Yes
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
This paper tests for discriminatory treatment of the Roma minority by public officials in the Czech Republic. Our focus is on public servants at local job centers whose job is to advise unemployed individuals and process applications for unemployment benefit. Our experimental design facilitates testing for the presence of each of two key (but intertwined) drivers of discrimination: ethnic animus and socioeconomic status prejudice. We find substantial evidence for the presence of discrimination based on both of these sources. Since Roma tend to have lower socioeconomic status, the two sources of discrimination compound for them.
Citation
PapersMikula, Štˇepán, and Josef Montag. 2021. Roma and Bureaucrats: A Field Experiment in the CzechRepublic. Department of Economics, Faculty of Law, Charles University. Online at https://ssrn.com/abstract=3805631.

Reports & Other Materials