Testing for ethnic discrimination within the Czech Social Security system

2019-11-11

2019-12-20

The primary outcome will be a binary variable (Yes, No) for receiving a response to the query. Second, we will measure the time to respond. Third, we will measure the courteousness and informativeness of the responses.

Courteousness and informativeness will be measured using two independent evaluators. The scale will have three levels (-1, 0, 1), zero for neutral, one for courteous/informative, minus one for non-informative/non-courteous. If the evaluators disagree on how to code a particular response, this will be flagged and evaluated by the researchers.

This is a correspondence experiment.

Because of legal constraints, neither the Census nor any administrative dataset containing ethnicity and names is available in the Czech Republic. In order to obtain names signaling ethnicity, we use a sample of surnames extracted from a convenience survey of poor families in Brno (second largest city in the Czech Republic). From these, we have selected a sample of 20 potentially Roma and typical Czech-sounding names. We have tested the ethnicity signal associated with these names at the end of a lab experiment (unrelated to this project) in which we asked the participants (mostly students of Masaryk University in Brno) to assign one of four ethnicities (Czech, Slovak, Roma, or Hungarian) to each name. For the two names most strongly associated with Roma ethnicity (Lakatoš and Gaži), 70 percent of our participants believed they belong to Roma (2-5 percent thought they are Czech and between 10 to 15 percent stated they are Slovak or Hungarian). For the two names most strongly associated with the Czech majority (Svoboda, Pospíšil), over 95 stated they belong to the Czech majority. We use these four names to signal ethnicity in our emails.

Between one to three employees in each local branch will receive our emails. Given the limited sample size (457 public servants, 2.3 from each local branch on average), our design involves between as well as within-subject variation. Each recipient will receive three emails with at least 10 days gap between them. Both ethnicity and education signals will vary between subjects in each batch of emails we send out. Within-subject, two initial emails each official will receive will differ in the ethnic signal, keeping education signal constant, the third will vary the educational signal. We will be sending our emails twice a week (on Tuesdays and Thursdays) for a total of five weeks so that each branch receives at most one email in any given day from us.

The contact information to unemployment benefit specialists was retrieved from the website of the Czech Social Security Service in July 2019 (portal.mpsv.cz, the Service recently introduced a new website, www.uradprace.cz, which also contains the contacts to individual employees). For each employee, the website states their name, rank, specialization, and contact information. We focus on employees specializing in unemployment benefits and advising the unemployed.

Computer (R version 3.6.1. functions "sample", package base, and "sample_n", package dply, set.seed(42)).

All offices with a single unemployment benefits specialist will be included. Within each local branch of the Czech Social Security Service with more than one unemployment benefits specialists, one-half of unemployment benefits specialists, but no more than three in each branch, will be randomly selected to receive our emails.

No

457 employees from 198 local offices.

457 subjects * 3 emails, giving 1371 observations.

Each unemployment benefits specialist included in this study (457 public servants from 198 local branches) will receive three emails from us. Four types of emails will be sent, with two different signals of ethnicity (Roma, Czech) and two signals of educational attainment (low education, normal education). In the first emails, each type will have a sampling probability of 0.25. The second email will differ (for each subject) in the ethnicity signal (the education signal will be kept constant). In the third email, we will change the education level (for each subject), the ethnicity signal will vary randomly between subjects.

For each subject (unemployment specialist) we have two observations with varying ethnicity (and constant literacy) and two observations with varying literacy (and constant ethnicity). McNemar’s test (paired binomial test) is therefore the relevant non-parametric test for our data. Denote p11, p10, p10, p00 the sampled probabilities that a subject responds to both Czech and Roma, only Czech, only Roma, and neither of the two ethnicities, respectively. We have p11 + p10 + p01 + p00 = 1. Let pC = p11 + p10 and pR = p11 + p01 be the overall response probabilities of receiving a response for the putative Czech and Roma senders, respectively. Finally, let \delta = pC - pR be the response differential between the two ethnicities (the discrimination effect), which after substituting yields \delta = p10 - p01. Let n be the number of subjects (paired observations), then McNemar’s test statistic is s = (p10 n - p01 n)^2 / (p10 n + p01 n) = (\delta^2 n) / (p10 + p01), which under H0:=0 asymptotically follows a chi-squared distribution with one degree of freedom. Fagerland, Lydersen, and Laake (2013) investigate Type I error frequencies and the power of alternative methods to compute the p-values. Under a wide range of parameter scenarios, the Exact unconditional McNemar test, and McNemar mid-p test, the Type I errors frequency never exceeds five percent and are almost as powerful as the asymptotic McNemar test. We, therefore, base our power calculations on the Exact unconditional McNemar test (Suissa and Shuster 1991). In our notation, the power of the test depends on three parameters, n, \delta, p01. In our case n=457 and we consider =0.05 a substantively significant discrimination coefficient (Giulietti, Tonin, and Vlassopoulos, 2019, found four percentage points differential between whites and blacks). In order to gauge p01, the baseline response rate in Giulietti et al. (2019) was 70 percent, setting our expectation for pC = 0.7 and implying a constraint p01 = 0.3 - p00. One now has to make a judgment about the actual size of p01. Responses to only Roma senders may happen for two main reasons: positive discrimination in favor of Roma enquirers by some subjects, and the fact that some subjects may respond to emails randomly. We believe that positive discrimination of Roma is not likely very frequent, but random responses may be. If we set p01 = 0.05 (randomness in the response occurs with the same frequency as discrimination), \delta = 0.05 implies p10 = 0.1. The power for the one-sided Exact unconditional McNemar test with the rejection criterion \alpha = 0.05 under the stated parameters is 0.85. If we set p01 = 0.06, the corresponding power is 0.80.

This paper tests for discriminatory treatment of the Roma minority by public officials in the Czech Republic at the stage of initial contact preceding a potential application for unemployment benefit. Our correspondence experiment facilitates testing for the presence of each of two intertwined drivers of discrimination: ethnic animus and socioeconomic status prejudice. We find substantial evidence for the presence of discrimination based on both of these sources. Since Roma tend to have lower socioeconomic status, the two sources of discrimination compound for them.

Mikula, Štěpán, and Josef Montag. 2022. Roma and Bureaucrats: A Field Experiment in the Czech Republic. MUNI ECON Working Paper No. 2022-01.

https://ideas.repec.org/p/mub/wpaper/2022-01.html