x

We will perform our scheduled update at 10:30 AM EST on DEC 13, 2017, for 20 minutes.
Participation in Regulatory Drafting and Downstream Compliance
Last registered on October 23, 2014

Pre-Trial

Trial Information
General Information
Title
Participation in Regulatory Drafting and Downstream Compliance
RCT ID
AEARCTR-0000529
Initial registration date
October 21, 2014
Last updated
October 23, 2014 11:44 AM EDT
Location(s)
Primary Investigator
Affiliation
Duke University, Political Science
Other Primary Investigator(s)
PI Affiliation
National University of Singapore
Additional Trial Information
Status
In development
Start date
2014-10-26
End date
2015-08-31
Secondary IDs
Abstract
Our project aims to assess the current participatory process that exists in Vietnam for draft legislation and regulations, testing whether participation by businesses can improve labor rights and safety. We use an RCT to analyze the downstream compliance to an important chemical storage regulation of five randomly assigned groups of firms: 1) those simply receiving a presentation about the regulation; 2) those invited to provide comments on the draft regulation; 3) those invited, but also subsequently notified of the response of the responsible regulatory agency to their comments; 4) those not invited, but told about the participation of similar firms; and 5) a control group not invited to participate, but instead receiving a placebo. After the final regulation is promulgated, the team will monitor the firms in all five groups to test compliance with the final legislation. Because safety requirements for hazardous chemicals are readily observable, we will have clear evidence of whether participation led to downstream compliance and which of the 3 mechanisms is most responsible.

Many developing countries, including Vietnam, suffer from a challenging combination of a rapidly expanding business sector (especially small and medium sized firms) and limited government capacity to carry out its mandate of regulating the impact of businesses on their surrounding environments, employees, and communities. The consequences of the gap between these two realities are substantial, in terms of both tangible harm to members of society and a reduced legitimacy for government.

International development organizations invest millions of dollars each year into legislative participation campaigns in an effort to foster improved governance. Political scientists in the Deliberative Democracy literature, in particular, argue that participation in the drafting of new rules, through public comment periods, provides information about citizens’ needs and preferences to government. Governments can then use this information, they argue, to make laws and regulations that will enjoy wider societal compliance. Relatedly, studies in psychology and organizational behavior indicate that individuals are more likely to follow rules they have had the opportunity to influence. We propose that a similar dynamic between participation and compliance applies among businesses as well.

Testing this hypothesis in a developing country context constitutes a critical contribution to research on how participation affects policy outcomes because of the uniquely powerful influence firms have over workers and the environment. Recent newspapers headlines have documented the substantial harm caused by firms not complying with government labor, safety, and environmental regulations. In preparation for our project, Vietnam’s Ministry of Labor and Social Affairs (MOLISA) calculated that less than 10% of firms are currently in compliance with rules on worker safety, with the frequency of violations particularly large among smaller, more informal operations. Because of the limited capacity of many developing countries, greater enforcement does not seem to be a viable option; encouraging self-compliance is truly the best hope.

There are important policy implications to be drawn from better understanding of the precise mechanisms by which firm participation in rule making influence subsequent compliance. Three underlying mechanisms have been proposed by Deliberative Democracy theorists: 1) firms have a better understanding of regulations and so are less likely to make mistakes; 2) firms view the legislative process and governing institutions as more legitimate after participation; 3) the resulting legislation is of higher quality and better tailored to local preferences, thereby reducing conflict between firms and the state. These ideas have had real influence in developing countries, including our setting of Vietnam, where extensive new participation measures—such as online commenting on draft laws—have been introduced in recent years.
External Link(s)
Registration Citation
Citation
Malesky, Edmund and Markus Taussig. 2014. "Participation in Regulatory Drafting and Downstream Compliance." AEA RCT Registry. October 23. https://www.socialscienceregistry.org/trials/529/history/2983
Experimental Details
Interventions
Intervention(s)
Intervention Start Date
2014-10-31
Intervention End Date
2015-01-31
Primary Outcomes
Primary Outcomes (end points)
Compliance Measure 1: Calls to VCCI Regulatory Hotline. During the pre-treatment survey, all firms will be provided with a credit card-shaped thumb drive (USB – see images of product and planned content below) that has VCCI’s seal on the cover. The thumb drive cover will also advertise a toll-free hotline that businesses can call if they have questions about adjustments that need to be made to be considered in compliance with all new sanitary, environment, and labor safety regulations. The VCCI has agreed to host the hotline and MOLISA has agreed to provide chemical experts who can answer any detailed questions of businesses. As an initial measure of interest in compliance, we will record whether firms called the hotline for assistance, which will provide an initial intention to treat effect. Compliance Measure 2: Self-Reported Compliance. 45 days after the posting of the final regulation by MOLISA and acceptance by the Vietnamese National Assembly, our teams will return to visit all business to conduct a brief post-treatment survey (June 2015). During this 15-minute survey, firms will be asked about 3 specific requirements of the Draft Regulation on Handling and Storage of Hazardous Chemicals: 1) equipment, including gloves, masks, and safety showers; 2) storage, including proper containers and designated safety labels; 3) factory organization, including placement of storage facilities and ventilation. Firms will be asked to answer yes or no questions about whether their current processes in these 3 areas are in compliance with Final Regulation on Handling and Storage of Hazardous Chemicals. Compliance Measure 3: Observed Compliance. After the post-treatment survey at the firm’s factory location, firms will be asked to show the enumerator evidence of their compliance with the final regulation in the 3 highlighted areas (equipment, storage, and factory outlay). Assessments will only be taken of firms that volunteer to show enumerator their facilities and on particularly easy requirements. A key part of our search for the appropriate regulation has been that they include such easy to observe components and–while it is not possible to specifically identify these components until the draft regulation for public commentary is complete–MOLISA chemical experts have strongly recommended the three areas highlighted above are particularly feasible. Times will differ by factories, but we estimate observing compliance should take a maximum of twenty minutes. Only if firms agree, photographs of chemical equipment, storage, and outlay will be taken and stored with the dataset. Like the dataset, these photographs will be only identified with a firm id generated by the PIs that is only available to the PIs. The connection between firm identifying information and firm id is also only available to PIs. Further, we do not plan to publish these photographs in any publication or presentation relating to this research. Express consent would need to be sought for any such actions. Summary of Stage 3: A hotline number for complying with regulations will be provided to firms during the pre-treatment survey in September 2014 and will be live immediately. Self-reported compliance will be measured in a post-treatments survey administered to all 1800 firms that will take about 15 minutes. Direct observations of compliance will take place immediately after the post-treatment survey. Observations of compliance will be voluntary and will not take longer than 20 minutes. This is simply an assessment of enumerators and it will be made very clear that this is not a formal inspection. It is impossible to measure whether participation improved the quality of the law and therefore stimulated compliance through experimental design. The problem is that, unless firm-level preferences are highly variegated, it is unlikely that control and treatment groups will differ dramatically in their preferences over the regulation. Consequently, if the treatment groups’ ideas significantly improve the quality and acceptability of the laws, control firms are as likely to change their behavior as treatment firms. We would observe no difference between the compliance of the different groups statistically. In short, a highly successful experiment would appear as a non-result. Q1 (Quality of Regulation): To ensure that we capture this potential effect, we have designated a fifth group of 100 firms as “judges” of legal quality. These firms will also be randomly sampled from the population at the beginning of the experiment, and will match the other groups on all observable characteristics. Rather than being subject to an experimental manipulation, these firms will simply be given untitled and unmarked versions of the draft regulation and final regulation resulting from the controlled experiment during June 2014. Their primary task will be to identify which of these blind documents they think is the superior legislation. The selection should not take longer than 10 minutes. Q2 (Watered Down Regulation): To ensure that high quality law does not simply mean “watered down” regulations, we are also preparing to engage in an equivalent exercise with a group of 30 labor safety experts and knowledgeable workers. This group will also be asked to assess whether the final regulation is better than the draft in terms of its protections for workers. The selection should not take longer than 10 minutes.
Primary Outcomes (explanation)
Secondary Outcomes
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
Experimental Design
Our project aims to assess the current participatory process that exists in Vietnam for draft legislation and regulations, testing whether participation by businesses can improve labor rights and safety. We use an RCT to analyze the downstream compliance to an important chemical storage regulation of five randomly assigned groups of firms: 1) those simply receiving a presentation about the regulation; 2) those invited to provide comments on the draft regulation; 3) those invited, but also subsequently notified of the response of the responsible regulatory agency to their comments; 4) those not invited, but told about the participation of similar firms; and 5) a control group not invited to participate, but instead receiving a placebo. After the final regulation is promulgated, the team will monitor the firms in all five groups to test compliance with the final legislation. Because safety requirements for hazardous chemicals are readily observable, we will have clear evidence of whether participation led to downstream compliance and which of the 3 mechanisms is most responsible.
Experimental Design Details
Randomization Method
Randomization done by computer
Randomization Unit
Individual businesses, which use or produce hazardous chemicals. Businesses are clustered with provinces (Hanoi and surrounding provinces, and individual districts within Hanoi).
Was the treatment clustered?
Yes
Experiment Characteristics
Sample size: planned number of clusters
6 provinces (Hanoi, Bac Ninh, Hai Duong, Vinh Phuc, Phu Tho, Hung Yen), 43 districts
Sample size: planned number of observations
2000 businesses
Sample size (or number of clusters) by treatment arms
Arm 1: Placebo: 600; Information Treatment 400; Participation Treatment 800
Arm 2: Control 300; Indirect Democracy 300; No Response from Government 400; Positive Response from Government 400
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Background: In our preliminary research for this project in September 2013, we worked with the MOLISA Labor Safety officials to estimate the number of firms that would be affected by the new hazardous chemical storage regulation and the share of firms complying with the current version of the regulation, which is considered to be outdated. MOLISA has yet to complete its pre-regulation study, so the numbers they gave us our ballpark estimates. According to their current inspection records, only 8% of affected firms are complying with the existing regulations. MOLISA did not have a population size of affected firms, but did provide us with a list of the industrial sectors that would be covered by the new chemical storage regulation, including a wide range of industries from chemical producers to paint manufacturers to garment companies. Using this list, we matched these sectors to the list of registered firms in the Hanoi metropolitan area from the National Tax Authority database. This matching exercise produced 5,200 operations that would be affected by new chemical storage guidelines. The Tax Authority list also includes information on size (annual revenue), four digit ISIC code, and gender of the business owner/manager. This sample frame is preliminary, because the sectors of affected firms may shift after the final pre-regulation study is completed. Key Assumptions: 1. Current compliance rates are extremely low (8%), implying that variation in compliance is also quite low and the expected effect size of each additional treatment will be reasonably large (after all, we are measuring growth from a near zero level). 2. To keep the power calculations tractable, we conceptualize the empirical analysis as three separate tests: H1 (Information) = T1>C; H2 (Participation)= T2>T1; H3 (Legitimacy)= T3>T2. 3. We calculate our estimated treatment effects based on two sources: 1) theoretical literature on the participation-compliance relationship; and 2) observational data from the Vietnam Provincial Competitiveness Index survey on current levels of compliance with labor laws. From these, we expect the effect size for T1 to be about .22 standard deviations (or a 6 percentage point change in the level of compliance). Conservatively, we estimate the effect size of participation (T2) to be about the same (6 points). Based on the theoretical literature, we estimate the effect size of legitimacy to be twice that of information alone (.44 standard deviations or 12 percentage points). 4. Further, using observational data from we estimate that explained variation from our three blocking variables (size, narrow sector, and gender) to be about 10%. Calculation: Using the Optimal Design software to measure effects with 5% statistical significance and with 80% power, we calculate that we would need about 522 observations in the sample. To account for potential non-response and to provide some cushion, we conservatively estimate 600 observations for both the control group and T1 (See Figure 1). Repeating this exercise for the T2, we employ the same estimated treatment effect, requiring 300 additional observations (we used 400). For H3, the legitimacy treatment, the larger effect size means that we require only 133 observations to reach 80% power (See Figure 2). To be conservative, maintain simplicity, and to allow room for a study of heterogeneous effects with the blocking variables, we maintained the same 600 observations as
IRB
INSTITUTIONAL REVIEW BOARDS (IRBs)
IRB Name
NUS INSTITUTIONAL REVIEW BOARD (IRB)
IRB Approval Date
2014-08-05
IRB Approval Number
A-14-126
IRB Name
Duke University’s Institutional Review Board (IRB) for the Protection of Human Subjects in Non-medical Research
IRB Approval Date
2014-07-02
IRB Approval Number
C0469
Post-Trial
Post Trial Information
Study Withdrawal
Intervention
Is the intervention completed?
No
Is data collection complete?
Data Publication
Data Publication
Is public data available?
No
Program Files
Program Files
Reports and Papers
Preliminary Reports
Relevant Papers