Participation in Regulatory Drafting and Downstream Compliance

The RCT will take place in 4 stages: 1) Initial Participation and Information Treatments; 2) Follow-up Treatments; 3) Post-Treatment Survey and Compliance Assessment; and 4) Quality of Law Analysis.

1.) Initial Participation and Information Treatments (August/September 2014)
The drafting committee of MOLISA plans will provide us with a copy of the Draft Regulation at the end of August 2014, which will allow us to finalize our research instruments (pre-treatments survey and presentation), as these must take into account the specific details of the regulation. One week later, MOLISA will post the regulation online for public comment for sixty days, as is required by the Law on Laws. At that time (end of August/early Sept 2014), we will roll-out the first stage of our experiment, which includes the information and participation treatments.

Pre-Treatment Survey: All 1,800 firms will participate in a short 10-15 minute pre-treatment survey prior to our intervention. The 4 main purposes of the pre-treatments survey are to: 1) assess the representativeness of our sample with the larger population of private businesses in Vietnam; 2) collect baseline data of our sample on interactions with government and regulatory compliance; 3) assemble a database of variables that can be used to assess heterogenous treatment effects (i.e. size, sector, performance); and 4) record baseline levels of regulatory knowledge and feelings about confidence in government and government processes. Please note that none of these questions are considered sensitive and all of them have been previously asked on VCCI’s annual Provincial Competitiveness Index Survey. The pre-treatment survey instrument is at Appendix 6.

The survey will be administered using a tablet computer, so the respondent will be able to read a version of the draft regulation, record their opinions, and submit them immediately. Vietnam has a 93% literacy rate, and over 60% of managers have college degrees, so comprehension should not be a problem in this setting. Currently, we are working to design our tablet interface in a manner that is visually and technologically compatible with VCCI’s current platform. Our tablet will be fitted with OKD web-based survey software, and will be designed to collect data on sample firms for both stages of company visits, which will store encrypted data until it is uploaded to a cloud-based database. This will both maximize data security and allow us an additional means of oversight of the research teams, since all uploads will come with a time and GPS location stamp.

After the pre-treatment survey, the 1,800 firms will be randomly assigned to 3 groups:
- T1 (Information, n=400): The first group treatment group will receive a presentation on the Draft Regulation on Handling and Storage of Hazardous Chemicals, detailing the motivation for the draft regulation, and the specific rights and responsibilities currently described. They will not be provided with an opportunity to participate, only receiving an informational lecture (Please See Appendix 9 for a Mock Version of the Presentation, which cannot be finalized until the draft regulations is completed by MOLISA). This presentation will not take longer than 15 minutes.

- T2 (Participation, n=800): The second treatment group will receive the information treatment, but will also be provided with an opportunity to comment on the law through the online portal available on the enumerator’s tablet device. Following VCCI’s template, the participation treatment will be sub-divided into the articles of the draft regulation. For each article, the firm will be asked to answer whether it agrees with the article. If yes, the program will direct them to the next article. If no, they will be provided with a short comment box and stylus to record their comments in the program directly. In providing extended written comments, the respondents will be offered three choices: 1) they can write directly into the tablet using the stylus or keyboard, so that their comments are recorded anonymously without any knowledge of the enumerator; 2) they can orally communicate their comments to the enumerator who will write them in using the stylus; 3) they can record their comments using the recorder on the Android tablet and have those comments transcribed by the research team and submitted for them. Given the 93% literacy rate in Vietnam, we suspect that most business managers will select Option 1. We estimate that the provision of comments will take between 20 and 30 minutes.

- C1 (Control, n=600): To avoid a Hawthorne effect, a control group is also drawn that will be given a Placebo treatment. Enumerators will present an innocuous placebo power-point presentation on VCCI’s Provincial Competitiveness Index Survey (Please see Appendix 10 for this presentation). This presentation will not take longer than 15 minutes.

Summary of Stage 1: 600 firms will receive pre-treatment survey and placebo presentation, requiring 30 minutes of their time. 400 firms will receive pre-treatment survey and presentation of the Draft Regulation on Handling and Storage of Hazardous Chemicals, requiring 30 minutes of their time. 800 firms will receive pre-treatment survey, presentation of the Draft Regulation on the Draft Regulation on Handling and Storage of Hazardous Chemicals, and will be allowed to comment on each article of the draft regulation, requiring 50 minutes of their time.

2.) Follow-up Treatments (November 2014)
According to the Law on Laws, after the 60-day comment period, MOLISA drafters are obligated to respond to the comments made by individuals on the draft regulation. These comments are assembled into “Table of Comments and Responses”, which must be submitted to the Ministry of Justice prior to the finalization of regulation. The Table is particularly useful because it must itemize each of the comments received during the public comment period through different media (including internet, telephone, and during conferences), answer whether the comment was included in the final draft, and provide a written explanation for why the comment was not considered, allowing us to formally document whether the respondent firms’ comments found their way into the legislation. Importantly, the LoL does not require MOLISA to release the Table publically. However, we have a formal agreement with their drafting committee to provide us with the Table, so that we can inform participants in our experiment of how their comments were used by the drafting committee.

- T3 (Government Responsiveness, n=400): Once we have the Table of Comments and Responses (most likely in November 2014), we will categorize MOLISA’s responses into 2 groups: 1) the firm’s comment was accepted by the drafting committee and was used in revising the Draft Regulation on Handling and Storage of Hazardous Chemicals; and 2) the firm’s comment was rejected by the drafting committee directly with a response. Blocking according to these categories, we will return to the group of firms that participated in the first stage (T2), and randomly sample half of them (400 firms) to the third treatment. T3 will be informed of the drafting committee’s response to their comments in order to see whether a government response is necessary for legitimacy, and whether the nature of the response (positive or negative) matters for this effect.

- T4 (Indirect Representation, n=300): To test whether government responsiveness matters even for firms that did not participate themselves, we will also return to half of the Control Group (C1), and will share excerpts of the Table with them. The identity of the firms originally providing comments will be concealed, but we will show T4 sample the responses of government officials to anonymous firms that resemble their own (i.e. similar size and narrow industrial sector). The T4 sample will be randomly sub-divided into 3 groups: 1) those shown only positive government responses; 2) those shown only negative government responses; and 3) those shown a mix of positive and negative responses. We consider this treatment arm to be critically important for both theoretical and practical reasons.
Theoretically, very few jurisdictions in the world use direct democracy. Almost everywhere, countries and subnational governments rely on forms of representative government with elected officials speaking for their constituents. T4, therefore, allows us to test the impact of indirect representation on feelings of legitimacy, confidence in government, and ultimately compliance with important regulations.

Practically, direct consultation with every affected firm on the hundreds of regulations developed each year is infeasible in a country with roughly 300,000 private businesses and three million informal operations. For this reason, business associations play a critical role in collecting, collating, and channeling firm opinions to government actors. Since our experimental partner, VCCI, is the country’s largest business association, it is important that our experiment reveal practical information to them about how they can most efficiently improve regulations, so that they will general compliance. We believe that the T4 arm will teach VCCI whether consultation with only a sample of firms can have the same effects as general consultation, as long as the process and outcomes are adequately communicated to the larger population of affected businesses.


Summary of Stage 2: 400 firms, who participated in Stage 1 (T2), will learn about the government response to their comments. This group will be divided into those who had their comments accepted (n=200) and those who had their comments rejected (n=200). This informational treatment will take about 10 minutes. 300 firms, who were originally included in the control group, will learn about the government response participating firms. This group will be divided into

2014-10-31

2015-01-31

Compliance Measure 1: Calls to VCCI Regulatory Hotline. During the pre-treatment survey, all firms will be provided with a credit card-shaped thumb drive (USB – see images of product and planned content below) that has VCCI’s seal on the cover. The thumb drive cover will also advertise a toll-free hotline that businesses can call if they have questions about adjustments that need to be made to be considered in compliance with all new sanitary, environment, and labor safety regulations. The VCCI has agreed to host the hotline and MOLISA has agreed to provide chemical experts who can answer any detailed questions of businesses. As an initial measure of interest in compliance, we will record whether firms called the hotline for assistance, which will provide an initial intention to treat effect.

Compliance Measure 2: Self-Reported Compliance. 45 days after the posting of the final regulation by MOLISA and acceptance by the Vietnamese National Assembly, our teams will return to visit all business to conduct a brief post-treatment survey (June 2015). During this 15-minute survey, firms will be asked about 3 specific requirements of the Draft Regulation on Handling and Storage of Hazardous Chemicals: 1) equipment, including gloves, masks, and safety showers; 2) storage, including proper containers and designated safety labels; 3) factory organization, including placement of storage facilities and ventilation. Firms will be asked to answer yes or no questions about whether their current processes in these 3 areas are in compliance with Final Regulation on Handling and Storage of Hazardous Chemicals.

Compliance Measure 3: Observed Compliance. After the post-treatment survey at the firm’s factory location, firms will be asked to show the enumerator evidence of their compliance with the final regulation in the 3 highlighted areas (equipment, storage, and factory outlay). Assessments will only be taken of firms that volunteer to show enumerator their facilities and on particularly easy requirements. A key part of our search for the appropriate regulation has been that they include such easy to observe components and–while it is not possible to specifically identify these components until the draft regulation for public commentary is complete–MOLISA chemical experts have strongly recommended the three areas highlighted above are particularly feasible. Times will differ by factories, but we estimate observing compliance should take a maximum of twenty minutes. Only if firms agree, photographs of chemical equipment, storage, and outlay will be taken and stored with the dataset. Like the dataset, these photographs will be only identified with a firm id generated by the PIs that is only available to the PIs. The connection between firm identifying information and firm id is also only available to PIs. Further, we do not plan to publish these photographs in any publication or presentation relating to this research. Express consent would need to be sought for any such actions.

Summary of Stage 3: A hotline number for complying with regulations will be provided to firms during the pre-treatment survey in September 2014 and will be live immediately. Self-reported compliance will be measured in a post-treatments survey administered to all 1800 firms that will take about 15 minutes. Direct observations of compliance will take place immediately after the post-treatment survey. Observations of compliance will be voluntary and will not take longer than 20 minutes. This is simply an assessment of enumerators and it will be made very clear that this is not a formal inspection.

It is impossible to measure whether participation improved the quality of the law and therefore stimulated compliance through experimental design. The problem is that, unless firm-level preferences are highly variegated, it is unlikely that control and treatment groups will differ dramatically in their preferences over the regulation. Consequently, if the treatment groups’ ideas significantly improve the quality and acceptability of the laws, control firms are as likely to change their behavior as treatment firms. We would observe no difference between the compliance of the different groups statistically. In short, a highly successful experiment would appear as a non-result.

Q1 (Quality of Regulation): To ensure that we capture this potential effect, we have designated a fifth group of 100 firms as “judges” of legal quality. These firms will also be randomly sampled from the population at the beginning of the experiment, and will match the other groups on all observable characteristics. Rather than being subject to an experimental manipulation, these firms will simply be given untitled and unmarked versions of the draft regulation and final regulation resulting from the controlled experiment during June 2014. Their primary task will be to identify which of these blind documents they think is the superior legislation. The selection should not take longer than 10 minutes.

Q2 (Watered Down Regulation): To ensure that high quality law does not simply mean “watered down” regulations, we are also preparing to engage in an equivalent exercise with a group of 30 labor safety experts and knowledgeable workers. This group will also be asked to assess whether the final regulation is better than the draft in terms of its protections for workers. The selection should not take longer than 10 minutes.

Our project aims to assess the current participatory process that exists in Vietnam for draft legislation and regulations, testing whether participation by businesses can improve labor rights and safety. We use an RCT to analyze the downstream compliance to an important chemical storage regulation of five randomly assigned groups of firms: 1) those simply receiving a presentation about the regulation; 2) those invited to provide comments on the draft regulation; 3) those invited, but also subsequently notified of the response of the responsible regulatory agency to their comments; 4) those not invited, but told about the participation of similar firms; and 5) a control group not invited to participate, but instead receiving a placebo. After the final regulation is promulgated, the team will monitor the firms in all five groups to test compliance with the final legislation. Because safety requirements for hazardous chemicals are readily observable, we will have clear evidence of whether participation led to downstream compliance and which of the 3 mechanisms is most responsible.

Randomization done by computer

Individual businesses, which use or produce hazardous chemicals. Businesses are clustered with provinces (Hanoi and surrounding provinces, and individual districts within Hanoi).

Yes

6 provinces (Hanoi, Bac Ninh, Hai Duong, Vinh Phuc, Phu Tho, Hung Yen), 43 districts

2000 businesses

Arm 1: Placebo: 600; Information Treatment 400; Participation Treatment 800
Arm 2: Control 300; Indirect Democracy 300; No Response from Government 400; Positive Response from Government 400

Background: In our preliminary research for this project in September 2013, we worked with the MOLISA Labor Safety officials to estimate the number of firms that would be affected by the new hazardous chemical storage regulation and the share of firms complying with the current version of the regulation, which is considered to be outdated. MOLISA has yet to complete its pre-regulation study, so the numbers they gave us our ballpark estimates. According to their current inspection records, only 8% of affected firms are complying with the existing regulations. MOLISA did not have a population size of affected firms, but did provide us with a list of the industrial sectors that would be covered by the new chemical storage regulation, including a wide range of industries from chemical producers to paint manufacturers to garment companies. Using this list, we matched these sectors to the list of registered firms in the Hanoi metropolitan area from the National Tax Authority database. This matching exercise produced 5,200 operations that would be affected by new chemical storage guidelines. The Tax Authority list also includes information on size (annual revenue), four digit ISIC code, and gender of the business owner/manager. This sample frame is preliminary, because the sectors of affected firms may shift after the final pre-regulation study is completed. Key Assumptions: 1. Current compliance rates are extremely low (8%), implying that variation in compliance is also quite low and the expected effect size of each additional treatment will be reasonably large (after all, we are measuring growth from a near zero level). 2. To keep the power calculations tractable, we conceptualize the empirical analysis as three separate tests: H1 (Information) = T1>C; H2 (Participation)= T2>T1; H3 (Legitimacy)= T3>T2. 3. We calculate our estimated treatment effects based on two sources: 1) theoretical literature on the participation-compliance relationship; and 2) observational data from the Vietnam Provincial Competitiveness Index survey on current levels of compliance with labor laws. From these, we expect the effect size for T1 to be about .22 standard deviations (or a 6 percentage point change in the level of compliance). Conservatively, we estimate the effect size of participation (T2) to be about the same (6 points). Based on the theoretical literature, we estimate the effect size of legitimacy to be twice that of information alone (.44 standard deviations or 12 percentage points). 4. Further, using observational data from we estimate that explained variation from our three blocking variables (size, narrow sector, and gender) to be about 10%. Calculation: Using the Optimal Design software to measure effects with 5% statistical significance and with 80% power, we calculate that we would need about 522 observations in the sample. To account for potential non-response and to provide some cushion, we conservatively estimate 600 observations for both the control group and T1 (See Figure 1). Repeating this exercise for the T2, we employ the same estimated treatment effect, requiring 300 additional observations (we used 400). For H3, the legitimacy treatment, the larger effect size means that we require only 133 observations to reach 80% power (See Figure 2). To be conservative, maintain simplicity, and to allow room for a study of heterogeneous effects with the blocking variables, we maintained the same 600 observations as