Experimental Design
We collected a baseline survey during May – July 2023 in five districts (Kalahandi, Mayurbhanj, Bolangir, Ravagada, and Ganjam) of Odisha state, India. These districts spanned five different agroclimatic zones and were selected from among the top ten districts in terms of MGNREGA asset construction over the five year period prior to the survey (limited to assets related to natural resource management). Our goal in doing so was to ensure this program was operational and thus relevant in our study context. We then selected 50 Gram Panchayats (GPs) per district at random from the universe of GPs, with probability proportional to the number of active 2021-22 job card holders in that GP (individuals must be from a household with a job card to be able to participate in the program and select assets to be constructed under it). In each GP, we selected 15 households at random from among all registered MGNREGA job card holders. In each household, our primary respondent was the woman who had worked the highest number of days on the program over the five-year period preceding the survey. If there was more than one such woman, we selected the youngest. If there was no woman in the household who had worked on the program over the five-year period, we selected the wife of the man who had worked the largest number of days over the same period. Our target sample was 3750 women from 250 GPs. At baseline, we were able to interview 3,426 women from 230 GPs. 20 GPs in Ganjam district refused to allow the survey to be conducted.
Due to cost constraints for the project, we selected 1,400 “target” women from the baseline with whom to conduct our interventions. We chose them as follows. First, we made the decision to discontinue the study in Ganjam district. Both the baseline survey and some additional qualitative data we collected suggest that the MGNREGA program does not function well in this district and that it operates in a top-down manner. This means that our planned interventions are less likely to be relevant in this context. Our baseline survey also shows that sample households in Ganjam are significantly wealthier than households in other districts, a characteristic generally associated with a lower demand for MGNREGA work and assets. Finally, the high refusal rate at baseline made it likely that our interventions would not be successful here. The baseline sample in the remaining four districts was 2,982 women, spread across 200 GPs in 60 blocks (an administrative unit above the GP). Second, we reduced the sample size by including only 94 GPs (with 14 additional GPs selected as back-ups) in the remaining 4 districts in our study—Bolangir, Kalahandi, Mayurbhanj, and Rayagada—where travel times were lowest for our survey firm. This helps us substantially with the cost effectiveness of the project. We instructed our implementing partners to always target all women in any GP entered, and to stop when they reach the target of 1400 women included.
We randomized our three study conditions (placebo, T1, T2) at the target woman level, stratifying on GP (there are generally 15 women in each GP, so roughly 5 women in each treatment arm). We handled misfits globally.
Our sample for analysis comprises these 1400 “target” women and one of the 3-5 friends/ neighbors she invites, who we assert must also be a MNREGA job card holder. As we lack baseline survey data for these friends, we are administering a short, pre-treatment questionnaire to this woman. Both sets of women (target women and friends) will be administered an endline questionnaire in February 2024.
We will analyze outcomes, Y_igt, using an ANCOVA specification, as follows:
Y_igt = Beta_0 + Beta_1 T1_ig + Beta_2 T2_ig + Beta_3 Y_ig,t-1 + Delta_g + F_ig + Error_ig
where i indexes women, g indexes GPs, t indexes time (t-1 is baseline, t is endline), T1 is an indicator for assignment to treatment 1, T2 is an indicator for assignment to treatment 2, and F is an indicator for being a friend of the target woman (as opposed to the target woman herself). As we stratify on GP, we include GP fixed effects. The effects of each treatment are relative to the base group of assignment to the placebo group.
We will not have baseline data for our secondary outcomes for all women; for women who are friends/ neighbors of target women, we need to keep our survey instrument short, and thus can only include our primary outcomes. For secondary outcomes, we will thus estimate two specifications: (a) one specification following equation (1), above, but using only our target women; and (b) another specification that includes all women, but omits the baseline (lagged) value of the outcome.