| Field | Before | After |
|---|---|---|
| Field Last Published | Before September 24, 2020 06:14 PM | After September 24, 2020 09:21 PM |
| Field Intervention (Public) | Before We plan to cross randomize the incentive amount offered and the emphasis placed on government involvement in the study on the flyers advertising the baseline survey. Individuals will see either a) a 10 dollar or incentive, or b) a 20 dollar incentive; and either a) messaging that emphasizes government involvement in the study, or b) messaging that emphasizes the involvement of academic researchers. Flyers will be randomized at the food shelf-day level.To test what affects item non-response on potentially sensitive questions, such as those asking for health information - we hypothesize that ethical framing may encourage people to answer questions. This takes two forms, the deontological (or duty based) vs. the consequential (or cost-benefit frame). Moreover, knowing others feel the same way (regarding the obligation or benefits of providing health in-formation), may amplify the motivation. Finally, there is the possibility that providing information on norms of response might be helpful. Prior to seeing several potentially sensitive survey blocks, individuals will see either a) messaging that emphasizes the public health benefits of answering the survey questions (cost-benefit frame); b) messaging that emphasizes an individual’s responsibility to their community (duty frame); c) emphasizes the disproportionate impact of COVID-19 on certain ethnic and racial groups; or d) provides no messaging. Messaging content will be randomized at the individual level. | After We plan to cross randomize the incentive amount offered and the emphasis placed on government involvement in the study on the flyers advertising the baseline survey. Individuals will see either a) a 10 dollar incentive, or b) a 20 dollar incentive; and either a) messaging that emphasizes government involvement in the study, or b) messaging that emphasizes the involvement of academic researchers. Flyers will be randomized at the food shelf-day level. To test what affects item non-response on potentially sensitive questions, such as those asking for health information - we hypothesize that ethical framing may encourage people to answer questions. This takes two forms, the deontological (or duty based) vs. the consequential (or cost-benefit frame). Moreover, knowing others feel the same way (regarding the obligation or benefits of providing health in-formation), may amplify the motivation. Finally, there is the possibility that providing information on norms of response might be helpful. Prior to seeing several potentially sensitive survey blocks, individuals will see either a) messaging that emphasizes the public health benefits of answering the survey questions (cost-benefit frame); b) messaging that emphasizes an individual’s responsibility to their community (duty frame); or c) no messaging. Messaging content will be randomized at the individual level. |
| Field Experimental Design (Public) | Before 1 Recruitment and Sampling We are recruiting subjects through 11 food shelves in the Twin Cities area. Food shelves will place flyers (see Figure 1 and 2) advertising the baseline survey in bags of food and prepackaged meals that are distributed to food shelf users. We require all participants to be age 18 or older and to speak English or Spanish. We will use a touchless delivery systems to drop off and redeem flyers on a daily basis. Our target sample size is 1000 survey responses. We may stay in the field (i.e. oversample) food shelves that tend to serve minority individuals so that we can increase their representation in the survey. 2 Experimental Protocols 2.1. Randomization of Flyers (Unit Response) (a) Randomly assign survey advertising flyers at the food shelf-day level. See Appendix A for flyer examples. (b) Food shelves distribute the randomized daily flyer via food bags and prepackaged meals 2.2. Recruitment and Baseline Survey (a) For all potential study participants, elicit a preference for English or Spanish. Those with a preference for Spanish will be given a Spanish language consent form and survey questions. (b) Individuals provide consent. (c) Collect demographic information 2.3. Randomization of messaging content before a sensitive survey modules (Item Response) (a) After completing the demographics survey module, participants are randomly assigned to one of four ethical motivation messaging options; deontological, consequential, acknowledgement of racial inequities, or nothing. (b) Participants are shown the same randomly assigned message prior to each sensitive survey module. (c) Participants answer in survey modules 2-5 on media attitudes, health and COVID-19, and discrimination in healthcare. 2.4. Participants answer debrief questions and are prompted to get tested for COVID-19 if they have any officially recognized symptoms. Participants are also notified that they may share the survey with family and friends 2.5. Electronic gift cards are texted to participants who complete the survey to compensate them for their time. 3 Randomization Within each strata we will randomly assign food shelf dates to the four different treatments using Stata14.2 for the unit nonresponse outcomes. Robust standard errors will be used as the level of outcome is the same as the unit of randomization. For item nonresponse outcomes, we will randomize at the individual level in qualtrics. 4 Survey Messaging Content One of the following message options will be randomly assigned to participants after completing the demographics section of the survey module. 4.1.No framing– No message displayed. 4.2.Consequential framing– "Answering the questions in this survey is an easy way that you can help improve the public health response to COVID-19 in your community." 4.3.Deonteological framing– "It is important for everyone to do their part to protect their community during the COVID-19 pandemic." 4.4.Acknowledgement of racial inequities– "COVID-19 is affecting everyone, but is hitting African American and Latinx communities particularly hard." | After 1 Recruitment and Sampling We are recruiting subjects through 10 food shelves in the Twin Cities area. Food shelves will place flyers (see Figure 1 and 2) advertising the baseline survey in bags of food and prepackaged meals that are distributed to food shelf users. We require all participants to be age 18 or older and to speak English or Spanish. We will use a touchless delivery systems to drop off and redeem flyers on a daily basis. Our target sample size is 1000 survey responses. We may stay in the field (i.e. oversample) food shelves that tend to serve minority individuals so that we can increase their representation in the survey. 2 Experimental Protocols 2.1. Randomization of Flyers (Unit Response) (a) Randomly assign survey advertising flyers at the food shelf-day level. (b) Food shelves distribute the randomized daily flyer via food bags and prepackaged meals 2.2. Recruitment and Baseline Survey (a) For all potential study participants, elicit a preference for English or Spanish. Those with a preference for Spanish will be given a Spanish language consent form and survey questions. (b) Individuals provide consent. (c) Collect demographic information 2.3. Randomization of messaging content before a sensitive survey modules (Item Response) (a) After completing the demographics survey module, participants are randomly assigned to one of four ethical motivation messaging options; deontological, consequential, acknowledgement of racial inequities, or nothing. (b) Participants are shown the same randomly assigned message prior to each sensitive survey module. (c) Participants answer in survey modules 2-5 on media attitudes, health and COVID-19, and discrimination in healthcare. 2.4. Participants answer debrief questions and are prompted to get tested for COVID-19 if they have any officially recognized symptoms. Participants are also notified that they may share the survey with family and friends 2.5. Electronic gift cards are texted to participants who complete the survey to compensate them for their time. 3 Randomization: We will randomly assign food shelf-dates to the four different treatments using Stata 14.2 for the unit nonresponse outcomes. Robust standard errors will be used as the level of outcome is the same as the unit of randomization. For item nonresponse outcomes, we will randomize at the individual level in Qualtrics. 4 Survey Messaging Content One of the following message options will be randomly assigned to participants after completing the demographics section of the survey module. 4.1.No framing– No message displayed. 4.2.Consequential framing– "Answering the questions in this survey is an easy way that you can help improve the public health response to COVID-19 in your community." 4.3.Deonteological framing– "It is important for everyone to do their part to protect their community during the COVID-19 pandemic." 4.4.Acknowledgement of racial inequities– "COVID-19 is affecting everyone, but is hitting African American and Latinx communities particularly hard." |
| Field Randomization Method | Before We will use the statistical package Stata 14 for unit response randomization and computer randomization via Qualtrics for item response randomization. | After We will use Stata for unit response randomization and computer randomization via Qualtrics for item response randomization. |
| Field Randomization Unit | Before 1. Unit Response Randomization: 74 food shelf-days. 2. Item Response Randomization: individual-level randomization. | After 1. Unit Response Randomization: randomization at the food shelf-day level. 2. Item Response Randomization: individual-level randomization. |
| Field Planned Number of Observations | Before 1,000 individuals | After 1,000 individuals. |
| Field Power calculation: Minimum Detectable Effect Size for Main Outcomes | Before Our power calculations for the effect of non-government framing on unit response compute minimum detectable effects (MDEs) conditional on given levels of control group unit response (i.e. take-up), which we vary from 10% to 30%. We also vary the amount of intra-cluster correlation (ICC) as our unit of randomization is the foodshelf-date. Our calculations assume that our data features 84 food shelf-dates, evenly allocated to three framing arms --- government, government with an acknowledgement of racial inequities, and researcher --- with an average of 100 flyers distributed per food shelf-date. Fixing government framing as the control group and control take-up at 10%, our calculations imply a MDE of 1.93 percentage points (or 19.3% greater than the control group mean) in the absence of intra-cluster correlation. Holding control take-up constant at 10%, the required MDE rises to 3.43 percentage points (+34.3%) given 2% intra-cluster correlation and to 6.94 percentage points (+69.4%) given 10% ICC. If take-up in the control group is 30%, the required MDE falls to 2.87 percentage points (or 9.6% greater than the control mean) in the absence of ICC, 5.00 percentage points (+16.7%) given 2% ICC, and 9.68 percentage points (+32.3%) given 10% ICC. In computing MDEs for item response, we assume our target sample size of 1000 survey respondents is evenly divided across four messaging arms. Given 70% item response in the control group of no messaging, we would require a 10.77 percentage point effect size (or 15.38% higher than the control mean) for treatment effects to be detectable. As item response in the control group rises, MDEs fall: given 90% item response in the no messaging group, the MDE is 6.31 percentage points (+7.01% from the control mean). | After Our power calculations for the effect of non-government framing on unit response compute minimum detectable effects (MDEs) conditional on given levels of control group unit response (i.e. take-up), which we vary from 10% to 30%. We also vary the amount of intra-cluster correlation (ICC) as our unit of randomization is the foodshelf-date. Our calculations assume that our data features 74 food shelf-dates, evenly allocated to two framing arms --- government and researcher --- with an average of 100 flyers distributed per food shelf-date. Fixing government framing as the control group and control take-up at 10%, our calculations imply a MDE of 2.07 percentage points (or 20.7% greater than the control group mean) in the absence of intra-cluster correlation. Holding control take-up constant at 10%, the required MDE rises to 3.68 percentage points (+36.8%) given 2% intra-cluster correlation and to 7.46 percentage points (+74.6%) given 10% ICC. If take-up in the control group is 30%, the required MDE falls to 3.07 percentage points (or 10.2% greater than the control mean) in the absence of ICC, 5.34 percentage points (+17.8%) given 2% ICC, and 10.35 percentage points (+34.5%) given 10% ICC. In computing MDEs for item response, we assume our target sample size of 1000 survey respondents is evenly divided across four messaging arms. Given 70% item response in the control group of no messaging, we would require a 10.77 percentage point effect size (or 15.38% higher than the control mean) for treatment effects to be detectable. As item response in the control group rises, MDEs fall: given 90% item response in the no messaging group, the MDE is 6.31 percentage points (+7.01% from the control mean). |
| Field | Before | After |
|---|---|---|
| Field Document | Before |
After
COVID_Minnesota_PAP_24sept2020.pdf
MD5:
49bdef9e47e848049e8220b96241e3f0
SHA1:
ddf670b628bee5f80435478b9143d4559db299cd
|
| Field Title | Before | After Pre Analysis Plan |
| Field | Value |
|---|---|
| Field Document |
Value
COVID_Minnesota_PAP_17sept2020.pdf
MD5:
ed8339d2638777da6abff35c54c522cb
SHA1:
e9de0eec3a4617f171f9b5b8ededf3b70fc33971
|
| Field Title | Value Pre-Analysis Plan |