Documenting and Understanding the Invisible Load

In the interest of preserving the integrity of the experiment, the intervention is being described in full in the experimental design sections that will remain hidden until the experiment is completed.

2025-09-01

2027-12-31

Reply received (0/1). Note: although we are listing this as a primary outcome we do this mostly because it is the first outcome we can measure, rather than the outcome of primary interest to our research questions.

ReplyLatency: Continuous measure of time to reply (in hours or days) from when the email was sent. This is the primary definition for latency and will be used in all main analyses. Note on robustness: While the primary definition of ReplyLatency is continuous, for robustness and interpretability we may also categorize response time into bins (e.g., within 1 day, within 7 days).
Sentiment: Tone / sentiment of reply.
Length: Length of reply (word/character count).
Content: Measures based on the content of the response (e.g. spot availability, offer to schedule a tour/appointment, provision of alternative options or helpful resources, etc.)

In the interest of preserving the integrity of the experiment, the experimental design is being described in full in the experimental design sections that will remain hidden until the experiment is completed.

Not available

Randomization is done by computer. Stratification ensures balance across demographic attributes of the centers like provider type, region, and other characteristics.

Childcare or eldercare center.

No

In our full roll out we include about 52,000 childcare centers and 22,000 eldercare centers. Each center represents one cluster, as responses are measured at the center level. We included about 2000 daycares in the first pilot study. And we include approximately 3000 daycare centers and 1500 eldercare centers in the second pilot study.

Each center represents one cluster, as responses are measured at the center level. Following the pilot, we will update our power calculations based on the observed effect sizes.

Because each center is contacted only once, the total number of observations will be the same as the number of clusters for the primary outcomes in the first and second pilots. Each observation corresponds to one unique center’s response (or non-response) to our email. The secondary outcomes (e.g. sentiment of the email response) will be determined by the response rate to our various treatments.

We are running a 3×6 factorial design with eighteen treatment arms. For our pilots we ran approximately equal samples for all arms. For our full roll out we vary sample sizes in each treatment and have the following approximate number in each cell:
T1_M_S: “Unexpected Need” message, male sender, single parent, 6000
T1_F_S: “Unexpected Need” message, female sender, single parent, 6000
T2_M_S: “Dropped the ball” message, male sender, single parent, 6000
T2_F_S: “Dropped the ball” message, female sender, single parent, 6000
T3_M_S: No reason message, male sender, single parent, 1000
T3_F_S: No reason message, female sender, single parent, 1000
T1_M_N: “Unexpected Need” message, male sender, non-single parent, 6000
T1_F_N: “Unexpected Need” message, female sender, non-single parent, 6000
T2_M_N: “Dropped the ball” message, male sender, non-single parent, 6000
T2_F_N: “Dropped the ball” message, female sender, non-single parent, 6000
T3_M_N: No reason message, male sender, non-single parent, 1000
T3_F_N: No reason message, female sender, non-single parent, 1000
T1_M_C: “Unexpected Need” message, male sender, child, 5000
T1_F_C: “Unexpected Need” message, female sender, child, 5000
T2_M_C: “Dropped the ball” message, male sender, child, 5000
T2_F_C: “Dropped the ball” message, female sender, child, 5000
T3_M_C: No reason message, male sender, child, 1000
T3_F_C: No reason message, female sender, child, 1000
We plan to have about 52,000 childcare centers and 22,000 eldercare centers in our experiment.

See pre-analysis plan for power calculation details