The Adoption of Generative Artificial Intelligence

We will run a survey with a randomized information provision experiment on the capabilities of ChatGPT in workers’ job tasks.

Treatment 1 ("productivity") exposes workers to information about the general productivity of ChatGPT in each job task. Following Eloundou et al. (2023), we inform workers whether access to ChatGPT can reduce the time it takes for an average worker to complete a task by at least half (50%).

Treatment 2 ("complementarity") exposes workers to information about whether the time savings from ChatGPT are smaller, similar, or larger for workers with expertise in that task.

The treatment groups see a comparison of their assessments and the expert assessments, together with brief explanations of the expert assessments. The control group only sees a summary of their prior assessments.

2023-11-17

2024-01-31

1. Beliefs about the capabilities of ChatGPT in workers' job tasks (productivity and expertise-complementarity).
2. Adoption of ChatGPT (intended and actual).

Our experiment exposes workers to expert assessment of the capabilities of ChatGPT in different job tasks. Our expert assessments are based on Eloundou et al. (2023), adapted to the Danish context, and validated by industry specialists.

We have three treatment arms:

Treatment 1 ("productivity"): Workers receive information about the general productivity of ChatGPT in each job task. Following Eloundou et al. (2023), we inform workers whether access to ChatGPT can reduce the time it takes for an average worker to complete a task by at least half (50%).

Treatment 2 ("complementarity"): Workers receive information about whether the time savings from ChatGPT are smaller, similar, or larger for workers with expertise in that task.

Control: The control group only sees a summary of their prior assessments.

The treatment groups see a comparison of their assessments and the expert assessments, together with brief explanations of the expert assessments.

Prior to the treatment, we ask workers about the importance of the job tasks and their expertise in the tasks. We also ask workers about their experiences with ChatGPT in the job tasks. Furthermore, we elicit workers' prior beliefs about the general productivity (treatment 1) and expertise-complementarity (treatment 2) of ChatGPT in the job tasks.

After the treatment, we ask workers about their intended use of ChatGPT in their job tasks. We also allow workers to sign up for additional material on potential use cases of ChatGPT in their occupations.

Finally, we ask workers about their beliefs about the productivity of ChatGPT in their own jobs. We also ask workers about misalignments in their stated beliefs and intended behaviors. The follow-up questions unpack the chain from information provision via updated beliefs to adoption behaviors.

We will conduct a follow-up survey two weeks after the workers' response to the main survey. The follow-up will assess (i) whether workers' beliefs about ChatGPT persist over time, (ii) whether workers' intended adoption of ChatGPT manifests into the actual adoption, and (iii) short-run spillovers across coworkers in belief formation and adoption behaviors.

Additional information on selecting occupations:
We survey occupations that meet the following criteria:
1. The occupation has a well-identified occupational code (ISCO and SOC).
2. The occupation has at least one job task in which ChatGPT can reduce the time it takes for an average worker to complete it by at least half (Eloundou et al., 2023).
3. The occupation has enough workers for statistical analysis.

The resulting list of occupations is:
Accountants
Customer supporters
Financial advisors
HR professionals
IT supporters
Journalists
Legal professionals
Marketing professionals
Software developers
Teachers
Office clerks

Additional information on selecting job tasks:
We use the Occupational Information Network (O*NET) to identify the job tasks for each occupation. For each occupation, we select 6 job tasks that represent the capabilities of ChatGPT ("general productivity" and "expertise-complementarity") in the occupation. For task combinations with similar representation, we prioritize combinations with higher task relevances, as measured by O*NET.

Not available

We randomize sampling and treatment using a two-stage randomization procedure by workplaces and workers. In the first stage, we split workplaces into strata based on sampling and treatment intensities. In the second stage, we sample workers and assign them treatments based on their workplace-specific sampling and treatment intensities. We use Statas random-number generator (“runiform”) with a pre-specified seed.

Sampling: We randomly allocate workplaces into three strata with sampling rates of 0%, 50%, and 100% (with an equal number of workplaces in strata 2 and 3). Workers are sampled randomly according to their workplace-specific sampling rates.

Treatment arms: We divide workplaces into four strata with the following shares of (treatment 1, treatment 2, and control) workers: (2/9, 2/9, 5/9), (2/9, 4/9, 3/9), (4/9, 2/9, 3/9), (4/9, 4/9, 1/9). Sampled workers are randomly allocated to treatment arms based on their workplace-specific treatment rates.

Prizes: Our survey includes randomized participation incentives. In particular, we use simple randomization to allocate respondents into four prize categories (1000, 2500, 5000, and 10000 DKK) with equal shares.

We randomize by workplaces and workers.

Yes

We randomly allocate workplaces into three strata with sampling rates of 0%, 50%, and 100% (with an equal number of workplaces in strata 2 and 3). We divide workplaces into four strata with the following shares of (treatment 1, treatment 2, and control) workers: (2/9, 2/9, 5/9), (2/9, 4/9, 3/9), (4/9, 2/9, 3/9), (4/9, 4/9, 1/9).

Please see the attached note “Sampling and Randomization” for details on the number of workplaces in the strata.

We will contact a total of 115,000 individuals in November-January 2023. We first invite 100,000 individuals to our main survey. All respondents of the main survey are invited to the follow-up. In addition, we invite 15,000 individuals only to the follow-up. We split the sample equally among the 11 occupations listed above. We use register data to identify the occupations of individuals. For occupations with fewer workers than the required sample, we distribute the excess sample equally among the remaining occupations. Please see the attached note “Sampling and Randomization” for details on the number of sampled individuals by occupation.

Please see the attached note “Sampling and Randomization” for details on the number of workplaces and workers by treatment arms.

Unknown, due to several unknown factors, including the final number of survey respondents eligible for treatment.