|
Field
Trial Status
|
Before
in_development
|
After
on_going
|
|
Field
Trial End Date
|
Before
July 17, 2026
|
After
September 30, 2026
|
|
Field
Last Published
|
Before
July 04, 2026 02:50 PM
|
After
August 04, 2026 03:21 PM
|
|
Field
Intervention End Date
|
Before
July 17, 2026
|
After
September 30, 2026
|
|
Field
Planned Number of Clusters
|
Before
300 participants.
|
After
300 participants.
August 4, 2026 update: The original experiment recruited 297 participants. The new experiment recruiting those who work in creative jobs (including graphic designers, game developers, content creators, and copywriters) will include up to 180 participants, unless we cannot recruit this many participants with these jobs from Prolific. In that case, we will recruit as many as we can.
|
|
Field
Planned Number of Observations
|
Before
300 participants, split across three treatment groups. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all.
We exclude (by asking to return the study) participants who give more than three incorrect answers to comprehension questions.
|
After
300 participants, split across three treatment groups. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all.
We exclude (by asking to return the study) participants who give more than three incorrect answers to comprehension questions.
August 4, 2026 update: See above about the up to 180 new participants.
|
|
Field
Sample size (or number of clusters) by treatment arms
|
Before
In expectation: 100 see the AI design after they design their first game, 100 only after they design their second game, and 100 not at all.
|
After
In expectation: 100 see the AI design after they design their first game, 100 only after they design their second game, and 100 not at all.
August 4, 2026 update: Same randomization probabilities as the original experiment.
|
|
Field
Intervention (Hidden)
|
Before
Participants design three games, starting with a base game and using an AI coding agent to improve or redesign it. They do so over the course of three rounds for each game, where they send a message to the AI, and it re-codes the game in response to what they say. Throughout we collect participants WTP to keep designing the game rather than give up, as well as pre- and post-design predictions of their "customer's" WTP to play the game. We also elicit their WTP to keep the game (be sent a durable link to it) as well as to include it in a public gallery that other participants can see. Finally we elicit likert pride and enjoyment measures, as well as participants' beliefs about their ability relative to other participants.
In the customer phase (which happens after the design phase) participants express their WTP to play another participants' (or the AI's) game, relative to a basic game.
There are three treatment groups, assigned randomly. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all.
We also cross-randomize whether participants are told which games they are offered in the customer phase were designed by other participants or by the AI.
|
After
Participants design three games, starting with a base game and using an AI coding agent to improve or redesign it. They do so over the course of three rounds for each game, where they send a message to the AI, and it re-codes the game in response to what they say. Throughout we collect participants WTP to keep designing the game rather than give up, as well as pre- and post-design predictions of their "customer's" WTP to play the game. We also elicit their WTP to keep the game (be sent a durable link to it) as well as to include it in a public gallery that other participants can see. Finally we elicit likert pride and enjoyment measures, as well as participants' beliefs about their ability relative to other participants.
In the customer phase (which happens after the design phase) participants express their WTP to play another participants' (or the AI's) game, relative to a basic game.
There are three treatment groups, assigned randomly. The first treatment group sees the AI design a game after they design their first game. The second treatment group sees the AI design a game only after they design their second game. The final treatment group does not see the AI design a game at all.
We also cross-randomize whether participants are told which games they are offered in the customer phase were designed by other participants or by the AI.
August 4, 2026 update: We will conduct a second experiment with up to 180 participants recruited from Prolific, screened on work experience as creatives (including graphic designers, game developers, content creators, and copywriters). The experiment is identical to the first, except these participants will not undergo the customer phase (a future round of Prolific participants will act as customers for these participants' games).
|