Confidence and Preferences over Rewards for Innovating: Field Experimental Evidence

2019-04-25

2019-05-24

1) whether or not a participant submits a project to the contest
2) the quality of projects conditional on submitting

Quality of project submissions will be measured primarily by the scores judges assign submissions. Judge scores are based on 5 categories: Functionality, User Friendliness, Wide Scope of Use Cases, Novelty, Addresses Contest Problem. Each category is scored on a scale of 1-5 according to a rubric provided to all judges. We will generate normalized aggregate scores and rankings using judge evaluations.

We will also measure whether participants subsequently commercialize or sell their submissions for commercialization as a proxy for submission quality.

1) short-median run labor market outcomes of participants

We will measure the labor market outcomes of participants using follow-up surveys and LinkedIn. We will measure whether they made any changes to their employment, and if so, whether they changed job title, industry, or moved out of the labor market.

We are running an RCT within an innovation contest that is being run for the purposes of this research, and in partnership with Thermo Fisher’s office in Tijuana, Mexico. The contest is open to all Baja California, Mexico residents over the age of 18. It is a digital hackathon in which participants work on a specific problem that requires a software-based solution remotely, and submit their projects digitally. The contest is being promoted as a Hackathon that is part of a research study because we are not permitting participants to request to remove themselves from the data once they sign up for the contest. Furthermore, the contest is advertised as having up to $15,000 in prizes to be won, but the specific structure of rewards is not disclosed.

Everyone interested in signing up for the contest must first complete a survey that asks about their demographics, educational and work experience backgrounds, their programming knowledge, their experience in other innovation contests, their beliefs about their relative capabilities, and their risk preferences. They will also be asked to consent to have their survey and contest performance data used for research purposes. The sign-up deadline is approximately 48 hours before the start of the contest.

Following the sign-up deadline, participants will be randomly assigned to one of two reward structures: an all or nothing structure in which the first place winner receives the full $15,000 and a multiple prize reward structure in which there are prizes for the top ten winners. In the multiple prize reward structure, first prize is awarded $6,000, second prize $3,000, third prize $1,500, 4th place $900, and those who place in the 5th-10th place will receive $600. Contest participants will be notified about their reward structure by email at the start of the contest. The email will also provide information about the contest problem they are being asked to provide a solution for (information about the specific contest problem is not given until the contest start time to avoid people who signed up earlier having a mechanical advantage over those who signed up later). Importantly, participants in each of the two reward structures will be ranked relative to those in their reward structure. However, the same set of judges will judges both sets of projects to allow for judge fixed effects.

The contest is 54 hours after the start time. Contest submissions will be judged by academic and industry leaders in computer science and healthcare who will not be aware submitter treatment status.

We will use data on gender, innovator team composition (size of team and team member differences, and prior experience in innovative activities to examine heterogeneous treatment effects.

Within each reward structure, half of the participants will be randomly selected to receive additional information about the skill sets of competitors. This information will be based on average survey responses to our programming skills and innovation contest experience questions. This information will also be included in the email announcing the contest problem. As we have individual responses to the same questions, we will have a measure of whether the information provided demonstrates a participant is above or below the capabilities of her average competitor.

Randomization using Stata's (gen uniform()) command with an arbitrarily set and recorded seed.

Randomization into both treatments is at the individual level. Individuals will first be randomized into reward structure, and, within each reward structure, individuals will be randomized into information treatment.

No

400 individuals

400 individuals

100 individuals in winner-takes-all reward structure, no information group
100 individuals in winner-takes-all reward structure, information group
100 individuals in multiple prizes reward structure, no information group
100 individuals in multiple prizes reward structure, information group

Using the baseline submission rate of 10% (s.d. 0.03) from a previously run innovation contest, we would need a difference of 19% to detect a difference between two treatment group's output quantity at the 95% level 80% of the time.