License adoption and Project success: Field evidence from a randomised controlled trial in open source software

Last registered on January 21, 2020

Pre-Trial

Trial Information

General Information

Title
License adoption and Project success: Field evidence from a randomised controlled trial in open source software
RCT ID
AEARCTR-0004478
Initial registration date
January 19, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
January 21, 2020, 2:01 PM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region

Primary Investigator

Affiliation
ETH Zurich

Other Primary Investigator(s)

PI Affiliation
ETH-Zurich

Additional Trial Information

Status
In development
Start date
2020-01-01
End date
2021-06-01
Secondary IDs
Abstract
In a collaborative, innovative project, do licensing terms that govern use of project output affect the team's effort and project success? This research seeks to provide experimental evidence on this question, in the important case of open-source software development. We propose a randomized field experiment on a popular code hosting platform. We create exogenous variation in licensing terms of projects through preference shifts of project owners, which we induce through suggestions and selective information provision. We measure the causal effect of license adoption on metrics for team size, team effort as well as product quality
External Link(s)

Registration Citation

Citation
Ash, Elliott and Swagatam Sinha. 2020. "License adoption and Project success: Field evidence from a randomised controlled trial in open source software." AEA RCT Registry. January 21. https://doi.org/10.1257/rct.4478-1.0
Experimental Details

Interventions

Intervention(s)
Our intervention will comprise of contacting open source software developers, who have not yet adopted a license, and nudge them to adopt a license. Depending on the type of message or nudge that we send, our subject population of software projects will be divided into three treatment arms:
1. Control group - These projects will not receive a message.
2. 'Copyleft' group - These projects will receive a message advising adoption of a copyleft license(e.g., GPLv3)
3. 'Permissive' group - These projects will receive a message advising adoption of a permissive license(e.g., Apache license)
Intervention Start Date
2020-01-30
Intervention End Date
2020-04-30

Primary Outcomes

Primary Outcomes (end points)
1. Team Effort
2. Product Quality
Primary Outcomes (explanation)
1. Team Effort (measured by number of commits to the project, and number of forks of the original project)
2. Product Quality (measured by user ratings, number of dependencies, and text based quality measures)

Secondary Outcomes

Secondary Outcomes (end points)
1. Team size (measyred by the total number of individual committers to the porojects of choice, as compared to the the total number of contributors in the 3 months preceding the intervention).
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The intervention will be evaluated with a randomised controlled trial (RCT) with three arms, with randomisation at the level of open source software projects. We will stratify our study on the basis of software ecosystem, and focus on 4 software ecosystems for the current study, from which we will randomly pick 300 projects.
Experimental Design Details
Based on extant research and current data about GitHub repositories, we have created a shortlist of software package managers that have a large enough presence on GitHub and also have a large number of unlicensed repositories: NPM for Javascript, NuGet for .NET libraries, Packagist for PHP libraries and RubyGems for Ruby packages. (refer Tidelift dataset and Alexander Decan paper). Each of these package managers have more than 20,000 unlicensed repositories in them. We will randomly select 1500 packages from each repository, which will be divided equally into each of the four treatment arms (Passive Control, Active Control, Copyleft and Permissive groups)
Randomization Method
From an exhasutive list of candidate projects, we will randomly select projects by using a pseudo random generator on a pre designed program.
Randomization Unit
Treatment will be randomised at the level of software projects.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
6000 software projects
Sample size: planned number of observations
6000 software projects
Sample size (or number of clusters) by treatment arms
1500 projects in each of Passive Control group, Active Control group, Permissive License group and Copyleft license group.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
Ethics Commission of ETH Zurich
IRB Approval Date
2019-04-09
IRB Approval Number
N/A

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials