Experimental Design
To test our research questions, we will hire undergraduate computer science students enrolled in 4-year college programs in Dhaka to complete a computer-programming job that asks programmers to implement five feature additions to an existing Java-based script. Participants will be given four and a half hours to complete the task. All features can be implemented independently of each other and are designed to be of similar difficulty. Layered onto this program is task tracking that allows us to track effort over time by collecting data on key strokes, pixels scrolled, and clicks participants make each minute. The program also has a tool that automatically checks the functionality of features upon submission, and programmers can submit each feature multiple times until they have confirmation that they have completed it correctly. The number of times each feature is submitted to be evaluated by the programmer is also stored by the program.
We chose the population of Dhaka computer science undergraduate students for our study for several reasons. First, Dhaka experiences very high temperatures in summer; the average maximum temperature in September is 32 degrees Celsius and the average minimum is not much lower at 26 degrees Celsius. Second, the IT sector contributes significantly to Bangladesh’s economy; in 2017, it was estimated that the IT sector in Bangladesh was worth $600 million and accounted for about 250,000 jobs . Ensuring IT worker productivity is maintained as temperatures rise is, therefore, likely to be important to the continued economic development of the country.
Our randomized control trials involve a series of treatments that will be randomly assigned across session-room temperatures. To generate temperature variation in our study, we will randomly assign the air conditioner temperature in each room that participants are working in to be 23 degrees or 29 degrees Celsius (73.4 degrees Fahrenheit or 84.2 degrees Fahrenheit). We will run two session-rooms at a time with 10 participants per room for both the individual sessions and the team sessions. We have selected the temperature values 23 degrees or 29 degrees to generate sufficient temperature variation across participants, and to ensure that we are not subjecting participants to uncomfortably high temperatures.
In addition to temperature variation, we will vary how the bonuses are allocated across sessions. In half of all sessions, participants will receive a constant bonus rate for each feature successfully implemented as determined by the automated testing tool. The bonus will be the will be equal to 9% of their total salary, which is 13USD/1100BDT, allowing them to earn an additional 45% of their salary in bonuses. In the other half of all sessions, participants receive an increasing bonus rate for each feature successfully implemented. The bonus will increase for each feature implemented by 3 percentage points, and will start at 3% of salary. Therefore, participants in both bonus structure treatments can earn the same total amount in bonuses. This treatment is intended to establish whether cost of effort increases faster when temperatures are higher such that workers benefit more from an increasing than a constant pay-for-performance rate over time when they are in warmer temperatures.
Lastly, we will randomly vary whether the task is to be completed in pairs of two programmers or individually. Within the team sessions, pairs will be randomly assigned and will be instructed to work on the task according to a pair programming format where members take turns being the driver (typing) and navigator (coming up with solutions). With 10 participants per session room, each session room assigned to the team treatment will have 5 teams. This treatment is intended to establish whether the impacts of higher temperature differ when team work is required relative to when it is not. In the team sessions, bonuses will be paid for joint performance such that if a team successfully implements a feature, both team members will receive an equivalent bonus. Both team and independent sessions will be randomly assigned the different incentive designs.