Ramping Up Learning-by-Doing: Evidence from a Field (Work) Experiment

2022-10-12

2022-11-25

The primary outcomes for this study will be measured under three key domains:
0. Take Up:
- Measures of supervisor presence during household surveys conducted by the worker: we use interview-level supervisor reports, worker reports, and GPS coordinates.
- Knowledge Flows: we use interview-level field notes from supervisors on i) their assessments of worker performance; and ii) whether they provided workers any advice/tips upon the completion of the interview. Workers also report the receipt of advice, the nature of advice received, and the mode of communication (if any) with their supervisors regarding each interview.

1. Production Efficiency:
- Time spent on a survey module by each worker. This is measured using time stamps (in milliseconds) in the electronic survey platform.
- Minutes over/under a benchmark range of time spent to complete each survey module

2. Output Quality
- Kullback-Liebler Divergence or Relative Entropy, i.e., the distance between the distribution of responses collected by worker x vs. all other workers. This measure is calculated at the survey-question level.
- The use of survey codes: ‘Don’t Know’, ‘Refused’ and ‘Other, Specify’ in each survey module.
- Error rates from backchecks of a random sample of respondents.

This project examines the performance and on-the-job learning processes of 68 workers who are recruited to conduct household surveys in the Central Region of Uganda for a (distinct) research project. We partner with the survey implementation company to implement the following experimental design as part of their data collection activities: (a) supervisors conduct random field inspections (spot checks) of workers during household interviews; and (b) supervisors are encouraged to share tips, tricks, or advice to help coach worker performance on a list of (pre-specified) priority survey modules. The randomization in part (a) will allow us to isolate the impact of supervision on worker performance by comparing those workers who receive more vs. less intense supervision at any given point in time. The design in part (b) will further allow us to pin down the effects of supervision on worker performance and learning through coaching alone vs. other competing hypothesis, e.g., monitoring, Hawthorne effects, managerial traits. That is, we will compare a worker’s average performance on survey modules where the aggregation and transmission of knowledge from supervisors is encouraged by design vs. survey modules where workers must learn-by-doing or seek out information from their co-workers (as they normally would). We will also shed light on the transmission of information within a firm, by testing if the effects of (randomly allocated) knowledge flows from supervisors to individual workers are magnified through a ‘social multiplier’, i.e., learning from peers.

Not available

The randomization of workers to daily supervisor checks is done in the office by a computer, using the software Stata.

The random assignment of interviews to workers (i.e., survey enumerators) is carried out in-person: workers assigned to a village/community draw from an urn with folded pieces of paper.

The unit of randomization is the worker-day.

No

N/A

(1) Outcome Measures = 74,800. This consists of 68 workers, and 1100 measures of performance per worker (50 surveys * 22 modules per survey). The variation in worker performance across interviews in (1) will be useful for the estimation of worker learning curves and the analysis of causal mechanisms (e.g., the effects of monitoring). However, for part of the analysis, these observations will be aggregated to a sample size of approximately 2040, i.e., 68 workers * 30 measures of performance per worker (10 bins of experience/production time * 3 groups of ‘production tasks’ described in the experimental design). (2) Baseline Measures of Worker/Manager Characteristics = 105.

The research design does not have a clearly delineated set of treatment arms, as we randomly vary: (1) the exposure to supervision across workers and days; (2) exposure to feedback from supervisors across three groups of production tasks. In other words, everyone is eligible to receive the treatment over the course of the trial but will differ in the intensity of treatment that they receive.