The Role of Mechanistic Explanations in Technology Experimentation and Adoption: Evidence from a Lab-in-the-Field Experiment in Uganda

Last registered on October 28, 2024

Pre-Trial

Trial Information

General Information

Title
The Role of Mechanistic Explanations in Technology Experimentation and Adoption: Evidence from a Lab-in-the-Field Experiment in Uganda
RCT ID
AEARCTR-0014649
Initial registration date
October 28, 2024

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 28, 2024, 1:44 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Primary Investigator

Affiliation
Princeton University

Other Primary Investigator(s)

PI Affiliation
Stanford University
PI Affiliation
Harvard Graduate School of Education
PI Affiliation
Development Innovation Lab, University of Chicago
PI Affiliation
Stanford University
PI Affiliation
Agriworks Uganda
PI Affiliation
Makerere University

Additional Trial Information

Status
In development
Start date
2024-11-25
End date
2025-06-01
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In many environments, agents need to learn a payoff function of their choices. The fact that these functions are often qualitatively complicated, for example encoding many interaction effects among several choice dimensions, has led many economists to model agents' learning of these functions as black boxes. Since it is costly to explore choices, risk averse agents learning payoffs through a black box can easily get stuck in some local maxima, usually only alleviated by observing an exogenous signal of a better alternative. But black-box learning is not the only learning mode available to real world agents, who can appeal to scientific advancements. Science has tapered the complexity of complicated functions to make them more useful for both ex-ante prediction and ex-post extrapolation, trading off exactness with a global model of the “first order” structure. We consider two levels of scientific advancements: (1) a conceptual dimension reduction, rewriting complicated functions in a smaller space of conceptual features, and (2) mechanistic knowledge of an approximating function to the true payoff function, described on the domain of conceptual features. We examine whether equipping agents we hypothesize to be predominantly black-box learners of a relevant payoff function, with information about these two levels of scientific discovery, progressively improves their learning efficiency.

Specifically, we consider the problem of synthetic fertilizer choice among smallholder tomato farmers in Eastern Uganda. Qualitative evidence tells us that they have a black-box understanding of this important technology and are not deploying them optimally, which is also supported by other empirical work in Africa. We provide agricultural extension demonstrations to 900 tomato farmers where we demonstrate the causal impact of an interesting fertilizer recipe, and randomize the provision of two additional explanations relevant to why (biophysically) this recipe achieved its result: (1) a conceptual dimension reduction of fertilizers to the three macronutrients that drive their effects; (2) mechanistic knowledge about how these nutrients move through observable features of the soil and participate in plant growth. We measure impacts of these components on incentivized beliefs about fertilizer use, WTP for novel fertilizers, and choices in a bandit problem. We consider payoffs (yields and profits) on both a grey soil archetype on which we demonstrated the recipe, as well as a counterfactual “black soil’, to speak to signal translation across contexts with heterogeneous returns. We also measure a variety of other welfare-relevant aspects documented in the psychology literature to improve with mechanistic understanding.
External Link(s)

Registration Citation

Citation
Davies, Ben et al. 2024. "The Role of Mechanistic Explanations in Technology Experimentation and Adoption: Evidence from a Lab-in-the-Field Experiment in Uganda." AEA RCT Registry. October 28. https://doi.org/10.1257/rct.14649-1.0
Experimental Details

Interventions

Intervention(s)
We study the impacts of mechanistic explanations on beliefs about fertilizers and adoption choices among smallholder tomato farmers in Eastern Uganda. Mechanistic explanations are explanations which break down a system or process into the causal interactions among its parts. In the context of this study, these explanations include descriptions of fertilizer nutrients, their roles in plant growth, and the processes through which they move through the soil and the plant. We recruit 900 smallholder farmers in Bugisu sub-region, Uganda. Fieldwork conducted by the authors found that farmers in this region have little understanding of how fertilizers work, rarely experiment with novel combinations on their own, and rely on outside recommendations that may or may not fit their own context or be based in agronomic evidence. We design a set of treatments to test whether mechanistic explanations can address these challenges. One-third of the farmers will receive a placebo training that introduces an agronomist-recommended fertilizer recipe and demonstrates it yield, but provides no further explanations. Another third will receive the same training, plus a short module that identifies the primary macronutrients contained in fertilizers and teaches the macronutrient composition of fertilizers available in the area. The final third will in addition receive mechanistic explanations about fertilizers and the mechanistic rationale behind the demonstrated recipe.

In the first group, the control group, farmers attend a demonstration of a specific agronomist-recommended fertilizer recipe for grey soil. The training will inform the farmers of the types, timings, and amounts of fertilizer used, and the yields that were generated. Farmers will be able to see for themselves the condition of the fruiting plants, and ask the facilitator questions about the fertilizer recipe. The training will also cover methods of fertilizer application, such as how fertilizer should be inserted into planting holes or dispersed around the plant stem as a top dressing. Importantly, this control training will not inform farmers about the macronutrients that are contained in fertilizers, nor the functions of these nutrients and the processes through which they affect plant growth. The control training corresponds in content to what is typically seen in a standard extension or commercial demo.

The second group, treatment 1, will consist of the same activities as the control training. The only difference is that farmers will receive an additional module that describes which macronutrients are contained in synthetic fertilizers, and the macronutrient composition of common fertilizers on the market. This training will not inform farmers about the causal mechanisms through which each nutrient affects plant growth, nor the specific roles that different nutrients play. This training corresponds to an intermediate level of mechanistic understanding in which the learner is able to unpack the black box of a technology and identify which components drive its operation, but has not yet learned how these components work and interact — only that they exist. This group is important for two reasons. Theoretically, it allows us to identify the importance of dimension reduction in learning processes. That is, does knowing which elements of a technology are essential allow one to better learn and apply it, even if one does not know how these elements work? Practically, it also corresponds to a light-touch, low-cost version of a full mechanistic training, and so understanding its impact is relevant for policy.

The third group, treatment 2, will receive the treatment 1 training, as well as an additional module that describes how nutrients affect plant growth, and the processes by which they move and interact in the soil and the plant. This module will describe different soil archetypes, including how soil texture and components affect the ability to retain nutrients, the nutrients that are naturally available in different soils, and mechanisms through which nutrients are retained or lost (e.g., nitrogen leaching, potassium binding). The module also also describes the roles of each of the primary macronutrients in plant growth. These roles are then given mechanistic explanations: for example, phosphorus is needed early in the planting season because the plant can recycle this nutrient many times throughout the season. This training, if leveraged by the farmer, can generate beliefs about the production function through deductive reasoning. It can do this ex-ante. For example, a farmer who learns about clay’s tendency to contain and retain potassium can deduce ex ante (without experimentation) that a soil with high clay content needs little potassium supplementation. A farmer who understands mechanisms can also generalize more effectively from observed experiments (ex-post learning). If the above farmer observes a high return to a certain amount of potassium supplementation on a high-clay soil, he can extrapolate that the marginal return will be even higher on a gray soil that is low in clays.
Intervention Start Date
2024-11-25
Intervention End Date
2025-06-01

Primary Outcomes

Primary Outcomes (end points)
Exploration in the Bandit Problem
Regret in Bandit Problem
Final Profit in Bandit Problem
Exploitation in the Bandit Problem
Difference in yield predictions for macronutrient-equivalent fertilizer recipes
Correct identification of fertilizer substitutes
Difference in yield predictions – good timing vs. poor timing
Correct diagnosis of nutrient deficiencies
Difference in yield predictions – splitting vs. not splitting nitrogen
Signal translation
Self-efficacy in technology usage
WTP for MOP
Primary Outcomes (explanation)
Exploration in the Bandit Problem: We will define the Euclidean distance over the space of fertilizer applications and timings. This captures the intuition that two combinations of fertilizers are close if they provide similar amounts of the same nutrients at similar points in time, and distant if they provide different amounts of nutrients, different types of nutrients, and/or different timings. We will then calculate the average distance between consecutive fertilizer combinations chosen by the participants, and take the average across rounds. This captures the average change, or exploration distance, between rounds.
Prediction: Treatment 2 (++), Treatment 1 (+)

Regret in Bandit Problem: The total difference between the optimal yield/profit and the realized yields/profits in the bandit problem.
Prediction: Treatment 2 (--), Treatment 1 (-)

Final Profit in Bandit Problem: The profit achieved in the last round of the fertilizer game. Since there is no exploration gain in the final round of the game, a rational participant should choose the combination that maximizes flow profits.
Prediction: Treatment 2 (++), Treatment 1 (+)

Exploitation in the Bandit Problem: In the bandit game, we give farmers the option to resubmit the same fertilizer combination they chose previously. We call this “exploitation.” This is a dummy variable for whether the farmer ever chooses exploitation during the bandit.
Prediction: Treatment 2 (--), Treatment 1 (-)

Difference in yield predictions for macronutrient-equivalent fertilizer recipes: We will elicit farmers predictions for yields of different fertilizer recipes that provide the same macronutrients. We will show farmers a sample of each recipe so they can understand the amounts that are being applied.
Prediction: Treatment 2 (--), Treatment 1 (--)

Correct identification of fertilizer substitutes: We will ask a quiz question that prompts a farmer to choose a substitute for a fertilizer (type and amount) if it was not available in their area. The outcome will be the difference between the amount of macronutrients provided by the missing fertilizer and the farmer's substitute (rescaled so that higher values indicate closer distances).
Prediction: Treatment 2 (++), Treatment 1 (++)

Difference in yield predictions – good timing vs. poor timing: We will elicit farmers’ predictions for pairs of fertilizer recipes. These recipes will vary the timing of fertilizer application but hold the amount and type of fertilizer constant. The outcome will be the predicted yield for the good timing recipe minus the predicted yield for the poor timing recipe. This is an incentivized test for whether the farmer understands key functional explanations presented in the mechanisms training. The outcome will be distance from the true difference.
Prediction: Treatment 2 (--), Treatment 1 (0)


Correct diagnosis of nutrient deficiencies: We will present farmers with a series of vignettes that describe poor plant growth. We will then ask farmers to diagnose the problem. We will test whether farmers correctly diagnose nutrient deficiencies, and conditional on a correct diagnosis, prescribe the correct fertilizer to address the problem.
Prediction: Treatment 2 (++), Treatment 1 (0)

Difference in yield predictions – splitting vs. not splitting nitrogen: We will capture whether farmers understand an important mechanistic process, nitrogen leaching, by asking them to compare the yields of two different N applications, one where nitrogen is split across several applications, and one where it is applied in fewer applications. The outcome will be the distance from the true difference.
Prediction: Treatment 2 (--), Treatment 1 (0)

Signal translation: We will ask the farmer to predict the yield of a known fertilizer combination on a different soil type. For example, in the demonstration we will reveal the yields obtained with the demo fertilizer usage. Then we will ask the farmer to predict the yield on a randomly selected soil type, other than the grey soil. The outcome will be distance from the true yield.
Prediction: Treatment 2 (--), Treatment 1 (-)

Self-efficacy in technology usage: We will ask the farmer a series of questions regarding her feelings of self-efficacy and ability with respect to fertilizer usage.
Prediction: Treatment 2 (++), Treatment 1 (+)

WTP for MOP: We will elicit the farmer’s WTP for a bag of MOP fertilizer, one of the fertilizers used in the demo fertilizer recipe. MOP is an uncommon fertilizer in this region, and so represents a novel technology.
Prediction: Treatment 2 (++), Treatment 1 (+)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
This study explores the impact of mechanistic explanations on the adoption of and experimentation with synthetic fertilizers among smallholder farmers in Eastern Uganda. A total of 900 tomato farmers will be randomly assigned to one of three groups: control, treatment 1, and treatment 2 (300 farmers per group). The randomization will be stratified by locality to account for geographic differences in soil types and agricultural practices. Treatment is assigned at the individual farmer level. The randomization is not clustered.

After the training sessions, farmers participate in two activities designed to measure the impact of the interventions. First, they engage in lab-in-the-field activities that test their knowledge and beliefs about fertilizers. Second, they take part in a digital multi-armed bandit game simulating multiple growing seasons, allowing them to experiment with different fertilizer applications and observe the resulting yields.

To analyze the results, we will use standard regression techniques, regressing the outcome on treatment indicators and (in some specifications) baseline covariates.
Experimental Design Details
Not available
Randomization Method
Randomization will be conducted by computer
Randomization Unit
Individual
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
900 farmers
Sample size: planned number of observations
900 farmers
Sample size (or number of clusters) by treatment arms
Control Group: 300 farmers
Treatment 1: 300 farmers
Treatment 2: 300 farmers
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
Stanford University Institutional Review Board
IRB Approval Date
2023-10-27
IRB Approval Number
72226
IRB Name
Princeton University Institutional Review Board
IRB Approval Date
2024-10-23
IRB Approval Number
17113
IRB Name
Uganda National Council for Science and Technology
IRB Approval Date
2024-08-05
IRB Approval Number
SS2112ES