Understanding the Impact of Customer Feedback: Evidence from a Field Experiment with Entrepreneurs in Rwanda

Last registered on March 31, 2021

Pre-Trial

Trial Information

General Information

Title
Understanding the Impact of Customer Feedback: Evidence from a Field Experiment with Entrepreneurs in Rwanda
RCT ID
AEARCTR-0006470
Initial registration date
September 24, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
September 25, 2020, 1:59 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
March 31, 2021, 8:04 PM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Primary Investigator

Affiliation
Stanford University

Other Primary Investigator(s)

PI Affiliation
London School of Economics and Political Science
PI Affiliation
The University of Chicago Booth School of Business
PI Affiliation
The University of Texas at Austin

Additional Trial Information

Status
On going
Start date
2020-08-01
End date
2021-09-30
Secondary IDs
Abstract
This research seeks to assess the impact of customer feedback on firm performance and understand the mechanism of that impact. Both academic researchers as well as industry practitioners recognize that the voice of the customer is a powerful tool which businesses can wield in order to enhance their performance. However, very little is known about the business impact of listening to customers. We conduct a randomized controlled field experiment in Rwanda in order to study the impact of customer feedback on a sample of small entrepreneurs. We hypothesize that customer feedback could operate through two broad mechanisms – (1) just the act of seeking feedback could change the utility of the customer, or (2) the feedback causes firms to improve the products and services they offer, which in-turn changes the utility the customer derives from the firm. Our study aims at teasing apart these two effects, with clear implications for firm policy.
External Link(s)

Registration Citation

Citation
Anderson, Stephen et al. 2021. "Understanding the Impact of Customer Feedback: Evidence from a Field Experiment with Entrepreneurs in Rwanda." AEA RCT Registry. March 31. https://doi.org/10.1257/rct.6470-1.1
Experimental Details

Interventions

Intervention(s)
We conduct a randomized controlled field experiment in Rwanda in order to study the impact of customer feedback on a sample of small entrepreneurs. We hypothesize that customer feedback could operate through two broad mechanisms – (1) just the act of seeking feedback could change the utility of the customer, or (2) the feedback causes firms to improve the products and services they offer, which in-turn changes the utility the customer derives from the firm. Our study aims at teasing apart these two effects, with clear implications for firm policy.
Intervention Start Date
2020-09-01
Intervention End Date
2021-08-31

Primary Outcomes

Primary Outcomes (end points)
Sales, Profits, Number of customers, Number of loyal customers, number of new customers
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
We will randomly assign the firms into one of the two experimental groups:

1. Group 1 (Treatment): Entrepreneurs will receive a smartphone with a mobile app tool that is focused on tracking customer contacts. Next, the customers’ contacts in the application will be further randomized into two experimental groups. Customer Group 1 (CG1) will consist of customers from which the firm will seek feedback and Customer Group 2 (CG2) will consist of a hold-out sample of customers from which feedback won’t be sought Overall, this solution aims to increase an entrepreneur’s likelihood of seeking feedback from their customers. Further, it also increases their ability to pay attention to, and act on the customer feedback that they collect.

2. Group 2 (Control): Entrepreneurs will form a counter-factual group that only receives the smartphone with the application. This group will only track their customers. The control firms will not be primed to seek feedback from their customers.
Experimental Design Details
The study consists of two parts. In part one, we have already conducted a Growth Potential Index (GPI) screening survey to identify an initial sample of 274 more established and growth-oriented business owners in and around the greater Kigali area. As such, our sample is representative of entrepreneurs who: (1) have a business that’s been trading for three or more months; (2) operate their businesses out of a permanent physical structure (e.g. storefront or shipping container); (3) are not affected by COVID lockdowns (e.g. we don’t have any bars or spa lounges in our sample) and (4) are motivated to grow their businesses. In part 2, we have just concluded the collection of the baseline data for the identified firms and verified it for our sample of firms (n=~274). Next, we will randomly assign the firms into one of the two experimental groups:

1. Group 1 (Treatment): Entrepreneurs will receive a smartphone with the Contacts+ mobile app tool that is focused on tracking customer contacts. The tool allows the entrepreneurs to daily enter the contact details of the customers who purchased from their shop. Next, the customers’ contacts in the application will be further randomized into two experimental groups. Customer Group 1 (CG1) will consist of customers from which the firm will seek feedback and Customer Group 2 (CG2) will consist of a hold-out sample of customers from which feedback won’t be sought. The comparison between CG1 and CG2 will help us detect effects of purely the act of seeking customer fact by the firm, which, as described earlier, is a key metric of interest in our study. Each entrepreneur will be provided with a daily schedule to seek feedback from the identified list of customers. We have a team of client managers (CMs) who will be visiting these firms on a bi-weekly basis to help the entrepreneurs with the application usage and ensuring that the customer feedback seeking schedule is being adhered to. Overall, this solution aims to increase an entrepreneur’s likelihood of seeking feedback from a random half of their customers. Further, it also increases their ability to pay attention to, and act on the customer feedback that they collect.

2. Group 2 (Control): Entrepreneurs will form a counter-factual group that only receives the smartphone with the Contacts+ application. This group will only track their customers and enter that data into their Contacts+ application. The CMs will visit this group on a biweekly basis as well just to ensure that they are able to use the application well and are not facing any challenges. The control firms will not be primed to seek feedback from their customers.

Note that the comparison between Group 1 and Group 2 will provide us with the impact of customer feedback seeking on firms (as the only difference between Group 1 and Group 2 firms is that the Group 1 firms are seeking feedback from their customers). Further, the comparison of the CG1 and CG2 customers will provide us with the impact of only the act of seeking the feedback from customers. The comparison of CG1 and the control group’s customers will provide us with the joint impact of both – the act of seeking feedback and the actions that the firm performs to address the feedback. Lastly, the comparison between CG2 and the Control Group’s customers will provide us with the impact of only the actions that the firm performs in order to address the feedback they receive (without capturing the impact of the act of seeking the feedback). Figure 1 presents a schematic of the hypothesized mechanism as explained earlier.

We will collect detailed intervention and outcome data on participants’ firm performance (including sales, profits, firm actions and customer purchase behavior), on a monthly basis for our sample of 274 firms. The high frequency data collection for the firms in our sample should help us further improve the power of our experiment (McKenzie 2012). We shall also conduct an endline customer survey about 6 months post the start of the intervention. This survey will be a telephonic survey conducted with the customers of the firms as recorded in the Contacts+ application. This will help us capture the metrics from the customers’ side too (this way we can gain insights from both sides in the dyadic relationship between the firm and the customer).

Given our RCT research design, we can use our Treatment and Control variable at the firm level (i.e. randomly offered customer feedback seeking priming) as an orthogonal treatment variable and calculate the intention-to-treat (ITT) effects of improving access to customer feedback on performance outcomes (e.g. firm survival, sales, profits). In our analysis, we can also control for potential confounds using variables measured pre-treatment (e.g. gender, age, firm size, industry). Next we will do a similar analysis for the CG1, CG2 and control-customers’ groups in order to understand the mechanism of impact better. To analyze our final dataset, we can use a difference-in-differences (DID) approach with a panel dataset constructed from measuring the same set of firms (large N) over multiple periods (T > 2). This allows us to account for unobserved heterogeneity at the firm level, while still exploiting the randomization in our design. We can also analyze the data using analysis of covariance (ANCOVA), which has greater statistical power in an experimental setting (McKenzie 2012).
Randomization Method
Randomization done on computer using Stata
Randomization Unit
Two levels of randomization - (1) Firm Level (2) Customer Level
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
Treatment is not clustered, we follow individual randomization where each individual observation is a firm in our case
Sample size: planned number of observations
(1) 274 firms, and (2) For the number of customers, that would depend on the number of customers who purchase from the firms in our sample during the period of the experiment.
Sample size (or number of clusters) by treatment arms
137 firms in treatment and 137 in control
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials