Understanding the Impact of Customer Feedback: Evidence from a Field Experiment with Entrepreneurs in Rwanda

We conduct a randomized controlled field experiment in Rwanda in order to study the impact of customer feedback on a sample of small entrepreneurs. We hypothesize that customer feedback could operate through two broad mechanisms – (1) just the act of seeking feedback could change the utility of the customer, or (2) the feedback causes firms to improve the products and services they offer, which in-turn changes the utility the customer derives from the firm. Our study aims at teasing apart these two effects, with clear implications for firm policy.

Prior studies (Woodruff 1997; Gourio and Rudanko 2014; Markey 2014) show that one of the key sources of competitive advantage for firms is building customer capital which is defined as the value that a firm can derive from its customer base. Listening to what the customers have to say can provide a steady stream of feedback for the firms which could potentially help them build customer capital by securing and growing their customer base. Customer Feedback Systems are a growing phenomenon and many companies are now centered primarily around provision of customer reviews – e.g. - Yelp, Trip Advisor, Travelocity and Angie’s List. Firms soliciting customer feedback has become so common-place that customers now treat it as a standard element of the purchase process (Bone et al. 2017). Yet, we don’t know much about the causal impact that seeking customer feedback has on firms or the mechanism through which that impact operates. Our study intends to contribute to this literature by addressing the question – what is the impact of customer feedback seeking, on firms’ performance?

Some studies allude to the possibility that proactive approaches to respond to customer feedback could lead to innovation and enhanced service provision by firms which could increase customer satisfaction (Griffin and Hauser 1993). On the other hand, studies also show that merely seeking feedback could have an impact on the impression the customer has of the firm which in-turn could make the customers more satisfied (Morrison and Bies, 1991). We hypothesize that the impact of customer feedback on firm performance could operate through two different mechanisms. The first mechanism takes into account the direct impact of the feedback seeking process on the customers. When a customer receives a phone call or message enquiring about their recent purchase experience, it could lead to a positive impact on the customer’s impression of the firm. This could happen due to various reasons such as - customer could feel (s)he is more valued (Ping, 1993) or the feedback seeking message could act as an advertisement which reminds the customer of the firm (Sahni et al. 2017). The second mechanism is indirect in nature, which means the feedback received by the firms helps them understand areas which need improvement. Thus, by acting on that feedback and making improvement in their products or services, the firms are able to satisfy the customers more (Griffin and Hauser, 1993). It is important for firms to understand the relative impact from these two mechanisms since it would help them use customer feedback in a more efficient manner. Our study will help in understanding these two mechanisms of impact.

Given this context, we aim to answer the following questions through our research:

1. How can seeking customer feedback improve the firm performance (sales and profits) in an affordable, scalable way (for instance, by using a “cheap channel” (e.g. a free mobile application) with a solution that is “open to all”)?
2. What is the mechanism of the impact of customer feedback? Is there any additional impact from just the act of seeking customer feedback when compared to acting on the feedback that the firm receives?
3. Do the marginal effects of customer feedback differ depending on individual characteristics (e.g. an owner’s age, gender, motivation, business acumen, background, and psycho-metric factors like relational vs transactional exchange orientation) and organizational characteristics (e.g. a firm’s level of establishment, industry, products/services, target customers, market differentiation, formalization, and operational history)?

We conduct our study with micro and small enterprises in Rwanda. The small firms in Rwanda provide an ideal setting for measuring the impact of customer feedback since, based on our pilot study, we find that almost all the firms in our sample have never had any formal process in place to seek customer feedback. This will help us in obtaining a clean control group which would be unexposed to any feedback process and this will help us later to make causal claims on the impact of seeking feedback.

To the best of our knowledge, researchers have not yet causally explored the impact on firms of their own customers’ feedback. The few customer-feedback studies that exist focus on the impact of customer ratings on other customers’ product choice (Sun 2012, Chintagunta et al. 2010). While they provide critical insights on the role of customer reviews, these papers do not touch on the potential impact of seeking customer feedback on the firm as well as the customer providing the feedback.

To address this gap in our knowledge, we use a randomized controlled field experiment (RCFE) to measure the impact on business performance of customer feedback. Through this research, we aim to better understand the role of customer feedback in driving enterprise growth, and how it can be used to overcome barriers to growth and unlock the potential of emerging-market enterprises in a cost-effective, scalable way. We wish to note that RCFE is the current gold standard in causal inference and will help us in making clean causal claims about the impact of customer feedback as well as the mechanisms of change.

We will use an easy-to-use and free mobile application tool (images in Appendix) that allows a business owner to track its customers’ contact information. This contact data of customers can then be used to seek feedback from the customers. By making their own customers’ views salient for the firms, the customer feedback may nudge entrepreneurs to think and behave differently. They could take steps to address the customers’ complaints or maintain aspects of their product which were praised by the customers, thereby leading to higher firm performance.

By manipulating entrepreneurs’ to seek feedback (and as a result we also manipulate the access of customer feedback to firms), we are interested in determining whether our main outcomes of interest (including firm survival, sales and profits) are affected differently, as well as the process mechanisms through which these differential effects occur (i.e. theory of change discussed above).

2020-09-01

2021-02-28

Sales, Profits, Number of customers, Number of loyal customers, number of new customers

We will randomly assign the firms into one of the two experimental groups:

1. Group 1 (Treatment): Entrepreneurs will receive a smartphone with a mobile app tool that is focused on tracking customer contacts. Next, the customers’ contacts in the application will be further randomized into two experimental groups. Customer Group 1 (CG1) will consist of customers from which the firm will seek feedback and Customer Group 2 (CG2) will consist of a hold-out sample of customers from which feedback won’t be sought Overall, this solution aims to increase an entrepreneur’s likelihood of seeking feedback from their customers. Further, it also increases their ability to pay attention to, and act on the customer feedback that they collect.

2. Group 2 (Control): Entrepreneurs will form a counter-factual group that only receives the smartphone with the application. This group will only track their customers. The control firms will not be primed to seek feedback from their customers.

The study consists of two parts. In part one, we have already conducted a Growth Potential Index (GPI) screening survey to identify an initial sample of 274 more established and growth-oriented business owners in and around the greater Kigali area. As such, our sample is representative of entrepreneurs who: (1) have a business that’s been trading for three or more months; (2) operate their businesses out of a permanent physical structure (e.g. storefront or shipping container); (3) are not affected by COVID lockdowns (e.g. we don’t have any bars or spa lounges in our sample) and (4) are motivated to grow their businesses. In part 2, we have just concluded the collection of the baseline data for the identified firms and verified it for our sample of firms (n=~274). Next, we will randomly assign the firms into one of the two experimental groups:

1. Group 1 (Treatment): Entrepreneurs will receive a smartphone with the Contacts+ mobile app tool that is focused on tracking customer contacts. The tool allows the entrepreneurs to daily enter the contact details of the customers who purchased from their shop. Next, the customers’ contacts in the application will be further randomized into two experimental groups. Customer Group 1 (CG1) will consist of customers from which the firm will seek feedback and Customer Group 2 (CG2) will consist of a hold-out sample of customers from which feedback won’t be sought. The comparison between CG1 and CG2 will help us detect effects of purely the act of seeking customer fact by the firm, which, as described earlier, is a key metric of interest in our study. Each entrepreneur will be provided with a daily schedule to seek feedback from the identified list of customers. We have a team of client managers (CMs) who will be visiting these firms on a bi-weekly basis to help the entrepreneurs with the application usage and ensuring that the customer feedback seeking schedule is being adhered to. Overall, this solution aims to increase an entrepreneur’s likelihood of seeking feedback from a random half of their customers. Further, it also increases their ability to pay attention to, and act on the customer feedback that they collect.

2. Group 2 (Control): Entrepreneurs will form a counter-factual group that only receives the smartphone with the Contacts+ application. This group will only track their customers and enter that data into their Contacts+ application. The CMs will visit this group on a biweekly basis as well just to ensure that they are able to use the application well and are not facing any challenges. The control firms will not be primed to seek feedback from their customers.

Note that the comparison between Group 1 and Group 2 will provide us with the impact of customer feedback seeking on firms (as the only difference between Group 1 and Group 2 firms is that the Group 1 firms are seeking feedback from their customers). Further, the comparison of the CG1 and CG2 customers will provide us with the impact of only the act of seeking the feedback from customers. The comparison of CG1 and the control group’s customers will provide us with the joint impact of both – the act of seeking feedback and the actions that the firm performs to address the feedback. Lastly, the comparison between CG2 and the Control Group’s customers will provide us with the impact of only the actions that the firm performs in order to address the feedback they receive (without capturing the impact of the act of seeking the feedback). Figure 1 presents a schematic of the hypothesized mechanism as explained earlier.

We will collect detailed intervention and outcome data on participants’ firm performance (including sales, profits, firm actions and customer purchase behavior), on a monthly basis for our sample of 274 firms. The high frequency data collection for the firms in our sample should help us further improve the power of our experiment (McKenzie 2012). We shall also conduct an endline customer survey about 6 months post the start of the intervention. This survey will be a telephonic survey conducted with the customers of the firms as recorded in the Contacts+ application. This will help us capture the metrics from the customers’ side too (this way we can gain insights from both sides in the dyadic relationship between the firm and the customer).

Given our RCT research design, we can use our Treatment and Control variable at the firm level (i.e. randomly offered customer feedback seeking priming) as an orthogonal treatment variable and calculate the intention-to-treat (ITT) effects of improving access to customer feedback on performance outcomes (e.g. firm survival, sales, profits). In our analysis, we can also control for potential confounds using variables measured pre-treatment (e.g. gender, age, firm size, industry). Next we will do a similar analysis for the CG1, CG2 and control-customers’ groups in order to understand the mechanism of impact better. To analyze our final dataset, we can use a difference-in-differences (DID) approach with a panel dataset constructed from measuring the same set of firms (large N) over multiple periods (T > 2). This allows us to account for unobserved heterogeneity at the firm level, while still exploiting the randomization in our design. We can also analyze the data using analysis of covariance (ANCOVA), which has greater statistical power in an experimental setting (McKenzie 2012).

Randomization done on computer using Stata

Two levels of randomization - (1) Firm Level (2) Customer Level

No

Treatment is not clustered, we follow individual randomization where each individual observation is a firm in our case

(1) 274 firms, and (2) For the number of customers, that would depend on the number of customers who purchase from the firms in our sample during the period of the experiment.

137 firms in treatment and 137 in control