Experimental Design Details
Setting. The setting is one of the largest European kitchen manufacturer. The family-owned firm employs 3,700 workers, who produce more than 750,000 kitchens each year. All kitchens are planned (e.g., locations of kitchen appliances and cupboards, size, materials) by independent kitchen stores, who act as brokers for different kitchen manufacturers. The kitchen stores submit ready-planned kitchens to the study firm, which produces and delivers the kitchens.
In the firm, almost all kitchens are unique products that are tailored specifically to customers’ preferences. Given the complexity of the product, it seems to be no surprise that almost all kitchen orders that the firm receives from kitchen stores have at least one planning error. To detect and fix those errors, the firm has a department in which workers constantly check and partly re-design all kitchens right after receiving an order. Although many errors are detected and fixed, a large share of the kitchens is still defectively produced and installed. Overall, the firm receives for one third of the installed kitchens at least one customer complaint.
In our project, we focus on workers in the order processing department; the job of the workers is to check all kitchen orders. The workers are organized in 29 teams.
Pre-experimental survey. In preparation for our joint project, we conducted a survey among the workers in the order processing department in autumn 2020. The findings from the survey suggest the following.
First, peer-based knowledge sharing is one of the most important channels through which workers acquire their knowledge about kitchen planning.
Second, workers rarely interact with co-workers from other teams and they believe that other teams detect on average 50% less planning errors than themselves and their co-workers from their own team.
Third, workers from different teams share knowledge within, but not across teams.
In response to the results from the survey, the top management and our research team decided to introduce quality circles to foster knowledge sharing between workers. The decision was made in November / December 2020, when peer-based knowledge sharing between workers became increasingly difficult as employees had to work from home because of the Covid-19 lockdown.
First intervention. In January 2021, the company introduced quality circles for 14 out of the 29 teams in the order processing department. In the quality circles, the teams met for up to 30-45 minutes and discussed about planning errors.
The teams were randomly selected; workers from the 14 teams who participated vs. workers from the 15 teams who did not participate are balanced across a number of dimensions, such as the number of planning errors.
The quality circles were conducted online, using the virtual whiteboard “MIRO”.
Each quality circle was moderated by at least one member of an expert team consisting of four experienced workers from the study firm. For organizational reasons, this expert team could “only” chair two quality circles per day.
In January to March 2021, each of the 14 teams participated in six quality circles; sub-sequentially, the quality circles ended for those teams and we introduced quality circles for the remaining 15 teams between April and June 2021.
To analyze the mechanism of the treatment effects and identify potential heterogeneous treatment effects, we collect detailed data on the behavior of the workers during the quality circles (e.g. the number of post-its posted on the virtual whiteboard by the workers), discussion topics and knowledge shared during the circles.
Main intervention. In mid of August 2021 we start a new round of quality circles for 15 groups of workers. In contract to the first intervention, in the main interventions the individual workers who participate in the quality circles will be randomly selected individuals from all 29 teams (two teams are not part of the main intervention). The randomization will assure that workers of existing teams will not meet in these 15 newly formed groups.
In the quality circles, groups of workers will meet for up to 30-45 minutes and discuss about planning errors. The quality circles will be conducted online (using the virtual whiteboard “MIRO”) and will be repeated six to seven times over a period of a couple of months.
The composition of the groups of worker will remain constant over time.
First vs. main intervention. The major difference between the first and the main intervention is that the quality circles in the first intervention are conducted within, and in the main intervention across the existing teams in the order processing department.
Heterogeneous treatment effects. We expect the treatment effects to be heterogeneous in our main intervention on the following dimensions.
1. Tenure. We expect that workers who have less experience and know-how regarding detecting kitchen planning errors have more wiggle room to improve their performance.
2. Pre-treatment performance (quality and quantity). Here, we expect countervailing effects. On the one hand, workers with lower pre-treatment performance have more wiggle room to improve their performance. On the other hand, high-performing workers might be more ambiguous to reduce the errors that they made.
3. Order complexity. We expect the treatment effect to be larger for more complex kitchens in which planning errors are more common.
4. Team characteristics. Several teams have “dominant” team members who might stop co-workers from discussing about planning errors. We expect the treatment effects to be large from workers from those teams.
5. Survey measures. In our pre-treatment survey, we measured social interaction and knowledge sharing in teams; in a post-treatment survey, we will measure workers’ reciprocity. We expect the treatment effects to be larger for workers from teams in which knowledge sharing and social interaction is / was less common and in which workers are more often reciprocal.
Data protection. To ensure that workers’ anonymity is preserved in the project, we set up an elaborate privacy protection process with several “Chinese walls” between different datasets that were handled by different researchers and research assistants. The privacy protection process was approved by an Ethical Review Board.
Number of Observations. The company employs around 280 workers in the order processing department. The exact number of workers varies slightly over time, as workers constantly start working for the firm or quit. At the beginning of the main intervention, the number of workers in the treatment group is 142.