Secondary Outcomes (explanation)
The secondary measures we set out here are intended to allow us to explore mechanisms and wider impacts.
1. Attitudes towards chatbots/marketing automation technologies
Variable: Answers to a series of questions such as “Please indicate whether you agree with the following statements: chatbots of marketing automation technologies would lead to increased (a) revenues (b) profitability (b) customers” score 1-5, 1=strongly disagree, 5=strongly agree. This variable can be analysed both as a level at endline and as a change from baseline.
2. A measure of the change in intentions to adopt chatbots / marketing automation technologies
Variable: Answer to the question “Are you now more likely to adopt chatbots / marketing automation technologies than before the intervention?” Binary variable 0,1. This variable would be useful in the case of low adoption rates, but where a high proportion of SMEs state that they are likely to adopt in the baseline survey. At endline, it will explicitly ask SMEs to consider whether they have become more likely to adopt during the course of the programme.
3. Broader adoption of technologies or innovative organisational practices/ decision to adopt
Variable: Answer to “have you adopted, or taken the decision to adopt any other innovative technologies or innovative organisational practices over the past 6 months? Binary variable 0,1.This variable can be analysed both as a level at endline and as a change from baseline, and will give us information on whether a firm has moved into/ committed to a decision / implementation in a broader sense. Where firms indicate that they have taken the decision to adopt we will ask some more specific questions about the commitment mechanism (e.g. signed off by management/in business plan etc.).
4. Self-reported performance measures (short term): ln(turnover), ln(turnover per employee), number of customers – we would ask for the average of turnover or customers over the last 3 months.
The technologies within scope of this project seek to increase revenues by increasing the number of customers or better converting enquiries to sales. Therefore, revenues and number of customers are the most appropriate performance variables to measure. Normalising by employment, to give a productivity measure seems sensible. We will also consider impacts on employment itself.
We note that we expect that the impacts of technology adoption on firm performance will occur over a longer timeframe than that captured by the endline survey. Nevertheless, we will obtain basic self-reported performance measures (revenues, customers, employees) to conduct some exploratory performance analysis. In our analysis, we will take the natural log of turnover or turnover/employees as these variables tend to be highly skewed. Longer term analysis (beyond the scope of the initial evaluation report) will seek to analyse impacts by tracking firms in administrative data.