200SMEchallenge: Design-driven Open Innovation Challenge for 200 SMEs

Last registered on October 08, 2020

Pre-Trial

Trial Information

General Information

Title
200SMEchallenge: Design-driven Open Innovation Challenge for 200 SMEs
RCT ID
AEARCTR-0006246
Initial registration date
October 08, 2020

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
October 08, 2020, 7:26 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Region
Region
Region
Region
Region
Region
Region

Primary Investigator

Affiliation
FBK-IRVAPP

Other Primary Investigator(s)

PI Affiliation
HIT – Hub Innovazione Trentino

Additional Trial Information

Status
In development
Start date
2020-10-07
End date
2021-05-31
Secondary IDs
Abstract
Pursuing optimal design and user experience of digital products is key for companies to stay competitive in the market. User-centered design techniques inspired by design thinking, such as the “Design Sprint”, have the potential of improving the quality of digital products design. Yet, many SMEs are not aware of the added value of these techniques and are not equipped to adopt them. The 200SMEchallenge project aims at building evidence on the effectiveness of a two-day Design Sprint initiative coming in the format of a “UX Challenge” (User eXperience Challenge) on SMEs’ awareness and readiness to adopt user-centered design. More precisely, the project addresses the following evaluation question: for SMEs operating in the digital industry sector or other SMEs that develop products bearing digital interfaces, does participating in the UX Challenge, rather than not participating, enhance knowledge, awareness and intention to adopt the innovative approaches in the design of digital products? The evaluation features a randomized controlled trial that will be conducted in Winter-Spring 2021 and will involve 200 SMEs based in seven EU regions. The impact of the UX Challenge will be assessed on SMEs’ Digital Design Readiness and Awareness, measured with a set of survey-collected indices on knowledge, attitudes and planned actions related to digital design approaches. The project is funded by the European Commission (Grant Agreement number 824212 — 200SMEchallenge — H2020-INNOSUP-2018-2020).
External Link(s)

Registration Citation

Citation
Azzolini, Davide and Nicola Doppio. 2020. "200SMEchallenge: Design-driven Open Innovation Challenge for 200 SMEs." AEA RCT Registry. October 08. https://doi.org/10.1257/rct.6246
Sponsors & Partners

Sponsors

Experimental Details

Interventions

Intervention(s)
The UX Challenge is scheduled in a two-day format and involve the following actors:
a) Companies: 8 SMEs operating in the digital industry sector or in other sectors but developing products bearing digital interfaces;
b) Solvers: are university students (including Ph.D. students) and young professionals (recently graduated students, e.g. up to 18 months from graduation) mainly with a background in UX design, interaction design and human-computer interaction (computer scientists, designers sociologists, psychologists, economists). Solvers are organized in teams, and each team is mentored by at least one senior mentor (a UX design professional);
c) Mentors: UX design professionals that will support teams in the execution of the UX Challenge. They may be either freelancers or affiliated with a design firm, or even a software company. The buy-in for them to take part in the Challenge is to get in touch with potential clients (SMEs) and potential new talents to hire;
d) Testers: are potential users or customers of the products selected in the Challenge. And at least 4 users have to be selected per product / team (plus one reserve). This way, if 8 companies are selected to the treatment group and will take part in the challenge, each partner will have to select some 40 citizens applying to 8 different profiles.

Day one is normally a Thursday and day 2 a Friday, but this can change depending on local conditions. There are slots that involve both the solvers and the companies (or the larger public), from those that involve the solvers only. One very important slot is the final event (a.k.a. “plenary”, during which all teams briefly pitch the results not only to companies, but to a larger public too).

The UX challenge is organized in five consecutive phases.
• Phase 1: Scoping the Challenge.
* Phase 2: Ideating the solutions.
* Phase 3: prototyping.
* Phase 4: Test.
* Phase 5: Tune and deliver.

NB: Due to the ongoing Covid-19 pandemic, the above description of the UX Challenge is subject to change.

Intervention Start Date
2021-01-17
Intervention End Date
2021-05-31

Primary Outcomes

Primary Outcomes (end points)
Digital Design Readiness and Awareness (DDRA): knowledge, attitudes and behaviors related to digital design approaches in SMEs' activity
Primary Outcomes (explanation)
DDRA will be measured through three specific outcomes:
● Knowledge of user-centered design & design sprint
● Attitudes towards user-centered design
● Intentions to adopt user-centered design

The three outcomes will be measured through batteries of items and will be analyzed independently and no single composite index will be computed.

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The trial follows a parallel 2 arms design:
● Treated companies: receive access to the UX Challenge and post-experiment webinar;
● Control companies: have no access to the UX Challenge but can access the post-experiment webinar.


Experimental Design Details
Randomization Method
Randomization will be done in office by a computer using the statistical software STATA.
Randomization Unit
The unit of randomization is the single selected SME. If a company applies with more than one product, the company will enter the randomization procedure only with the product that is regarded as the most suitable.
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
0
Sample size: planned number of observations
200
Sample size (or number of clusters) by treatment arms
Allocation ratio: .28 (T= 56; C= 144). A larger control group is planned to increase the experiment’s statistical power.
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Sample size is fixed, as it is constrained by the budget available for the intervention. The size of the control group is planned as being larger than the treatment group, deviating from an equal distribution of treated and controls, to gain higher statistical precision. Since estimates of the variance of our outcome variables are not available, power analysis is calculated both in terms of minimum detectable effect size (in case of continuous variables) and dichotomous variables (assuming the worse scenario of a .50 distribution of the outcome). Estimated minimum detectable effect size is: .39 SD Estimated minimum detectable effect (dichotomous variable): .20 percentage points The assumptions made to calculate the minimum detectable effects of the experiment are: a) statistical significance level (p-value) = .05; b) statistical power: 80%; c) proportion of randomization units assigned to treatment: 28%. A number of tweaks in the RCT design are put in place to improve the experiment’s statistical power: 1) using a larger control group in order to increase sample size; 2) employ a stratified randomization based on companies' location and turnover or sector; 3) statistical analysis of the treatment effects based on regression-adjusted estimation of the impact including covariates and pre-treatment outcomes.
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials