Identifying management skill

Sessions will be randomly assigned to one of two conditions.

2023-10-26

2023-12-17

The primary outcome of interest is how well groups perform on a novel ‘allocation task’. Specifically, we are primarily interested in group performance conditional on the individual task-specific skills of their group members. Our proposed analysis and framework is described in Section 3 of the detailed analysis plan. The detailed analysis plan also includes a description of the group allocation task.

Group performance is assessed using the Group Allocation Task. The scoring for the task is described in the detailed analysis plan at the end of this document. The primary outcome of interest is “group efficiency”. This can be conceptualized as whether a group over- or under-performed controlling for the individual skill level of its members. See the Statistical Analysis Plan for details.

NA

NA

NA

This experiment consists of two elements: individual testing and group testing.

The individual tests happen at two different points in time: before and after the experiment. We present the tests here in brief. For details, see the detailed Statistical Analysis Plan (presented as a non-public appendix):

- Individual assessments taken before the group-work task (tests with an asterisk are taken at home; other tests are taken at the lab before the group-work begins)
o The Assignment Game (AG)*
o Emotional perceptiveness test*
o IQ or fluid intelligence test (CFIT)*
o Measure of individual task-specific skills (numeracy/analytical/spatial questions)
o Measure of willingness to be a manager (as opposed to a worker)

- Individual follow-up tests taken after the group-work session
o Demographic questionnaire
o Political Skill Inventory
o Indecisiveness Inventory
o Questionnaire about which skills are most useful in managers
The first phase of testing will involve participants completing 3 assessments at home (the Assignment Game, a test of emotional perceptiveness and the CFIT.

Lab sessions will be randomly assigned to one of two conditions: ‘self selection’ and ‘no choice’. The assignment will take place in blocks of 2 sessions (so that, within each pair, one session will be ‘self selection’ and the other will be ‘no choice’).
This treatment determines how participants’ roles will be assigned. There are two roles in the experiment: ‘manager’ and ‘worker’ (the latter is labelled as “team members” in the experiment). Participants in the self-selection condition will rate their preference for being a manager (on a scale of 1-10). Those with the strongest preference are assigned to be the manager (if there is a tie, we will randomly assign roles until all the manager places are filled). Those in the ‘no choice’ condition role are also asked for their role preferences but this preference does not impact their role assignment. Indeed, in the no choice condition roles will be randomly assigned.

Upon entering the lab subjects input their individual ID. They will then complete individual tests of task-specific skills (numeracy/analytical/spatial). If participants have randomly been assigned to a ‘self-selection’ session, they will then express how willing they are to take the role of ‘manager’ in the group assessments. If they are randomized to a ‘no choice’ session, they similarly express how willing they are to take the role of ‘manager’ in the group assessments.

Once each participant is given a role, this is fixed for the duration of the session. In total, sessions will last around 2 hours (including the individual assessments and manager preference elicitation). Over the course of the session, individuals will work in 4 different groups, each containing 3 participants (1 manager and 2 workers). Each group is formed by randomly assigning two workers to a manager. Assuming sessions of 15 subjects, each worker will be placed with a manger at most once. Within each group, subjects complete the Group Allocation Task, described in the appendix.

Overall, we anticipate that participants who fully participate in the experiment will spend 2.5-3 hours.

Randomization will take place via computer. There are two levels of randomization:
- Randomization of sessions to treatment or control and
- Randomization of groups.

We explain each in turn.

Randomization of Role Assignment Mechanism
We randomly allocate each session to either be a ‘self selection’ or ‘no choice’ treatment. Sessions will be equally split across these conditions, as we will use pair randomization. Pairs will be sessions that are closest together in time.

Randomization of Groups
To run a study ‘session’ there must be at least n=12 participants. Ideally, there will be n=15 participants at a time. Workers are randomly assigned to managers, with two constraints: if possible, we do not consider groupings in which workers are allocated to a manager they have worked with before. Second, workers are allocated to a manager at most once (e.g., when n=15, they are allocated to each manager at most once). See detailed analysis plan for further details. The ordering in which groups are matched to managers are randomized.

When the number of participants who show up to a session is not divisible by 3, we randomly select participants to withdraw until we have the correct number. Participants who are withdrawn are paid an appearance fee for their time.

The ‘role selection’ treatment will be randomized at the session level so that each session only contains a single treatment or control. Within each sessions subjects will be randomized into groups of three. This re-randomization of groups occurs five times following the procedure outlined in detail in the Appendix.

Yes

We plan to conduct around 30 sessions.

We plan to conduct the experiment until we reach 402 individual subjects. As we aim for each session to contain a maximum of 15 subjects (and, may contain as few as 12) we expect we will need around 30 sessions. However, we will conduct further/less sessions until we reach our goal of 402 individual subjects.

We plan to split sessions equally between the ‘self selection’ and ‘no choice’ arms. We aim to conduct around 15 no choice sessions and 15 self selection sessions, which will consist of approximately 201 subjects in each arm.

Given the novelty of our design – coupled with the fact that we are exploring new measurement instruments – our power calculations are extremely uncertain. We have set our sample size based on the sample size used in the only comparable experiment we are aware of (Weidmann and Deming, 2021). Weidmann and Deming (2021) also use repeated randomization and a similar estimation strategy. However, as we are using a different, and newly devised group task, we are unable to use their data to perform precise power calculations.