Effects of failure in a high-stakes application procedure on confidence in abilities and willingness to compete

Using the central application procedure for places in the medical fields in Germany, we study how the (non-)assigment of a study place through this procedure influences ability beliefs and the willingness to compete in an application-relevant ability domain. Medical studies are some of the most prestigious study fields in Germany. Each year, only about 1 out of 6 applicants receives a study place.

In particular, we ask the following questions: (i) How does failure (relative to success) in the application procedure influence the willingness to compete in an application-related ability test? (ii) How does failure (relative to success) in the application procedure influence absolute and relative confidence in application-related abilities? (iii) Is there a gender difference in willingness to compete and in confidence? Does failure in the application procedure (relative to success) have heterogeneous effects on the willingness to compete and confidence by gender, ability, and famility background? (iv) Does positive/negative feedback about the ability test have heterogeneous effects on willingness to compete by gender, application success/failure, and the gender–application success/failure interaction?

Subjects will conduct the experiment via an online platform (LimeSurvey). Subjects will be randomly assigned to one of three feedback groups, and each subject takes decisions in four consecutive stages.

[Question wording:]

[Stage 1: ability test]

“Below we present you with 5 questions that are similar to those in the preparation tests.

The more questions you answer correctly, the higher will be the payout you will receive in addition. For your payout you can choose between the following two options:

Piece rate: If you choose this option, you will receive a fixed amount of money for each question you solve correctly, regardless of the performance of other participants.

Competition: If you choose this option, you will participate in a competition and your payment will depend on your performance compared to a group of three, randomly chosen, other participants who are also taking part in the competition. If you answer more questions correctly than the other three participants, you will receive 1€ for each question you answer correctly. If you do not answer the most questions correctly, you will not receive any additional payment. In case of a tie, the draw will decide.

You will now answer the questions first. Then you can choose between the two payout options.

[5 multiple choice questions with 5 answer categories each, testing skills in the following areas: 1. Spatial thinking (mentally rotating an object), 2. Physics (spectroscopy), 3. Chemistry (dispersion), 4. Physics (masses), 5. Statistics (probability of side effects)]

[Note: Participants will answer different test questions in the second wave.]


[next screen]

[Stage 2: choice between competition and piece rate]

Please now choose between the two payout options:

Piece rate: If you choose this option, you will receive a fixed amount of money for each question you solve correctly, regardless of the performance of other participants.

Competition: If you choose this option, you will participate in a competition and your payment will depend on your performance compared to a group of three, randomly chosen, other participants who are also taking part in the competition. If you answer more questions correctly than the other three participants, you will receive 1€ for each question you answer correctly. If you do not answer the most questions correctly, you will not receive any additional payment. In case of a tie, the draw will decide.

You choose in the list between the piece rate and the tournament. What changes from row to row is the amount you will receive if you choose the piece rate. To give you an incentive to choose your preferred option in each row , we will randomly determine one row . Your decision according to this row will then apply to your payment.

Also note that you will be given the opportunity to adjust your decision regarding the payout options later on. With a probability of 50%, your decision here determines which payout option applies to you. With a probability of 50%, your later decision determines your payout option.

[table with multiple price list begins:]
Piece rate: 0.20€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece rate: 0.40€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece rate: 0.60€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece rate: 0.80€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece rate: 1€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
[table ends]

[Stage 3: elicitation of confidence in abilities]

What do you think: How many of the 5 questions did you answer correctly? If you guess the number correctly, you will receive an additional 1 euro. Please enter a number between 0 and 5 here.

[Field for entering number here]

[Answer categories:]

Integers between 0 and 5


[ next screen]

What do you think, how many of 10 randomly selected other participants answered more questions correctly than you? If you estimate the number correctly, you will receive an additional 1 euro. Please enter a number between 0 and 10 here.

[Field for entering number here]

[Answer categories:]

Integers between 0 and 10

[Stage 4: feedback and opportunity to reverse choice between competition and piece rate]

You will now receive feedback from us on your performance with respect to the five questions in the form of a feedback score. The feedback score is calculated as follows:

Feedback score = number of your correct answers + random number X.

The random number X takes the values 1, 0 and +1, i.e. each of the values (-1,0,+1) is exactly equally likely (i.e. occurs with a probability of 33.3%). The score thus reflects your number of correct answers with a certain inaccuracy but in expectation corresponds to the actual number of your correct answers.
You will then have the opportunity to adjust your decisions regarding your payout options.

[next screen]

Your feedback score is Y.
As a reminder: Feedback score = number of your correct answers + random number X.

You now have the opportunity to correct your previous decisions regarding the payout options and can now choose again between the two payout options. With a probability of 50%, your decision here determines which payout option applies to you. With a probability of 50%, your earlier decision determines your payout option.

Piece rate: If you choose this option, you will receive a fixed amount of money for each question you solve correctly, regardless of the performance of other participants.

Competition: If you choose this option, you will participate in a competition and your payment will depend on your performance compared to a group of three, randomly chosen, other participants who are also taking part in the competition. If you answer more questions correctly than the other three participants, you will receive 1€ for each question you answer correctly. If you do not answer the most questions correctly, you will not receive any additional payment. In case of a tie, the draw will decide.

You choose in the list between the piece rate and the tournament. What changes from row to row is the amount you will receive if you choose the piece rate. To give you an incentive to choose your preferred option in each row , we will randomly determine one row . Your decision according to this row will then apply to your payment.

[table with multiple price list begins:]
Piece-rate: 0.20€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece-rate: 0.40€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece-rate: 0.60€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece-rate: 0.80€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
Piece-rate: 1€ per correct answer guaranteed. -- Competition: 1€ per correct answer, if you answer the most questions correctly.
[table ends]


TREATMENTS:

The random number X added to the number of correct answers to give feedback score Y takes the values -1, 0, 1, which are randomized at the individual level and have equal probability to occur.

2022-06-23

2022-10-31

Competition entry
Absolute confidence: belief about number of correct items
Relative confidence: belief about relative performance
Competition entry after positive, neutral, or negative feedback

Performance in application-related ability test
Intention to re-apply
Self-reported competitiveness
Self-reported willingness to take risks
Goal setting for medical occupations that differ in their status and gender-dominance

About 3850 individuals will be invited to participate in both survey waves before and after learning the outcome of their application for a study place in medical studies and about 3850 will be invited to participate only in the survey wave taking place after learning the outcome. We expect a response rate of about 65% each, amounting to about 2500 individuals in each group. The survey is conducted in cooperation with the Studierendenauswahl-Verbund (stav). They send out invitations to participate in the survey containing individualized links. The polling is managed by the research team and hosted by the WZB Berlin Social Science Center, who collect the data via LimeSurvey. We guarantee full anonymity to our survey participants who answer the survey questions autonomously on their own digital devices. We ask for permission to recontact them and collect their e-mail addresses to do so. We also collect their bank and/or PayPal information to send them the participation rewards. Randomization is carried out using the survey software LimeSurvey at the individual level.

Our experiment is structured as follows:

Subjects will conduct the experiment via an online platform. Subjects will be randomly assigned to one of three feedback groups, and each subject takes decisions in four consecutive stages.

Stage 1: ability test
Stage 2: choice between competition and piece rate
Stage 3: elicitation of confidence in abilities
Stage 4: feedback and opportunity to reverse choice between competition and piece rate

Randomization is carried out by the survey software LimeSurvey.

at the individual level

No

2,500 in wave 1 and 5,000 in wave 2

2,500 in wave 1 and 5,000 in wave 2

2,500 adults in wave 1 (5,000 in wave 2) aged 17 years and older, approx. 1/3 will be assigned to each of the three feedback groups.