Goal-setting, effort, and performance: Evidence from a field experiment in a physical education course

2019-04-06

2020-06-30

Task performance: number of successful rope skipping jumps over a duration of 1 minute; corresponding grade score; number of failure times during the test
Effort measurements: changes in heart rate, oxygen saturation, respiratory rate, self-reported level of physical fatigue before and after for each test and some training tasks; calories expended during the test; maximum number of completion for some other training tasks.

Exercise frequency and intensity measures collected through card swiping records in the playground, gym and swimming pool: total/ valid length, total / valid times per semester, ranking among the undergraduates.
Physical fitness test results: undergraduate students are required to participate in the Physical fitness tests every year by the Ministry of Education, including BMI, Lung capacity, 50 meters running, sitting body flexion, standing long jump, pull-ups (male) / 1 minute sit-ups (female), 1000 meters running (male) / 800 meters running (female).
Health outcomes collected from the university hospital: number of times seeing doctors, when and in which department, number of times of being referred to a hospital of higher rank, medical payment, diagnosis and treatment plans, per semester.

About the data on student health outcomes collected from the on-campus hospital: all formally registered students at BNU receive compensation from their medical insurance only if they see doctors in the on-campus hospital first. If the need arises, they will then be referred to off-site hospitals.

Students, who take part in a graded, college physical training classes, are randomly assigned to one of three different types of goal-setting. We test for performance difference in a physical sport task across these treatments.

Our field experiment is conducted in a physical education course at Beijing Normal University (BNU), China. During their four years of undergraduate study, students are required to take at least four semester-long sports courses, which count towards their final GPA.
We were granted a fund to conduct an education reform project from BNU’s academic affairs office.
Since students are aware of such projects and have participated in them in various other courses before, it is possible for us to introduce new features to the course and ask students to answer surveys while preserving key features of a natural field experiments, i.e. subjects make regular decisions in real-world environment without doubting that a scientific study is undergoing.

Task
Over the course of four weeks, students train to improve their performance of a physical task (rope skipping). At the end of this period, their performance is tested, with their final grades increasing in their absolute performance. In addition to the final test, we also test the students in week 1, 2, and 3. The students’ performance on these tests does not affect their final grade but serves only as feedback for their progress. The grading scheme is made available to the students at the start.

Treatments
Students are randomly assigned to one of the broad-goal (BG), narrow-goal (NG), or updated-goal (UG) treatments. In the BG treatment, students are asked to set a performance goal for the final test after learning their week 1 test results. In the NG treatment, they must also set goals for the progress tests in week 2, 3 at the end of week 1. In the UG treatment, each week, students set goals for the upcoming test after learning about their previous test performance.

Experimental procedure
In each of the four weeks, students first perform a test of rope skipping, get informed of the test performance, set goals according to their respective treatment (only in week 1 in BG and NG, in weeks 1-3 in UG), then participate in the training session.

We start our manipulation when students can no longer switch the course for another. Students are informed that the course is part of an education reform project with the aim to improve teaching quality, but are blind to the experimental conditions other than their assigned ones and are not told of the existence or the purpose of the study. More specifically, students are told [1] that an education reform project assistant will add them on the Chinese messaging platform “WeChat”, and [2] that they will be required to answer a number of online surveys. The project assistant will use WeChat to conduct surveys through which they set goals and inform them of their test-score, promising the students that their responses remain anonymous and confidential from fellow students, course instructors and teaching assistants. Hence, the instructor and teaching assistants are also unaware of the specific treatment assignment.

In view of the number of students that take this physical education course each year, it is unfortunately not possible to run more than 3 treatments in the same term. As a result, we chose not to conduct a pure control treatment without goal-setting. We intend to run this treatment at a later stage.

randomization done in office by a computer. We employed a stratified randomization by class and gender.

The unit (clusters) of randomization in our experiment is at the individual student level.

No

113 students

113 students in the physical training course in the Spring semester in 2019, each of whom has four weeks’ test and four training session data. Possibly adding students from the same course in subsequent semesters.

there are 30-40 students in each of the three treatments