|
Field
Primary Outcomes (End Points)
|
Before
1- Course Performance: Exam grades. Within several days after each short writing assignment’s grades are posted, students will be taking exams consisting of 40 multiple choice questions.
2- Subsequent Educational Decisions: Major choice, course enrollment in future semesters. We will request from the University Registrar students’ declared major and course enrollment in the semester following the experiment.
|
After
1- Course Performance: Exam grades. Within several days after each short writing assignment’s grades are posted, students will be taking exams consisting of 40 multiple choice questions.
2- Subsequent Educational Decisions: Major choice, course enrollment in future semesters. We will request from the University Registrar students’ declared major and course enrollment in the semester following the experiment.
3- Grade disputes: Students are allowed to reach out to the instructor if they would like to ask for a regrading of any part of their assignments. We collect information on what part of their assignment they challenge their grades and whether it was a success as well as the text of the email sent to the instructor to obtain sentiment scores in these texts.
|
|
Field
Experimental Design (Public)
|
Before
This is a field experiment conducted in two large principles of economics courses at a large, comprehensive university. Students will be asked to complete 3 short writing assignments (SWAs), which are short essays answering some content-relevant prompt. All prompts have an objectively correct answer in the rubric provided to all graders, but partial credit can be awarded at the discretion of the grader. For each assignment, students are also asked to provide a grade expectation, the honesty of which is incentivized by providing extra credit for accuracy.
Once the assignment due date has passed, students’ assignments are randomly assigned to a grader. Graders are responsible for only grading and do not interact with students in any way. Grading is double-blind such that neither student nor grader is aware of the identity of the other. Graders are provided a standard rubric of ten elements, all valued at ten points each. The first seven are objective and related to the SWA prompt. The final three elements are concerned with grammar, typos, and writing quality. All graders are given guidelines on the grading of these elements at the start of the semester. However, all entered grades are at the sole discretion of the grader and are never altered by anyone other than the grader or the instructor. Once all assignments have been graded, all detailed rubrics are downloaded from the LMS before being posted. Students are immediately notified of posted grades by the LMS. At the same time, the rubric is available to students so that they can review the points they lost, if any. Once posted, the assigned grade is no longer double-blind as the student is now able to view the grader’s name.
Students are permitted to contest a grade only by contacting the course instructor. Graders are never permitted to interact with students and are instructed to forward any student communication to the study coordinator. The instructor has the sole discretion to alter any submitted grade. Instructors maintain a record of all grade contests, and any alterations are flagged when rubrics are downloaded at the end of the semester.
Several days after each short writing assignment, students complete a 40-question multiple choice exam covering the same material as the preceding SWA. The final question of each exam asks students to report the number of questions they expect to get correct. Similar to the SWAs, honesty is incentivized by providing extra credit for accuracy.
At the end of the semester, each SWA will be regraded blindly by two additional randomly-assigned graders.
|
After
This is a field experiment conducted in two large principles of economics courses at a large, comprehensive university. Students will be asked to complete 3 short writing assignments (SWAs), which are short essays answering some content-relevant prompt. All prompts have an objectively correct answer in the rubric provided to all graders, but partial credit can be awarded at the discretion of the grader. For each assignment, students are also asked to provide a grade expectation, the honesty of which is incentivized by providing extra credit for accuracy.
Once the assignment due date has passed, students’ assignments are randomly assigned to a grader. Graders are responsible for only grading and do not interact with students in any way. Grading is double-blind such that neither student nor grader is aware of the identity of the other during grading period until grades are posted. Graders are provided a standard rubric of ten elements, all valued at ten points each. The first seven are objective and related to the SWA prompt. The final three elements are concerned with grammar, typos, and writing quality. All graders are given guidelines on the grading of these elements at the start of the semester. However, all entered grades are at the sole discretion of the grader and are never altered by anyone other than the grader or the instructor. Once all assignments have been graded, all detailed rubrics are downloaded from the LMS before being posted. Students are immediately notified of posted grades by the LMS. At the same time, the rubric is available to students so that they can review the points they lost, if any. Once posted, the assigned grade is no longer double-blind as the student is now able to view the grader’s name. Grader's names seen by students are randomly assigned to a generic female and male sounding names (Amanda, Emma, Eric, James, Jessica, John, Katherine, and Michael).
Students are permitted to contest a grade only by contacting the course instructor. Graders are never permitted to interact with students and are instructed to forward any student communication to the study coordinator. The instructor has the sole discretion to alter any submitted grade. Instructors maintain a record of all grade contests, and any alterations are flagged when rubrics are downloaded at the end of the semester.
Several days after each short writing assignment, students complete a 40-question multiple choice exam covering the same material as the preceding SWA. The final question of each exam asks students to report the number of questions they expect to get correct. Similar to the SWAs, honesty is incentivized by providing extra credit for accuracy.
After the assignment grades have been posted for the first and second assignments, students will be asked to provide a rating of their perceived fairness of the each grade they were assigned (1 to 5 where 1 is “Very Unfair” and 5 is “Very Fair”). This response is not incentivized. On the final exam, students will be asked to provide a rating of their perceived fairness of the overall grading in the course (1 to 5 where 1 is “Very Unfair” and 5 is “Very Fair”).
At the end of the semester, each SWA will be regraded blindly by two additional randomly-assigned graders.
|