Can non-teaching role models reduce the gender gap in science?

The intervention consists in a one hour, one off visit of a high school classroom by a woman science career role model. The classroom visits are aimed at changing students' perceptions and attitudes towards scientific careers and towards women in science, with a view of ultimately reducing the gender gap in science by increasing female enrollment in science courses.

The classroom visits are performed by approximately sixty volunteer female scientists recruited by the program's sponsor (a large French company). Half of the scientists are young researchers of post-doctoral level, while the other half are young professionals working in the research and development department of the program's sponsor. All participants receive a one-day training session, which is divided into three workshops. The training begins with a roundtable, where each participant is invited to introduce herself and to share her experience about being a woman in science. The roundtable is followed by a detailed presentation of the intervention and of the evaluation protocol. The final part of the training consists of a four-hour performance-based public speaking coaching session under the supervision of a professional comedian.

Each one-hour classroom intervention takes place during normal teaching hours and is attended by all students in the class (females and males) in the presence of the teacher who has agreed to host the intervention. The female scientists begin their intervention with a slideshow that provides factual information about scientific careers in general, and about the underrepresentation of women in scientific studies and scientific occupations. This short introduction is followed by a three minutes video showing real-life students' stereotyped representations of scientists and of women in science in an entertaining way. For example, some students interviewed in the video claim that jobs in science are solitary, that women working in such occupations are unattractive, and that girls are not made for scientific careers. Students are then invited to react to these claims and to share their own views. During the second half of the intervention, the female scientist tells about her academic and professional background, shares her experience about being a woman in science, describes what a typical day is like for her, and answers any question that the students may have about scientific studies and careers.

The interventions take place in approximately one hundred public and private general high schools (Lycee d'enseignement general et technologique) located in the Paris region (Ile-de-France). The students selected for the classroom interventions are either in their first year of high school (Seconde generale et technologique, equivalent of 10th grade in the US) or final-year students preparing the scientific baccalaureate (Terminale scientifique, equivalent of 12th grade in the US). The choice of these two grade levels is motivated by the fact first- and final-year high school students need to take important and hard-to-reverse educational decisions which determine to a large extent whether or not they will pursue studies and careers in science or technology. At the end of their first year of high school, students who decide to prepare a general baccalaureate (Baccalaureat general) are required to choose between three different two-year tracks (11th and 12th grades) that lead to different baccalaureate specializations: in science (Baccalaureat S), in literature (Baccalaureat L) or in social and economic sciences (baccalaureat ES). Students who decide instead to prepare a technical baccalaureate (Baccalaureat technologique) must choose one of eight different fields of specialization, two of which put a strong emphasis on science subjects (Baccalaureat STI2D and Baccalaureat STL).

Classroom interventions are grouped into three one-hour sessions per high school, and are performed consecutively by the same female scientist during a half-day visit to the school. In each participating high school, the visited classrooms are two classrooms of first-year students and one classroom of final year students preparing the scientific baccalaureate.

2015-11-17

2016-04-15

A) Student attitudes towards science and the role women in science

Student perceptions about science and about the role of women in science will be measured through a paper-and-pencil questionnaire which will be completed by the control and treatment students in their respective classrooms, under the supervision of a member of the school staff. The survey questionnaire will be administered between one month and three months after the classroom interventions.

Key outcome variables:

A1) Self-assessed taste for and performance in science subjects

(i) Taste for the different subjects taught in high school (math, physics, French, English, history and geography, biology, etc.).
(ii) Self-assessed performance in the different subjects taught in high school.
(iii) Self-assessed performance relative to both the other boys and the other girls in the classroom: in math and French (for first-year students); in math and biology (for final-year students preparing thea scientific baccalaureate).
(iv) Self-confidence in ability to solve mathematics and science problems.

A2) Education and career plans

(i) Intended study plan for the following and subsequent years.
(ii) Career aspirations:
- preferred occupations (several responses allowed);
- whether the student would see herself /himself in particular occupations (engineer, physician, industrial designer, chemist, pharmacist, lawyer, researcher in biology, alternate energy technician, computer specialist, psychologist).
(ii) Motivations that influence educational and career choices: personal interest the field of study; fear that other fields of study might be too difficult; financial and employment prospects; gender balance; personal workload.

A3) Perceptions about science and scientific occupations

(i) Whether the student is interested in science in general.
(ii) Whether she/he considers that there are interesting jobs involving science.
(iii) The student sees herself/himself in a scientific career.
(iii) Jobs in science pay well.
(iv) Jobs in science require lengthy studies.
(iv) Jobs in science are monotonous.
(iv) Jobs in science are rather solitary.

A4) Perceptions about women in science

(i) There are more men than women in science.
(ii) Men are more skilled in mathematics than women.
(iii) Men and women's brains are different.
(iv) Women do not have much interest in science.

These items are designed to measure students' perceptions about the role of innate differences in ability ((ii) and (iii)), of differences in taste (iv) and of discrimination (v) when assessing gender differences in science.

A5) Descriptors for male/female scientists

Students' stereotypes representation of men and women with a science career are measured based on a list of positive/negative descriptors among which respondents are asked to choose to describe how they would portray a scientific person (interesting/boring, repetitive/creative, solitary/sociable, stylish/old-fashioned, respected/not respected, shy/outgoing, exemplary/ordinary). In each classroom, half the students are randomly assigned the question that refers to a man working in a science career, whereas the other half is assigned the question that refers to a woman working in a science career.

B) Educational outcomes

Information on students' post-intervention academic performance and choice of studies are obtained from administrative data sources that are linked with the survey data using an encrypted student identifier.

Key outcome variables:

B1) Academic performance across science subjects (for 12th grade students only)

12th grade students' academic performance in science subjects (mathematics, physics and biology) is measured from their grades in the corresponding final exams of the scientific baccalaureate.

B2) Choice of studies in the years following the intervention

(i) For 10th grade students: probability of preparing a baccalaureate in science (Baccalaureat S) or a technological baccalaureate with a major in science (Baccalaureat STL or STI2D) in 11th grade.

(ii) For 12th grade students preparing a scientific baccalaureate:
- Probability of applying to one or more undergraduate degree programs in science or technology (based on the data from France's centralized application system for higher education Admission Post-Bac).
- Probability of applying to one or more undergraduate degree programs in Science, Technology, Engineering and Mathematics (STEM) fields, as opposed to non-STEM scientific fields such as biology and medical sciences.
- Probability of enrolling in an undergraduate degree program in science or technology in the year following the intervention and in subsequent years.
- Probability of enrolling in an undergraduate degree program in a STEM field in the year following the intervention and in subsequent years.

Baccalaureate series and undergraduate majors will be grouped together to distinguish between science and non science fields of study. Within science majors, fields where women are the most underrepresented (e.g., math, physics) will be distinguished from other fields (e.g., biology, medical studies).
Some aggregated indexes may be built to summarize the information obtained from the questionaire's answers relating to a similar concept. Variables included in such indexes will be standardized according to standard procedures.

The one-hour classroom interventions will take place in approximately one hundred general track high schools located in the Paris region (Ile-de-France) during the school year 2015-2016.

The students selected for the classroom interventions are either in their first year of high school (Seconde generale et technologique, equivalent of 10th grade in the US) or final-year students preparing the scientific baccalaureate (Terminale scientifique, equivalent of 12th grade in the US). The choice of these two grade levels is motivated by the fact that first- and final-year high school students need to make important and hard-to-reverse educational decisions which determine to a large extent whether or not they will pursue studies and careers in science or technology.

In each of the three local education authorities covered by the program (Créteil, Paris and Versailles), public and private high schools were selected on a voluntary basis at the beginning of school year 2015-2016. A key challenge was to recruit high-schools to participate in the program. To minimize the number of journeys by the female scientists involved in the program, while ensuring that participating schools are reasonably representative, we first established the list of all 325 private and public general track high schools in the Paris region with at least four 10th grade classes and at least two 12th grade classes with a science major. At the start of the school year, the local education authorities--who officially support the program--informed the principals of those preselected high-schools about the program, its aims, and the evaluation protocol. Participation remained, however, non-mandatory.

Towards the end of September 2015, we contacted the principals of the pre-selected high schools and invited them to participate in the program's evaluation, after providing them with detailed information about the experimental design. With an approval rate of approximately 30 percent, we expect about one hundred schools to participate in the program.

In each participating school, the principals are invited to pres-select at least four 10th grade classes and two 12th grade /science major for the intervention. At least two of the pre-selected 10th grade classes and at least one of the pre-selected 12th grade classes are then randomly drawn by the evaluation team to host the intervention (treatment group). The other pre-selected classes are assigned to the control group.

The female scientists' interventions are grouped into two waves, the first during the first term of the school year (October-December 2015), and the second during the first half of the second term (January-February 2016). In each participating high school, the three classroom interventions (sometimes more) are scheduled during the same half day and are performed by a single female scientist, with two interventions for the two treatment classes of 10th grade students and one intervention for the treatment class of 12th grade students. The female scientists volunteering for the program commit to carry out at least half-day visits to the schools, representing a total of six classroom interventions.

The program's impact evaluation will rely on the combination of a post-intervention survey questionnaire and of several administrative data sources, which will be linked using an encrypted student identifier. The survey instrument is a 30-minutes 7-page paper-and-pencil to be completed by students from the treatment and control groups in their respective classrooms, under the supervision of a member of the school staff. The questionnaire consists of four main sections: (i) general questions about the student's home environment; (ii) questions on taste for and self-assessed performance in subjects taught in high school; (iii) educational and career pans; and (iv) perceptions towards science and attitudes towards women in science.

The questionnaire will be administered to all control and treatment classes of participating schools in two waves: (i) during the first half of February 2016, for approximately 30 percent of participating schools ; (ii) during the second half of March 2016, for the remaining 70 percent. Although the assignment of schools to either of these survey waves was primarily determined by the date of the classroom interventions, a subsample of schools which hosted the interventions during the first or second term of the school year was randomly assigned to either of the two survey waves. The aim of this randomization step is to test whether the program's impact on students' perceptions is sensitive to the time interval between the intervention and the survey.

As an attempt to distinguish between the "role model" and "information" components of the intervention, we randomly assign to some of the female scientists a version of the slideshow that does not provide information on wage differentials between STEM and non-STEM occupations. Due to logistical constraints, it is only possible to perform this randomization for relatively small subsample of female scientists whose first visit to a high school was scheduled during the first term of the school year (October-December 2015). We might therefore lack sufficient statistical power to detect meaningful effects regarding the specific contribution of information on STEM vs. non-STEM wage differentials on student outcomes.

The randomization of pre-selected classes to the treatment and control groups is done in office by a computer, as follows:
- create a unique identifier for each (high school, grade level) pair;
- for each pre-selected class, randomly draw a number between 0 and 1 from a uniform distribution, using the unique (high school, grade level) identifier as the seed for the sequence of pseudo-random numbers (for replicability);
- Within each (high school, grade level) cell, sort the preselected classes by ascending value of the random draw;
- Assign the k classes with the lowest value of the random number to the treatment group, where k= int[(n+1)/2] with n the number of pre-selected classes in the high school and grade level under consideration (in most schools, n=4 for 10th grade classes and n=2 for 12th grade classes).
- Pairwise matching: pairs of classes (one trated, one control) are also created within each (high school, grade level) by matching the treated classe having the lowest random draw with the control class having the lowest random draw, matching the treated class having the second lowest random draw with the control class having the second lowest random, etc. This pairwise matching is used to allow us to keep the maximum possible number of classes for the analysis in case tratment assignation has failed for some classes (in which case we only remove the corresponding pair). It may also be used for correcting standard errors.

The randomization of interventions is performed at the class-level within participating high schools and is implemented separately at the 10th grade level (Seconde generale et technologique) and at the 12th grade level (Terminale scientifique).

In each participating school, the school principal is invited to pre-select at least four 10th grade classes and at least two 12th grade classes for the intervention. The randomization is performed so that at least half of the pre-selected classes at each grade level are selected for the classroom intervention. Denoting n the number of preselected classes at a given grade level (either 10th grade or 12th grade), the number of classes assigned to the treatment group is int[(n+1)/2].

The survey design involves two additional levels of randomization:
- A subsample of schools with classroom interventions programmed for before January 15, 2016, were randomly assigned to either of the two survey waves, in order to test whether the program's impact on students' perceptions is sensitive to the time interval between the intervention and the survey. The randomization was performed at the school level, with two-thirds of the schools being assigned to the first survey wave (first half of February 2016), and the remaining third being assigned to the second survey wave (second half of March 2016).
- To control for potential biases induced by survey features, the order of answer choices was randomized for several questions. In addition, one question related to students' stereotyped representation of persons working in science occupations was randomly assigned across questionnaires, so that this question would refer to a male scientist in the questionnaires assigned to half of the students in each classroom, and to a female scientist for the other half.

Yes

The expected number of randomization units is 400 10th grade classes and 200 12th grade classes in approximately 100 high schools (usually four 10th grade classes and two 12th grade classes per school).

The planned number of observations is 18,000 students from 600 10th grade (50 percent females) and 12th grade/science major (40 percent females) classes in 100 high schools. Classes are evenly divided between treatment and control groups.

The principals of participating high schools provided us with a preselection of at least four 10th grade classes and at least two 12th grade classes. We then randomly draw half of them for treatment, and the other half for control. Pairs of trated/control classes are also formed (see above).

Minimum detectable effects are estimated for a 5 percent level of significance and 80 percent power level. The computations take into account clustering effects at the classroom level and at the school level, and use plausible values for the intra-class correlation coefficients of 0.1 at the classroom level of 0.3 at the school level. With 100 high schools each including two treated/two control 10th grade classes of 30 students (of whom 50 percent are females), and one treated/one control 12th grade classes (of whom 40 percent are female), we estimate a minimal detectable effect size (MDES) (without controlling for covariates) of approximately 10 percent of a standard deviation for 10th grade students and of approximately 14 percent of a standard deviation for 12th grade students. The corresponding MDESs for the subsample of female students are 11 percent and 16 percent respectively. In the absence of baseline estimates for our outcomes measures of students' perceptions towards science and of attitudes towards women in science, we restrict our computation of minimum detectable effects in natural units to the MDEs for the probability of choosing a science major in the year following the intervention. The MDE for the probability that a 10th grade female students enrolls in a science major in 11th grade is approximately 5 percentage point from a baseline of about 33 percent, which represents a 15 percent increase. This implies that we should be able to detect a treatment effect corresponding to three female students in every two treated classes switching from a non-science to a science major as a result of the intervention. The MDE for the probability that a 12th grade female students enrolls in a undergraduate program with a science major is approximately 7 percentage points from a baseline of about 28 percent, which represents a 25 percent increase This implies that we should be able to detect a treatment effect corresponding to three female students in every four treated classes switching from a non-science to a science major as a result of the intervention.