Towards a better understanding of the development of non-cognitive skills in children: Evidence from Bangladesh

We have a single treatment and a single control group. The 135 primary schools in our sample are divided into 69 treatment and 66 control group schools by stratified randomization. In cooperation with Lions Clubs International, we implement the social and emotional learning (SEL) program Lions Quest Skills for Growing (PreK-5) in grades 2 to 5 of the 69 treatment schools. The remaining 66 schools in the control group do not receive any treatment.

Implemented in more than 90 countries, Lions Quest is one of the most widely used SEL and prevention programs in the world. Typically, Lions Quest lessons take 45 minutes per week on top of the standard curriculum over a 33-week schedule where lessons usually take place in the classroom environment. Lions Quest is designed to build up children’s non-cognitive skills and positive attitudes. The non-cognitive skills in turn are supposed to promote academic success and civic engagement and to prevent risky behaviors and discrimination (for detailed information, please see https://www.lions-quest.org/elementary-social-and-emotional-learning/). Among others, the program’s schedule explicitly addresses self-control and self-discipline, patience, cooperative behavior, empathy, perspective-taking, relationship skills, avoidance of overly risky behaviors, self-confidence, and goal-setting. Skills are taught in presentations, discussions and group work. The Lions Quest Skills for Growing materials have already been translated into Bengali and culturally adapted to the context of Bangladesh.

2019-01-01

2019-11-15

Non-cognitive skills: risk, time, and social preferences, trust, Big Five personality traits, locus of control, self-control, self-esteem, IQ, outcomes at school (attendance, grades, test scores if available), life satisfaction, Strengths and Difficulties Questionnaire (SDQ), risky behaviors

Risk, time and social preferences are elicited using incentivized experiments. Each participant receives a token (star) as a show-up fee and is able to earn more stars during the experiments. At the end of all experiments, stars will be converted into money at an age-specific exchange rate. To avoid income effects, one of the experiments is randomly chosen for payment.

We use a popular experimental protocol to measure time preferences (e.g., Anderson and Mellor 2008; Bauer, Chytilová, and Pertold-Gebicka 2014) where subjects repeatedly choose between an early but smaller payment and a later but larger payment. For children, the early payment takes place either tomorrow or in a month, the later payment in three weeks, three months, or four months, respectively. For adults, delays are longer but equally structured. The experiment design allows classifying individuals according to their degree of patience (total number of patient choices among all choices), as time consistent or inconsistent (depending on whether their current and future discount rates are equal or not), and present biased (in case the current discount rate is larger than the future discount rate).

In order to measure risk preferences, we use an experiment originally designed by Binswanger (1980) and already widely applied (e.g., Bauer, Chytilová, and Pertold-Gebicka 2014) in rural settings in developing countries. Participants have to select one out of several gambles. Each gamble yields either a high or a low payoff with equal probability where each successive gamble is characterized by an increase in expected earnings and in the variance of earnings. Hence, higher number gambles are riskier, allowing to rank individuals in categories of decreasing risk aversion as well as classifying them as risk-averse, (close to) risk-neutral, or risk-seeking.

Concerning the social preferences, we adopt an experimental protocol inspired by Fehr, Bernhard, and Rockenbach (2008) and extended by Fehr, Glätzle-Rützler, and Sutter (2013) as well as Bauer, Chytilová, and Pertold-Gebicka (2014), for example. Each participant plays four binary dictator games. In each game, the child or adult has to decide between two alternative allocations of stars for himself/herself and an anonymous child or adult similar to him/her. The four games are the following, where in each allocation the first number corresponds to the decision maker’s own payoff: (i) costly pro-social game with (1,1) versus (2,0); (ii) costless pro-social game with (1,1) versus (1,0); (iii) costless envy game with (1,1) versus (1,2); and (iv) costly envy game with (1,1) versus (2,3). The design is suitable for classifying individuals to four social preference types: altruistic, egalitarian, spiteful, or selfish (see e.g., Fehr, Glätzle-Rützler, and Sutter 2013).

On top, for time, risk, and social preferences as well as for trust, we use one validated, age-adjusted survey item from the Global Preference Survey (Falk et al. 2016), respectively. Children have to indicate their agreement to the following statements using a 5-point Likert scale: “I am good at giving up something nice today in order to get something even nicer in the future.” (time preferences); “I often take risks.” (risk preferences); “One can trust unknown people.” (trust). Adults have to indicate their (un)willingness or (dis)agreement regarding the following questions/statements using an 11-point Likert scale: “How willing are you to give up something that is beneficial for you today in order to benefit more from that in the future?” (time preferences); “How willing are you to give to good causes without expecting anything in return?” (social preferences); “In general, how willing or unwilling are you to take risks?” (risk preferences); “I assume that people have only the best intentions.” (trust).

For the personality traits, further validated scales and survey items are used. Concerning the Big Five, for children aged 6 to 11, mothers answer a 10-item Big Five inventory (Weinert et al. 2007 based on Asendorpf and Van Aken 2003). Children of age 10 or older and adults answer a 15-item Big Five inventory derived from John, Donahue, and Kentle (1991) and evaluated in Gerlitz and Schupp (2005). Locus of control is measured with children answering 5 items related to external and internal locus of control (using a visualized 5-point Likert scale). For adults, 10 items have to be answered that are adapted from Rotter (1966) and used in the 2005 wave of the German Socio-Economic Panel. To elicit self-control, we employ the widely applied 13-item version of the Tangney self-control scale (Tangney, Baumeister, and Boone 2004) for children of age 12 and above as well as for adults, and for the younger children the Impulsivity Scale for Children developed by Tsukayama, Duckworth, and Kim (2013). We measure self-esteem using the 10-item Rosenberg self-esteem scale (Rosenberg 1965).

Regarding cognitive ability (IQ), we elicit one measure of crystallized and one of fluid IQ, which together form overall IQ (Cattell 1971). We measure fluid IQ using the matrix test of the well-established Wechsler Intelligence Scale for Children (WISC) or the Wechsler Adult Intelligence Scale (WAIS). For crystallized IQ, we use the vocabulary test for children and the corresponding word meaning test for adults that are both subtests of the respective Wechsler Intelligence scales, adapted to the specific context of Bangladesh.

We collect information on attendance, retention, dropout, grades, and, for grade 5, centrally administered tests for math and Bengali from the school records. Possibly, these will be combined with even more extensive data on children’s educational attainment if available. Furthermore, we collect information on teachers’ backgrounds that include their education, training and experience, as well as information on school infrastructure and management committee. Besides quantitative measures we gather qualitative data in order to understand possible pathways impacting outcomes.

To evaluate mental health, we use two complementary measures: First, a standard visualized life satisfaction 7-point Likert scale for children and a similar, 11-point Likert scale for adults. Second, mothers answer for their children the Bengali version of the Strength and Difficulties Questionnaire (SDQ) (Goodman 1997), a behavioral screening questionnaire that is a well-established tool for evaluating interventions. Its 25 items are divided between 5 scales: emotional symptoms, conduct problems, hyperactivity/inattention, peer relationship problems, and prosocial behavior.

While many typical risky behaviors like excess drinking, drug use, or teenage sex are not an issue in rural Bangladesh, we ask children from 10 years onwards and adults about their smoking and gambling behavior as well as more common risky behaviors such as climbing up trees and houses, or getting into physical fights, for example.


References:
Anderson, L. R., and J. M. Mellor. 2008. “Predicting Health Behaviors with an Experimental Measure of Risk Preference.” Journal of Health Economics 27 (5): 1260–74.
Asendorpf, J. B., and M. A. G. Van Aken. 2003. “Validity of Big Five Personality Judgments in Childhood: A 9 Year Longitudinal Study.” European Journal of Personality 17: 1–17.
Bauer, M., J. Chytilová, and B. Pertold-Gebicka. 2014. “Parental Background and Other-Regarding Preferences in Children.” Experimental Economics 17: 24–46.
Binswanger, H. P. 1980. “Attitudes toward Risk: Experimental Measurement in Rural India”. American Journal of Agricultural Economics 62 (179): 395–407.
Cattell, R. B. 1971. Abilities: Their Structure, Growth, and Action. New York: Houghton Mifflin.
Falk, A., A. Becker, T. Dohmen, D. Huffman and U. Sunde. 2016. “The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences.” IZA Discussion Paper 9674.
Fehr, E., H. Bernhard, and B. Rockenbach. 2008. “Egalitarianism in Young Children.” Nature 454: 1079–83.
Fehr, E., D. Glätzle-Rützler, and M. Sutter. 2013. “The Development of Egalitarianism, Altruism, Spite and Parochialism in Childhood and Adolescence.” European Economic Review 64: 369–83.
Gerlitz, J.-Y., and J. Schupp. 2005. “Zur Erhebung der Big-Five-Basierten Persönlichkeitsmerk-male im SOEP.” Research Notes 4, DIW, Berlin.
Goodman, Robert. 1997. “The Strengths and Difficulties Questionnaire: A Research Note.” Journal of Child Psychology and Psychiatry 38 (5): 581–86.
John, O. P., E. M. Donahue, and R. L. Kentle. 1991. The “Big Five” Inventory – Versions 4a and 54. Berkeley: University of California, Institute of Personality and Social Research.
Rosenberg, M. 1965. Society and the Adolescent Self-Image. New Jersey: Princeton.
Rotter, J. B. 1966. “Generalized Expectancies for Internal versus External Control of Reinforcement.” Psychological Monographs: General and Applied 80 (1): 1–28.
Tangney, J. P., R. F. Baumeister, and A. L. Boone. 2004. “High Self-Control Predicts Good Adjustment, Less Pathology, Better Grades, and Interpersonal Success.” Journal of Personality 72 (2): 271–324.
Tsukayama, E., A. L. Duckworth, and B. Kim. 2013. “Domain-Specific Impulsivity in School-Age Children.” Developmental Science 16 (6): 879–893.
Weinert, S., J. B. Asendorpf, A. Beelmann, H. Doil, S. Frevert, A. Lohaus, and M. Hasselhorn. 2007. “Expertise zur Erfassung von Psychologischen Personmerkmalen bei Kindern im Alter von Fünf Jahren im Rahmen des SOEP.” Data Documentation 20, DIW, Berlin.

We have a single treatment and a single control group, consisting of 69 and 66 schools, respectively (for details, see section “Intervention” above).

Randomization was done in an office by a computer, more specifically using STATA. In order to gain a balanced treatment-control group setting, we stratified by subdistrict (our schools are located in 11 subdistricts differing in their schooling authorities and hence their educational environment), by distance of the school to its respective subdistrict capital (as a proxy for schooling quality) and village literacy rate. No re-randomization was done but the first draw taken. Balance checks were run ex post for the stratification variables via two-sided t-tests (no significant differences at the 5%-level) as well as for wages, village population, distribution of educational levels, and student-teacher ratios (no significant differences either).

School level randomization

Yes

Treatment was assigned to 69 out of a total of 135 schools (hence, 66 schools in control group). These schools are serving 150 villages that we randomly selected from 11 subdistricts (chosen based on the availability of NGOs willing to collaborate) belonging to four main districts of Bangladesh.

3,000 households from the 150 villages that are served by the 135 primary schools in our sample; sampled by selecting at least 20 children from grades 2 to 5 of each school – larger samples from schools serving multiple sample villages, e.g. 40 children per school if the school serves two villages. We will have a sample that covers at least 3,000 pupils attending grades 2 to 5 plus information on at least one of their siblings (if they have siblings, the additionally sampled sibling is chosen randomly).

69 schools in treatment group (Lions Quest program gets implemented) and 66 schools in control group. In sum, 135 schools.

Reflecting the two-stage sampling procedure (subdistrict and village/school level), our power calculations largely follow Spybrook et al. (2011). We use STATA’s rdpower command that is tailored to computing the power for two-level cluster randomized designs. In our design, a cluster corresponds to a school that mostly coincides with the village level. Below, we report power calculations that we ran based on the assumption that school and village units coincide in our sample, i.e. that each of the 150 villages has a separate school. We only learned that in our sample 135 schools serve the 150 villages after we had run the power calculations. Since children in a village may be similar to each other in both observables and unobservables, the intra-class correlation (ICC) needs to be taken into account in power calculations. We also need information on cluster size, the number of clusters, the standardized effect size, and the significance level for hypothesis tests to estimate the power of our design. The power calculations assume a size of each cluster of 20 (i.e., 20 sampled households in each village) and as the total number of clusters 150. Following standard norms, we use a significance level of 0.05. We assume an effect size of 0.2 std. dev., based on our own previous work on the effect of mentoring on social preferences, trust, and the prosocial subscale of the SDQ (Kosse et al. 2016) and the meta-analysis of Durlak et al. (2011) on the effects of SEL programs on outcomes like self-esteem, SDQ, academic performance, or mental health. We compute estimated ICC using the previous data from the “non-intervention sample” (see abstract) in 2016 for children aged 7-11 that contain information on children’s Big Five personality traits, locus of control, time, risk, and social preferences only. The ICC varies from close to 0 for an indicator variable “non-risk averse” to 0.321 for locus of control. We ran power calculations for three different values of ICC. For the average ICC of 0.144, power to detect effect sizes of the SEL treatment with our proposed design is predicted to be 0.803, the level of power typically targeted. For the second lowest ICC of 0.06, power is very high (0.961); for the highest ICC of 0.321, power is low (0.533). Under these assumptions, our design would allow detecting effect sizes of 0.2 std. dev. for 6 out of 9 available measures of non-cognitive skills in 2016 in at least 80% of the time. References: Durlak, J. A., R. P. Weissberg, A. B. Dymnicki, R. D. Taylor, and K. B. Schellinger. 2011. “The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions.” Child Development 82 (1): 405–32. Kosse, F., T. Deckers, H. Schildberg-Hörisch, and A. Falk. 2016. “The Formation of Prosociality: Causal Evidence on the Role of Social Environment.” IZA Discussion Paper 9861. Spybrook, J., H. Bloom, R. Congdon, C. Hill, A. Martinez, and S. Raudenbush. 2011. “Optimal Design for Longitudinal and Multilevel Research: Documentation for the ‘Optimal Design’ Software.”