Assessing Opportunities for Solar Lanterns to Improve Educational Outcomes in Rural Off-Grid Regions

Last registered on August 06, 2018

Pre-Trial

Trial Information

General Information

Title
Assessing Opportunities for Solar Lanterns to Improve Educational Outcomes in Rural Off-Grid Regions
RCT ID
AEARCTR-0003210
Initial registration date
August 02, 2018

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
August 06, 2018, 12:12 AM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
Stanford University - Program on Energy and Sustainable Development

Other Primary Investigator(s)

PI Affiliation
Stanford University
PI Affiliation
Stanford University

Additional Trial Information

Status
Completed
Start date
2016-02-01
End date
2017-03-01
Secondary IDs
Abstract
Most rural areas in Sub-Saharan Africa have underperforming schools and tend to also be energy poor. As a result, solar lanterns have recently been promoted across the region to improve both lighting in homes and educational outcomes. We undertake a randomized controlled trial in Zimba District, Zambia, to evaluate whether solar lanterns help children study more effectively and improve academic performance. We find no evidence that receipt of a solar lantern improved performance on important national examinations (even though an ex post statistical power analysis demonstrates that our research design should detect economically significant impacts, if present). We also do not observe impacts on self-reported study habits. Several features of Zimba District that are likely to exist in other developing regions appear to drive our results. First, flashlights have become the dominant lighting source in rural Zambia rather than traditional options like kerosene lamps or candles. In such environments, solar lights may hold only limited appeal for prospective users. Second, our survey data suggests that other major barriers to educational attainment likely render improved energy access (whether through solar lanterns or otherwise) a relatively unimportant educational input.
External Link(s)

Registration Citation

Citation
Stojanovski, Ognen, Mark Thurber and Frank A. Wolak. 2018. "Assessing Opportunities for Solar Lanterns to Improve Educational Outcomes in Rural Off-Grid Regions." AEA RCT Registry. August 06. https://doi.org/10.1257/rct.3210-1.0
Former Citation
Stojanovski, Ognen, Mark Thurber and Frank A. Wolak. 2018. "Assessing Opportunities for Solar Lanterns to Improve Educational Outcomes in Rural Off-Grid Regions." AEA RCT Registry. August 06. https://www.socialscienceregistry.org/trials/3210/history/32674
Experimental Details

Interventions

Intervention(s)
Intervention Start Date
2016-04-25
Intervention End Date
2016-12-15

Primary Outcomes

Primary Outcomes (end points)
scores on grade 7 and grade 9 national examinations

self-reported study habits (most frequent location of studies, time of day for studies, who study with, type of light used for studying when dark)
Primary Outcomes (explanation)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The primary research question for the RCT was whether giving students solar lights would shift their study habits and, more importantly, improve academic performance. We prioritized undertaking the research in a rural location similar to other places in Sub-Saharan Africa where solar lanterns had successfully been sold at scale, while at the same time accounting for the risk of “contamination” through participants’ exposure to solar lanterns outside the research context. In addition, we had to recruit participants that could plausibly be motivated to use solar lanterns in their studies in order to improve academic performance. And we needed to introduce the solar lights in a manner similar to what such participants might encounter outside of a research setting (typically the sale of a light by a social enterprise that highlights its potential educational benefits).

We focused the research on students in grades 7 through 9—the last three grades of primary school in Zambia—for several reasons. First, children in earlier grades would likely have been too young to be able to answer the questions in our surveys. Second, our scoping research revealed that lower grades were generally not assigned much homework, making it less likely that improved lighting would influence studies and performance. Third, school officials pointed out that it is mostly grade 7 and beyond when children drop out of school altogether, so interventions that might improve performance and encourage ongoing enrollment might be particularly well targeted to those grades. Fourth, children in grades 7 and 9 take standardized national examinations. By all accounts, students across Zambia—as well as their parents and teachers—are well aware of the importance of these tests and take them seriously. We could safely assume that children would be quite motivated to use all tools at their disposal—including, potentially, solar lanterns—to improve performance. Moreover, the fact that the national examinations are standardized and graded equally between different schools and classrooms makes them an ideal way to measure academic performance outcomes in an RCT.

We carried out the RCT in 12 government-run primary schools randomly selected from a master file of all schools in Zimba District. Over 1,400 children in grades 7, 8 and 9 completed in-school surveys at the start of the school year in February 2016, as well as during national examinations season in October 2016. Not all children attend school every day but 80% of children who filled out the October survey also completed the earlier one in February. Through these surveys, we collected data on study habits and other factors that might influence educational outcomes. The student surveys took about one hour to complete. Following the first survey, a different team of researchers, who otherwise had no role in the study, conducted lotteries that served as the delivery mechanism for the RCT’s “treatments”, including the giveaway of over 200 solar lanterns to randomly selected children. These lotteries took place at the start of the second school term in May 2016, were designed to isolate the impact of solar lighting itself.

Although we collected the national examination scores directly from education officials, we nevertheless asked students to complete detailed surveys, for several reasons. First, we were interested in detecting impacts of solar lanterns on certain study habits irrespective of examination performance. In other words, the times of day that children study, study locations, study partners, and types of lights used for nighttime studies were additional outcomes of interest in our RCT. Second, having this additional data enabled us to control for additional background variables that could allow us to obtain more precise estimates of the impacts of solar lights on examination scores and study habits.

Third, the surveys were key to our additional research objective of examining the mechanisms through which solar lights are introduced and used. Simply handing out lights does not provide information about whether and how students actually use them nor how study habits relate to examination scores. Moreover, making the surveys broad in scope was itself a tool through which we blinded participants to the study’s goals. We did this in order to not have students feel like there were “right answers” when it came to reporting study habits, the use of solar lanterns and, most importantly, the relationship between the two.

Finally, the rich survey dataset covering students’ daily lives is what enabled us to gain important insights into the broader educational environment into which solar lanterns are deployed. That way we could address not just whether we detected academic impacts of solar lanterns but also why we might observe the relevant results. Ultimately, this broader examination of the relationships between household energy access, poverty, and children’s academic opportunities is key to understanding how improved household lighting might translate to improved educational outcomes.

TREATMENT IMPLEMENTATION DETAILS:

The priority for our experiment’s implementation was ensuring that any given student within a grade at a particular school had an equal and random chance of being “treated”. To do that, we conducted a series of 36 lotteries – one for each grade level at all 12 schools. Since not all children attend school every day, only those that both took the baseline survey and were present on the day of the lottery several months later were eligible to participate. Therefore, children that missed school during either of the two surveys or the lottery are left out of the RCT analysis.

We took precautions to ensure that the lotteries were not perceived as solar lantern giveaways. Instead, the goal was for school officials, teachers, students, parents, and even some of our own researchers to perceive the lottery as an exercise intended to thank children for participating in a general study of Zambian schooling. We therefore also gave away three other prizes or “treatments”: backpacks, battery-powered alarm clocks, and soap. The “control” students received a candy. There was no general emphasis on the lanterns; they were just one of several prizes that students were eligible to win thanks to completing broad surveys about their daily lives.

An advantage of the multiple prize design was that it enabled us to isolate the solar-powered lighting attribute of our target intervention and thereby control for any income effects that might have been triggered by the receipt of a solar lantern. The other prizes we handed out were worth approximately the same as the sales price of a solar lantern and, in some cases, could also be considered helpful for education. We could therefore study the impact of receiving a lighting product, distinct from the impact of receiving something that is worth approximately USD 10 and that could potentially be monetized and repurposed. An additional advantage was that the other three prizes were familiar items that children would have been aware are valuable and not normally given away. They were therefore useful to signal the value of solar lanterns to children that might not have previously been exposed to solar lights (or may possibly have viewed them as free goods that charities hand out). We note that we did not see evidence that the prizes were monetized, repurposed, traded or otherwise not used for their intended purpose. During the endline survey, 93% of backpack recipients and 87% of both solar lantern and clock recipients reported still owning the prizes they won in the lottery. Importantly, very few children reported having sold or given away their prize. We therefore believe that we were successful in implementing our research design with the goal of studying the impact of having received a product with solar-powered lighting attributes.

This approach also enabled us to deliver the lights in an educational setting and encourage students to use them, but at the same time hopefully avoid giving cues about any particular impacts we “wanted,” which could have resulted in data bias (or even potentially favorable treatment by teachers towards certain students). Another benefit of awarding multiple prizes was that it enabled us to have a consistent approach with the three “pure control” schools where no students received lights. By also awarding prizes to those schools’ students, we avoided the political and practical risks of control schools being perceived as different from the 9 “treated” ones.

Nevertheless, because solar lanterns were likely not as familiar to the children as the other prizes, we did take limited additional measures when handing them out. Lantern winners received an “information card” – a brief, easy-to-understand sheet (printed on high-quality cardstock) that consisted of instructions on proper use, emphasized that the lantern could be helpful for studying, and provided a number to call in case it stopped working (see Appendix 6). Research staff also demonstrated how to use the light and delivered the same messages from the information card verbally when giving a child a lantern. In this regard, we mimicked what a vendor might do when selling a solar lantern while still not drawing too much attention to the lights somehow being more special than the other prizes. Finally, we sent one research team member back to the schools on four different occasions to check whether students who had won a light still owned it and were using it. This was presented to participants as a routine part of warranty support for the lanterns by their distributor.




Experimental Design Details
Randomization Method
Public lottery (see Experimental Design Section)
Randomization Unit
Individual students (see Experimental Design Section)
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
12 schools
Sample size: planned number of observations
2000 students
Sample size (or number of clusters) by treatment arms
578 students control, 231 students solar lanterns, 133 students backpacks, 138 students alarm clocks, 131 students soap
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
December 15, 2016, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
March 01, 2017, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
Final Sample Size (or Number of Clusters) by Treatment Arms
Data Publication

Data Publication

Is public data available?
No

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials