Weak governance is a critical constraint in delivering quality education in developing countries. A recent review of a decade of education research in India concluded that “business as usual expansion in education spending is unlikely to yield improved outcomes without significant improvements in pedagogy and governance of the education system” (Muralidharan 2013). A striking indicator of weak governance in education in India is the high rate of teacher absence, estimated as 26.2% in 2003 and 23.6% in 2010, leading to an estimated fiscal cost of $1.5 billion annually (Muralidharan et. al. 2014).
We propose to conduct a randomized evaluation of one of the most ambitious recent attempts by a developing country government to improve education governance at scale. The Madhya Pradesh (MP) School Quality Assessment Program (MPSQA) – officially called the MP Shaala Gunwatta program – is a comprehensive attempt to improve school governance in the Indian state of MP and has been designed in a partnership between the Government of MP (GoMP), DFID, and Absolute Return for Kids (ARK). The program targets school governance directly through a combination of (a) regular monitoring of schools, (b) creation of school report cards, and a customized school improvement plan, (c) quarterly follow-ups on progress against this plan, and (d) the leveraging of ICT tools to collect and report real-time records of all assessment results, school improvement plans, and follow-ups to a dedicated online system. MPSQA was piloted and refined in a sample of 100 schools in 2013-14 (phase 1), and has been rolled out across 2,000 schools in the school year (2014-15) (phase 2). The Principal Investigators (PI’s) have worked with the project implementation team from ARK and GoMP and have successfully randomized the rollout of the program across ~2,000 schools in August 2014, to enable a credible evaluation of the MPSQA program on both school process outcomes and student learning outcomes.
The evaluation, with a sample of 534 schools (primary and secondary), will assess the effects of the program on key governance metrics (e.g. teacher absence) as well as student academic outcomes over a period of 18 months.
External Link(s)
Citation
Muralidharan, Karthik and Abhijeet Singh. 2020. "Improving School Governance at Scale: A Randomized Evaluation of the Madhya Pradesh School Quality Assessment Program." AEA RCT Registry. November 16. https://doi.org/10.1257/rct.851-1.4000000000000001.
Former Citation
Muralidharan, Karthik, Karthik Muralidharan and Abhijeet Singh. 2020. "Improving School Governance at Scale: A Randomized Evaluation of the Madhya Pradesh School Quality Assessment Program." AEA RCT Registry. November 16. http://www.socialscienceregistry.org/trials/851/history/79626.
The Madhya Pradesh (MP) School Quality Assessment (MPSQA) intervention is a comprehensive program that aims to improve school governance and outcomes through a combination of (a) regular monitoring of schools, (b) creation of school report cards, and a customized school improvement plan, (c) quarterly follow-ups on progress against this plan, and (d) leveraging of ICT tools to collect and report real-time records of all assessment results, school improvement plans, and follow-ups to a dedicated online system.
Intervention Start Date
2014-09-01
Intervention End Date
2016-04-30
Primary Outcomes (end points)
There are two kinds of outcome variables:
a) Intermediate process variables at the school level, teacher level, and student level: Teacher absence and effort; the pedagogical practices adopted by teachers
School infrastructure
School management structures like monitoring mechanisms, feedback mechanisms between school headmaster and teachers.
b)Student-level academic outcomes: These would be measured by test scores on customized tests designed by the evaluation team.
Primary Outcomes (explanation)
Secondary Outcomes (end points)
Secondary Outcomes (explanation)
Experimental Design
The randomization was conducted not at the level of an individual school, but at the level of an education “cluster” of around 40 schools (these clusters are referred to as a Jan Shiksha Kendra or a JSK).
Based on this design, we first drew a representative sample of 153 JSK’s out of a total of 308 JSK’s, across 29 blocks in 5 districts. We then randomized 51 of the 153 JSK’s into treatment status and 102 into control. Randomization was stratified within-block based on average school characteristics including previous year’s school-level assessment scores based on GoMP’s annual low-stakes measurement of learning. Treatment and control groups are balanced on key observables not just for means but also for distributions (tables available on request). A sample of 2 schools from each cluster was drawn, making it a total of 534 schools.
Experimental Design Details
Randomization Method
Randomization done in office by a computer.
Randomization Unit
The first unit of randomization is the JSK (school cluster administrative unit) and the second level of randomization is school.
Was the treatment clustered?
Yes
Sample size: planned number of clusters
153 JSKs and 534 Schools
Sample size: planned number of observations
534 schools, 2 classrooms observed and assessed per school (class size not pre-determined)
Sample size (or number of clusters) by treatment arms
153 JSK treatment and 102 JSK control.
216 Treatment and 318 control schools
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)