"Unearthing the Treasure: Assessing the Impact of the Financial, Fiscal and Civic Education Program in Brazilian Public Schools"

Last registered on June 28, 2023

Pre-Trial

Trial Information

General Information

Title
"Unearthing the Treasure: Assessing the Impact of the Financial, Fiscal and Civic Education Program in Brazilian Public Schools"
RCT ID
AEARCTR-0011363
Initial registration date
June 23, 2023

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
June 28, 2023, 4:33 PM EDT

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Locations

Primary Investigator

Affiliation
326.550.188-42

Other Primary Investigator(s)

PI Affiliation
PI Affiliation

Additional Trial Information

Status
In development
Start date
2023-04-10
End date
2023-10-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
This study investigates the impact of the Program "Em Busca do Tesouro" in promoting financial, fiscal, and civic education in elementary schools in the Federal District, Brazil. Developed by the National Treasury Team in partnership with the Maurício de Souza Institute, the program presents concepts related to financial decisions, public administration, and the relationship between the civic population and the government through comics with popular characters from Turma da Mônica (a famous national cartoon created by Maurício de Souza). The program is aimed at 4th and 5th-grade students, and its pilot is being conducted in a selected group of schools, in which students in the target age group will be exposed to program materials and involved in classroom activities. After the intervention, students will answer a questionnaire to assess the impact on their knowledge, attitudes, and behaviors related to fiscal, financial, and civic education. Expected results include increased students' understanding of the concepts covered. possible changes in their attitudes and habits regarding savings and other financial issues, a higher understanding and appreciation of public goods, and more active behavior regarding authority's accountability.

Registration Citation

Citation
Madeira, Ricardo, Luis Eduardo Negrão Meloni and Renata Serson. 2023. ""Unearthing the Treasure: Assessing the Impact of the Financial, Fiscal and Civic Education Program in Brazilian Public Schools"." AEA RCT Registry. June 28. https://doi.org/10.1257/rct.11363-1.0
Sponsors & Partners

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
Experimental Details

Interventions

Intervention(s)
The "Em Busca do Tesouro" program is an initiative designed to educate 4th and 5th-grade students in Brazil about financial, tax, and civic concepts through comic books from "Turma da Monica", a famous and traditional Brazilian cartoon by the "Instituto Mauricio de Souza".

This program is planned to span over eight weeks, not including the initial material distribution phase and an additional three weeks dedicated to teachers' training.

Each comic book story integrates elements from the four major areas in the Brazilian national curriculum for these grades: linguistics, mathematics, natural sciences, and human sciences. The objective is that students interact with the content in a cross-disciplinary and comprehensive manner, thereby enhancing the learning experience.

The program's pilot takes place during the first semester of 2023 in a randomized set of schools in Distrito Federal, Brazil. The schools had six weeks to enroll in the program. After that period, schools were paired according to relevant characteristics and randomized between treatment and control groups.
Intervention Start Date
2023-04-24
Intervention End Date
2023-06-09

Primary Outcomes

Primary Outcomes (end points)
The primary outcomes are the enlargement of students’ vocabulary and the development of skills, abilities, and attitudes related to financial, fiscal, and civic education (their recognition of the value of money, the budget constraints, the importance of establishing priorities in consumption, the importance of saving and financial planning for maintaining their family’s quality of life and the government’s budget balance, the importance of prudent use of public and private resources, their role in exercising democracy, the social function of taxes and the importance of transparency and accountability).
Primary Outcomes (explanation)
A) Students’ vocabulary related to fiscal, financial, and civic education topics
21 multiple-choice questions with unique answers covering the concepts of savings, planning, debt, budget, balance, spending, revenue, expense, taxes, public money, election, public service, transparency, and accountability.

B) Students’ recognition of the value of money
On a scale of 1 to 6, how much do you agree with “Every time I earn some money I leave some to spend later”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “Saving money can change a person's life, providing them with better living conditions”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6 how much do you agree with “Every time I earn some money, I spend it as quickly as possible.”? (1 = strongly disagree, 6 = strongly agree)


C) Students’ recognition of the budget constraints and the importance of establishing priorities in consumption
3 multiple choice questions with unique answers

D) Students’ recognition of the importance of saving and financial planning for maintaining their family’s quality of life and the government’s budget balance?
On a scale of 1 to 6, how much do you agree with “I think it's important to save money for a day when I need it instead of spending it on something fun.”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I have a budget (a plan with the amounts of money I should earn and spend) for how I intend to spend my money during the week”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I have money saved nowadays, whether with a family member, with a friend, in a piggy bank, in the bank, or anywhere else”? (1 = strongly disagree, 6 = strongly agree)
How important do you think it is that the government does not spend all its money? (Not important/ Slightly important/ Important/ Very important)

E) Students’ recognition of the importance of prudent use of public and private resources
How important do you think it is to keep track of how much money you earn and spend? (Not important/ Slightly important/ Important/ Very important)
How important do you think it is to monitor how often garbage is collected at school? (Not important/ Slightly important/ Important/ Very important)
How important do you think it is to monitor how often the school is cleaned? (Not important/ Slightly important/ Important/ Very important)
Do you usually keep track of whether garbage has been collected at your school? (Yes/ No/ Sometimes)
Do you usually keep track of whether your school has been cleaned? (Yes/ No/ Sometimes)

F) Students’ recognition of their role in exercising democracy
On a scale of 1 to 6, how much do you agree with “Elections are the fairest way to choose president, governor, and mayors”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “Elections are the only way citizens can influence public decisions”? (1 = strongly disagree, 6 = strongly agree)

G) Students’ recognition of the social function of taxes and the importance of transparency and accountability
On a scale of 1 to 6, how much do you agree with “When I see my parents paying their taxes, I think they are wasting money.”? (1 = strongly disagree, 6 = strongly agree)
How important do you think it is to follow what our government decides? (Not important/ Slightly important/ Important/ Very important)
How important do you think it is for the government to publish a plan on how it intends to spend its resources? For example, having a publicly accessible website where it is possible to consult information about how public money is used. (Not important/ Slightly important/ Important/ Very important)

Secondary Outcomes

Secondary Outcomes (end points)
Secondary outcomes are students’ interest in school and their assessment of whether the classes are dealing with their specific difficulties, students’ discussion about the themes of the activities with their parents, students’ beliefs about the capacity of the school to create a stimulating environment and to teach useful subjects to their daily life, students’ gains in basic math, impact on parents/family members and impact on children’s future self.

Also, through the teachers’ survey, outcomes related to the monitoring of the program and assessment of the impacts on teachers’ skills, abilities, and attitudes related to financial, fiscal, and civic education will also be measured.
Secondary Outcomes (explanation)
A) Students’ interest in school and their assessment of whether the classes are dealing with their specific difficulties
On a scale of 1 to 6, how much do you agree with “The subjects studied in the classroom deal with difficulties I have outside of school”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “This year's classroom activities are more interesting than previous years”? (1 = strongly disagree, 6 = strongly agree)

B) Students’ discussion about the themes of the activities with their parents
On a scale of 1 to 6, how much do you agree with “I talk to my parents about what I learn at school”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “In the last 2 months, I talked to my parents or family members about the importance of saving money"? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I feel ready to talk to my parents about money”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I feel ready to talk to my parents about politics”? (1 = strongly disagree, 6 = strongly agree)
How often do you talk to your family about the topics studied at school? (Never/ Rarely/ Sometimes/ Very often)
How often do you talk with your family about ways to save money and other money-related issues? (Never/ Rarely/ Sometimes/ Very often)
How often do you talk to your family about taxes? (Never/ Rarely/ Sometimes/ Very often)
How often do you talk to your family about the government and elections? (Never/ Rarely/ Sometimes/ Very often)
How often are you interested in following news about the country with your family? (Never/ Rarely/ Sometimes/ Very often)

C) Students’ beliefs about the capacity of the school to create a stimulating environment and to teach useful subjects to their daily life
On a scale of 1 to 6, how much do you agree with “I find school interesting”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I find school boring”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I believe that the school answers useful questions for my daily life”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “I learn in school about subjects that appear in my daily life and in my family's daily life”? (1 = strongly disagree, 6 = strongly agree)

D) Students’ gains in basic math
6 multiple choice questions of basic mathematics topics such as counting, comparing and ordering natural numbers, arithmetic, fundamental operations, fractional numbers, and decimal and percentage representation, among others.

E) Impact on parents/family members
On a scale of 1 to 6, how much do you agree with “My parents or immediate family members have recently changed the way they handle money, save and make purchasing decisions”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “My parents usually pay their taxes”? (1 = strongly disagree, 6 = strongly agree)

F) Impact on children’s future self.
On a scale of 1 to 6, how much do you agree with “When I'm an adult, I'll take the teachings from school to decide how I'll handle money, save and make my future shopping decisions”? (1 = strongly disagree, 6 = strongly agree)
On a scale of 1 to 6, how much do you agree with “When I'm an adult, I'll be a better taxpayer because of school teachings”? (1 = strongly disagree, 6 = strongly agree)

Experimental Design

Experimental Design
The Randomized Controlled Trial (RCT) structure ensures the comparison between schools from the treatment group - those that have received the EBT Program - with schools from the control group - those that have not been treated in any way.

Initially, 113 schools enrolled to receive the program. Seven of these schools were not considered because there was not enough administrative data available to properly pair them. One school was a private institution and would not be comparable to the others, so it was excluded from the sample. The remaining 106 schools were paired using Propensity Score Matching (PSM) that considered their size, student performance, and infrastructure characteristics of schools.

The randomization process resulted in 53 schools in each group, totaling a sample of close to 18,000 students. Teachers in the treatment group underwent a three-week training period in which they were encouraged to watch training videos and study the instructor manual. They then were able to implement the suggested activities. Students in the treatment group went through an eight-week intervention during which they were exposed to lessons on fiscal, financial, and civic education. Two monitoring surveys were sent to teachers during the eight-week program period. In the latter survey, teachers were also asked impact questions.

After the 11-week period, data is collected from students through an in-person questionnaire.

The outcomes will be calculated, and data will be analyzed using the RCT methodology.

Experimental Design Details
Randomization Method
Randomization was done in an office with a computer, using the software R and the Propensity Score Matching (PSM) and Mahalanobis Distance methodology, in which school size and infrastructure, and student performance were taken into account. The randomization is recorded and can be reviewed upon request.
Randomization Unit
The cluster unit for randomization is schools.
Was the treatment clustered?
Yes

Experiment Characteristics

Sample size: planned number of clusters
106 schools
Sample size: planned number of observations
Approximately 16.400 students
Sample size (or number of clusters) by treatment arms
53 schools control, 53 schools treatment
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Our power calculations are limited due to a lack of baseline data, and similar outcomes, especially for fiscal and civic education in the consulted literature. The following estimates are based on item 7.2.1 of the WP by Djimeu, 2019, a 3ie manual, using proxies for our primary outcome of interest from the WP by Bruhn et al 2013, which also runs an experiment on a financial program in Brasilia. We estimate a minimum detectable size of around 14% SD for this sample, with power = 0.8 and alpha = 0.05. Intra cluster correlation was set as 0,03 using the metanalysis estimation for the USA (Parker et al 2023), but because there is a chance it will be higher in this setting, a sensitive analysis was also run with ICC of 0,1, which would configure an upper bound for our MDE estimates. 1) (Bruhn et al WP 2013) Financial autonomy index (ranges 1 to 100): Mean 49.80 (SD 15.18) MDE 2.2 - 4.4% of the mean in the untreated state (with higher ICC: 3.8 - 7.6% of the mean in the untreated state). 2) (Bruhn et al WP 2013) Financial autonomy index (ranges 1 to 100): Mean 49.11 (SD 19.83) MDE 2.9 - 5.9% of the mean in the untreated state (with higher ICC: 4.9 - 10.1% of the mean in the untreated state). 3) (Bruhn et al WP 2013) Intention to save index (ranges 1 to 100) Mean 48.29 (SD 18.95) MDE 2.77 - 5.7% of the mean in the untreated state. (with higher ICC: 4.7 - 9.8% of the mean in the untreated state). MDE will be revisited when we have data on our main outcomes.
Supporting Documents and Materials

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number
Analysis Plan

Analysis Plan Documents

Post-Trial

Post Trial Information

Study Withdrawal

There is information in this trial unavailable to the public. Use the button below to request access.

Request Information

Intervention

Is the intervention completed?
No
Data Collection Complete
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
Reports, Papers & Other Materials

Relevant Paper(s)

Reports & Other Materials