The method of Analysis and the development of cognitive skills

Last registered on May 25, 2022

Pre-Trial

Trial Information

General Information

Title
The method of Analysis and the development of cognitive skills
RCT ID
AEARCTR-0008409
Initial registration date
November 09, 2021

Initial registration date is when the trial was registered.

It corresponds to when the registration was submitted to the Registry to be reviewed for publication.

First published
November 10, 2021, 10:30 AM EST

First published corresponds to when the trial was first made public on the Registry after being reviewed.

Last updated
May 25, 2022, 9:22 AM EDT

Last updated is the most recent time when changes to the trial's registration were published.

Locations

Region
Region

Primary Investigator

Affiliation
LUISS

Other Primary Investigator(s)

PI Affiliation
Aix-Marseille Universitè

Additional Trial Information

Status
Completed
Start date
2021-11-14
End date
2022-01-31
Secondary IDs
Prior work
This trial does not extend or rely on any prior RCTs.
Abstract
In order to validate the main assumptions and the functional equations of our educational planning and job search model, we investigate the effects of the method of analysis, or broadly speaking investigation inquiry-based learning (or inductive learning), on the development of cognitive skills. Our idea is that an analysis-based approach has a substantial benefit in the development of cognitive skills relative to the common synthesis-based learning process, or broadly speaking deductive learning. The trial is designed as follows: we take two primary school classes with the same teachers, one is used as a control group and the other as a treatment group. The treatment consists of an educational path, designed according to the methodologies of investigation-based science education (IBSE), that it should provide an exogenous variation in the amount of analysis done by the children. Cognitive skills are measured before and after the treatment and compared between the control and the treatment class.
A first stage of this experiment has been conducted in Rome, Italy, last year. The RCT will continue both in the same classes of the roman primary school in which is conducted the first stage , and in another primary school in Cameroon. The educational path used as treatment are the BUL project in Italy and the TALES project in Cameroon. More details about these projects can be found here. https://it.oiler.education/scuola/materiali/primaria

Registration Citation

Citation
Bernardi, Luigi and Riccardo Manghi. 2022. "The method of Analysis and the development of cognitive skills." AEA RCT Registry. May 25. https://doi.org/10.1257/rct.8409
Experimental Details

Interventions

Intervention(s)
Considering the most universally accepted aspects andon which students of the philosophy of logic are in agreement, we can think of Analysis in the educational environment as inquiry-based learning, in which the students build knowledge and abilities from their own by trials and errors. The learning process is slow but the concepts are deeply understood since they are rebuilt from the interior. On the other hand, through synthesis students are guided through the concepts. It may result in a lack of grasp on the learned notions, but the process is fast and it works in various contexts, including large classes. Throughout an analysis-based approach, students are guided along a process that helps the development of the ability to ”build” their knowledge from testing ideas and approaches based on their previous knowledge and experience by applying these new situations and integrating the newly acquired knowledge with preexisting intellectual constructs. We refer to this ability as Constructive Skill. This concept contrasts with the one of Applied skill, which consists in practical skills, knowledge and tools related to a specific context needed to do some specific task. I present an RCT aimed to assess the causal effect of analysis on the development of Constructive Skill, validating one of the most important theoretical assumptions of the paper. The trial is structured as follows: I use an educational path based on the method of analysis, BUL, to produce an exogenous variation in the amount of Analisys which is done. This project is conducted in two classes of a Roman primary school. In the other exercise, the educational path choosen is called TALES and the treatment and the control groups are from a primary school in Cameroon. Then, Constructive skill is measured for every child of the class and for the children of the other class with the same main teacher but that are not undertaking the project, both before and after the period in which the project is undergone. To measure constructive skill, two different proxies are used: an IQ test for children(repeated both before and after the treatment) and a set of logical questions taken from INVALSI tests (different at each measurement, to capture mental elasticity). Concerning the first trial, the one already conducted in Rome, we built the IQ measure following the MENSA criteria. The results gave me a normal shape for the distribution of the IQs, with the average being near 100 (respectively 101, 100 and 104). From the balancing tests the covariates related to nationality of origin, sex and age result to have similar distribution between the control class and the treatment class. The first return test is conducted just two months after the first one and is likely to be strongly influenced by learning effects due to the memory of questions performed only two months earlier. The test will be repeated during the next year- where the treatment will also continue -, both to quantify the entity of these learning effects and isolate them, and to verify the effects of the treatment after more prolonged exposure. intelligence and nationality have a relevant and significant effect in both exercises. The impact of the second is not very surprising, as it is evident that children with parents of foreign nationalities face language barriers that slow down not only the learning possibilities but also the effectiveness of an IQ test as these children have to do a greater effort to understand some questions. Furthermore, this effect becomes greater if the language spoken at home is not Italian. As regards our variable of interest, we note that the effect on the treatment IQ is not significant. This is probably due to the persistence of memory effects, as discussed above, combined with the possibility that IQ measures a set of skills related to the g factor, while the Analysis method favors a particular component, the one close to the concept of measured mental elasticity. For both of these two reasons, in fact, the second exercise reveals a significant effect of the treatment. In order to distinguish the weight and relevance of each of these two topics, the subsequent return tests will be fundamental, which will allow isolating the memory effect. The trial in the primary school in Cameroon will be conducted with the same methodology.
Intervention Start Date
2021-12-01
Intervention End Date
2022-01-10

Primary Outcomes

Primary Outcomes (end points)
Constructive\Cognitive Skill
Primary Outcomes (explanation)
Throughout an analysis-based approach, students are guided along a process that helps the development of the ability to ”build” their knowledge from testing ideas and approaches based on their previous knowledge and experience by applying these new situations and integrating the newly acquired knowledge with preexisting intellectual constructs. We refer to this ability as Constructive Skill. To measure constructive skill, two different proxies are used: an IQ test for children(repeated both before and after the treatment) and a set of logical questions taken from INVALSI tests (different at each measurement, to capture mental elasticity).

Secondary Outcomes

Secondary Outcomes (end points)
Secondary Outcomes (explanation)

Experimental Design

Experimental Design
The trial is structured as follows: I use an educational path based on the method of analysis, BUL, to produce an exogenous variation in the amount of Analisys which is done. This project is conducted in two classes of a Roman primary school. In the other exercise, the educational path choosen is called TALES and the treatment and the control groups are from a primary school in Cameroon. Then, Constructive skill is measured for every child of the class and for the children of the other class with the same main teacher but that are not undertaking the project, both before and after the period in which the project is undergone. To measure constructive skill, two different proxies are used: an IQ test for children(repeated both before and after the treatment) and a set of logical questions taken from INVALSI tests (different at each measurement, to capture mental elasticity).
Experimental Design Details
The trial is structured as follows: I use an educational path based on the method of analysis, BUL, to produce an exogenous variation in the amount of Analisys which is done. This project is conducted in two classes of a Roman primary school. In the other exercise, the educational path choosen is called TALES and the treatment and the control groups are from a primary school in Cameroon. Then, Constructive skill is measured for every child of the class and for the children of the other class with the same main teacher but that are not undertaking the project, both before and after the period in which the project is undergone. To measure constructive skill, two different proxies are used: an IQ test for children(repeated both before and after the treatment) and a set of logical questions taken from INVALSI tests (different at each measurement, to capture mental elasticity).
Randomization Method
Choice of the treatment class and control class random. The classes share the same teachers. Balancing tests show no ex-ante significant difference between the two classes
Randomization Unit
Class
Was the treatment clustered?
No

Experiment Characteristics

Sample size: planned number of clusters
2 classes for the trial in Rome, two classes for the trial in Cameroon
Sample size: planned number of observations
42 students for the trial in Rome, data still to collect for the trial in Cameroon
Sample size (or number of clusters) by treatment arms
2 classes
Minimum detectable effect size for main outcomes (accounting for sample design and clustering)
Supporting Documents and Materials

Documents

Document Name
Tests
Document Type
other
Document Description
Test used to measure cognitive skills
File
Tests

MD5: 2895181cb1502056e76ea087edc99ffd

SHA1: 3b957f500efc81c2c8986bb62c59197dd68e0d1a

Uploaded At: November 08, 2021

Document Name
Cameroon.Analysis
Document Type
other
Document Description
File with the date and the results of the trials on the Cameroonian sample.
File
Cameroon.Analysis

MD5: d4bb06390f40262d9bb4c667fd8787e0

SHA1: ec3324e608b8495bd81c0270d022ac3e5af1ad69

Uploaded At: March 11, 2022

Document Name
Italy. Analysis
Document Type
other
Document Description
Analysis and results for the italian sample + learning equation calibration for the whole sample
File
Italy. Analysis

MD5: 030f116159f1b760c6a889ed1baa1435

SHA1: c8699a3c4b1432a7e93b5744391465f79c3505d7

Uploaded At: March 11, 2022

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information
IRB

Institutional Review Boards (IRBs)

IRB Name
IRB Approval Date
IRB Approval Number

Post-Trial

Post Trial Information

Study Withdrawal

There are documents in this trial unavailable to the public. Use the button below to request access to this information.

Request Information

Intervention

Is the intervention completed?
Yes
Intervention Completion Date
January 28, 2022, 12:00 +00:00
Data Collection Complete
Yes
Data Collection Completion Date
January 28, 2022, 12:00 +00:00
Final Sample Size: Number of Clusters (Unit of Randomization)
4
Was attrition correlated with treatment status?
No
Final Sample Size: Total Number of Observations
124
Final Sample Size (or Number of Clusters) by Treatment Arms
2 schools, 2 treatment classes, 3 control classes
Data Publication

Data Publication

Is public data available?
No

Program Files

Program Files
No
Reports, Papers & Other Materials

Relevant Paper(s)

Abstract
Bul Game is an online software designed to support didactic activities about logic and its connections with everyday language. The aim of the game is to make correct choices based on statements made by knights, who always tell the truth, and by knaves, who always lie. With small adjustments, the game can be used at all grade levels. As part of a programme to introduce logic at primary school, carried out in two second grade classes of an Italian school with many foreign students and one fourth grade class of a French school in Rome, Bul Game was used at the end of each lesson to consolidate and revise the concepts discussed in class. At the secondary-school level, Bul Game can aid the introduction of connectives and propositional logic, together with the characters of knights and knaves, which are a useful tool for dealing with proofs conducted with game semantics. Bul Game was designed and developed by Giulia Balboni, Luigi Bernardi, Mattia Sanchioni, and Jacopo Zuliani, and it is available for free at www.oiler.education/bul in English, French, and Italian
Citation
Bernardi, L. (2022). Bul Game: playing with knights and knaves. In Handbook of research on International Approaches and Practices for Gamifying Mathematics (pp. 170–188). IGI global. https://doi.org/10.4018/978-1-7998-9660-9.ch009

Reports & Other Materials

Description
Data and analysis Italy
Citation
Bernardi, Luigi and Riccardo Manghi. 2022. "The method of Analysis and the development of cognitive skills." AEA RCT Registry. May 25. 2022. "Registration Entry Title: Data and analysis Italy." AEA RCT Registry. May 25 https://doi.org/10.1257/rct.8409
File
ANALISIDATIlearning.xlsx

MD5: 030f116159f1b760c6a889ed1baa1435

SHA1: c8699a3c4b1432a7e93b5744391465f79c3505d7

Uploaded At: May 25, 2022

Description
Data and Analysis Cameroon
Citation
Bernardi, Luigi and Riccardo Manghi. 2022. "The method of Analysis and the development of cognitive skills." AEA RCT Registry. May 25. 2022. "Registration Entry Title: Data and Analysis Cameroon." AEA RCT Registry. May 25 https://doi.org/10.1257/rct.8409
File
analisidati.camerun.xlsx

MD5: d4bb06390f40262d9bb4c667fd8787e0

SHA1: ec3324e608b8495bd81c0270d022ac3e5af1ad69

Uploaded At: May 25, 2022

Description
Draft of new article about BUL (teaching logic in primary school) with the RCT in the italian sample
Citation
Bernardi, Luigi and Riccardo Manghi. 2022. "The method of Analysis and the development of cognitive skills." AEA RCT Registry. May 25. 2022. "Registration Entry Title: Draft of new article about BUL (teaching logic in primary school) with the RCT in the italian sample." AEA RCT Registry. May 25 https://doi.org/10.1257/rct.8409
File
new_articolo (1).docx

MD5: 5d368d66a611ba0fe84258d523336073

SHA1: 095b0feea14263d1a3f80c396cabdaa53396b7bc

Uploaded At: May 25, 2022