Harnessing AI to Train Mindsets for Mathematical Higher Order Thinking

Harnessing AI to Train Mindsets for Mathematical Higher Order Thinking is sponsored by Institute of Education Sciences (IES). Develops and pilots a digital technology-based classroom supplement using trained generative AI to improve students' curriculum learning and attention to higher-order thinking in mathematics.

Harnessing AI to Train Mindsets for Mathematical Higher Order Thinking | IES Harnessing AI to Train Mindsets for Mathematical Higher Order Thinking Education Research Grants Cognition and Student Learning University of California, Irvine (07/01/2024 - 06/30/2028) Development and Innovation In this project, the researchers will develop and pilot test a digital technology-based classroom supplement that uses a trained generative artificial intelligence (AI) technology to improve grade 5 and 6 students' curriculum learning as well as train their attention to notice and attend to higher order thinking (HOT) opportunities within mathematics instruction.

This tool will build upon the researchers' prior IES-funded work ( R305A190467 ) that found the more students engage linguistically in HOT, the more they will gain an expert-like approach to mathematics. HOT Talk is language used to articulate comparisons, connections across contexts, inferences, hierarchical categorizations, or abstractions.

This project aims to develop and pilot test an instructional module that incorporates instruction via clips of real-world math lessons on ratio and proportions with a generative AI chatbot trained to produce HOT Talk as well as to engage students in producing HOT Talk. The researchers will test the feasibility of this developed tool as well as whether this is a pedagogical approach that could be leveraged to improve mathematics HOT.

The researchers will conduct two implementation studies in which usability, math learning, and qualitative implementation data are used to iteratively refine the developed tool, followed by a pilot study in which the tool will be compared against a business-as-usual control. The research will be conducted in public fifth and sixth grade classrooms in a diverse suburban school district in California.

Two implementation studies will be conducted with approximately 60 fifth and sixth grade students in study 1 and 120 students in study 2. The pilot study will include approximately 300 fifth and sixth grade students.

The researchers will develop a scalable instructional technology that could be used to individualize students' engagement in HOT Talk while learning math content, thereby both improving their math learning and training their attention to relations and serving to equalize opportunities for higher order thinking.

Previous work incorporated HOT Talk into instructional videos teaching ratio and proportion content interspersed with free-response prompts. This project builds upon that work by developing and testing the effectiveness of training a generative AI chatbot, called a "HOT Mindset Chatbot" to engage students in producing HOT Talk.

The chatbot will be trained on relevant math content from a textbook text and on HOT Talk routines through transcripts of everyday HOT Talk and HOT Talk during problem solving tasks. Research design and methods This project will begin with an implementation study in which the researchers will use usability, math learning, and qualitative implementation data to iteratively refine the developed tool.

In a second implementation study, they will compare the developed digital instruction plus generative AI HOT Mindset Chatbot versus the same digital instruction plus a non-interactive set of scripted HOT Talk questions students will answer. They will use results from study 2 to improve the intervention and update the chatbot if it does not lead to more HOT Talk and learning than the non-interactive questions.

Finally, they will run a pilot study of the intervention with 10 classes, comparing the tool against a business-as-usual control. For the implementation studies, there is no control condition.

For the pilot study, students will be randomly assigned within classroom to the HOT Mindset Intervention or to a business-as-usual control in which they spend the same allocation of time on the mathematics educational technology currently in use within this school system. All students will be administered the same measures.

Key measures include researcher-developed tests of conceptual and procedural knowledge of proportional reasoning, a test of students' spontaneous use of HOT on an unrelated set of problems, an executive function task, and surveys that capture situational interest, mind wandering, and qualitative responses to questions about their experience and the process of participating with the developed intervention.

The research team will explore the relationships between HOT Talk within the transcript of talk between the child and the chatbot, math performance, and engagement (situational interest and mind wandering). The team will also investigate whether individual differences in children's attention to relations are related to learning or whether these relationships are mitigated by engagement with the chatbot.

Analyses will use analysis of variance, conventional regression models, and Latent Class Analysis. The researchers will evaluate the cost for development as well as implementation of the tool. These relative costs will be considered separately as well as jointly since scaled usage of the technology would not incur the same original development costs.

For the development, the cost calculations include costs of personnel to create the video lessons (both researcher and teacher time), time of the videographer, video editor, and programmer. The programmer's time will be committed to training the generative AI chatbot, and the research team to examine the rate of HOT Talk use in the chatbot transcripts.

For the implementation, the researchers will include the small-time commitment for teacher training, the cost of teacher time to run the study, the fixed infrastructure costs for computers available to all students, and the opportunity cost of our intervention materials taking the place of other math curricula.

People and institutions involved Co-principal investigator University of California, Irvine Co-principal investigator Products and publications The researchers will develop an online HOT Mindset digital training tool, which is a curriculum supplement designed to be administered in a blended in-class format.

The project will also result in peer-reviewed publications and presentations as well as additional dissemination products that reach education stakeholders such as practitioners and policymakers. ERIC Citations: Find available citations in ERIC for this award here . Drawing Connections to Close Achievement Gaps in Mathematics Linguistic Input as a Malleable Factor in Higher Order Thinking about Mathematics Questions about this project?

To answer additional questions about this project or provide feedback, please contact the program officer. Questions about this project? To answer additional questions about this project or provide feedback, please contact the program officer.

Data Science for Education (DS4EDU) Exploring Implementation of Play-Based Learning in... New Resource to Support Rural School District Lead...