Study: Multimodal AI Improves Edtech Decisions
What happened
A new study details an “intelligent educational decision-making system” that uses multimodal data fusion and knowledge graphs to personalize instruction. The system synthesizes diverse data streams, including voice, text, and behavioral logs, to map student progress and infer the next best action. This approach reportedly demonstrates superior accuracy in tailoring recommendations compared to single-modal systems, and the use of knowledge graphs offers greater transparency for its adaptation pathways.
Why it matters
- Automatic speech recognition (ASR) systems trained on adult voices perform poorly with children due to differences in acoustic properties like higher pitch and greater variability. Even when trained specifically on children's speech, error rates can be 60% to 176% higher than for adult-trained systems on adult speech. - Deep Knowledge Tracing (DKT), which often uses recurrent neural networks, is a common technique to model a student's knowledge over time. Multimodal approaches enhance DKT by incorporating not just text but also image, cognitive, and other data modalities to create a more accurate representation of a student's knowledge state. - To decide the "next best action," many adaptive systems use a multi-armed bandit (MAB) framework to manage the exploration-exploitation tradeoff. This allows the system to efficiently test the effectiveness of different educational activities for a student while also serving the content that has historically worked best. - Reinforcement learning (RL) can be used to create an optimal "policy" for sequencing content that maximizes a student's long-term learning. This involves mapping a student's current state, derived from interaction data, to the next piece of content, with the goal of improving metrics like future quiz performance. - Knowledge graphs help make an AI tutor's decisions transparent by explicitly mapping the relationships between educational concepts. This allows the system to justify its recommendations, such as showing that a new topic is being introduced because the student has mastered all the prerequisite skills. - Carnegie Learning's Cognitive Tutor, first developed at Carnegie Mellon in the 1980s, is an early and influential example of an AI-powered adaptive learning system in mathematics. The company's latest tool, LiveHint AI, uses a large language model trained on 25 years of student interaction data.
Key numbers
- Even when trained specifically on children's speech, error rates can be 60% to 176% higher than for adult-trained systems on adult speech.
- Carnegie Learning's Cognitive Tutor, first developed at Carnegie Mellon in the 1980s, is an early and influential example of an AI-powered adaptive learning system in mathematics.
- The company's latest tool, LiveHint AI, uses a large language model trained on 25 years of student interaction data.
What happens next
- To decide the "next best action," many adaptive systems use a multi-armed bandit (MAB) framework to manage the exploration-exploitation tradeoff.
- This involves mapping a student's current state, derived from interaction data, to the next piece of content, with the goal of improving metrics like future quiz performance.
- The system synthesizes diverse data streams, including voice, text, and behavioral logs, to map student progress and infer the next best action.
Quick answers
What happened in Study: Multimodal AI Improves Edtech Decisions?
A new study details an “intelligent educational decision-making system” that uses multimodal data fusion and knowledge graphs to personalize instruction. The system synthesizes diverse data streams, including voice, text, and behavioral logs, to map student progress and infer the next best action. This approach reportedly demonstrates superior accuracy in tailoring recommendations compared to single-modal systems, and the use of knowledge graphs offers greater transparency for its adaptation pathways.
Why does Study: Multimodal AI Improves Edtech Decisions matter?
Automatic speech recognition (ASR) systems trained on adult voices perform poorly with children due to differences in acoustic properties like higher pitch and greater variability. Even when trained specifically on children's speech, error rates can be 60% to 176% higher than for adult-trained systems on adult speech. Deep Knowledge Tracing (DKT), which often uses recurrent neural networks, is a common technique to model a student's knowledge over time. Multimodal approaches enhance DKT by incorporating not just text but also image, cognitive, and other data modalities to create a more accurate representation of a student's knowledge state. To decide the "next best action," many adaptive systems use a multi-armed bandit (MAB) framework to manage the exploration-exploitation tradeoff. This allows the system to efficiently test the effectiveness of different educational activities for a student while also serving the content that has historically worked best. Reinforcement learning (RL) can be used to create an optimal "policy" for sequencing content that maximizes a student's long-term learning. This involves mapping a student's current state, derived from interaction data, to the next piece of content, with the goal of improving metrics like future quiz performance. Knowledge graphs help make an AI tutor's decisions transparent by explicitly mapping the relationships between educational concepts. This allows the system to justify its recommendations, such as showing that a new topic is being introduced because the student has mastered all the prerequisite skills. Carnegie Learning's Cognitive Tutor, first developed at Carnegie Mellon in the 1980s, is an early and influential example of an AI-powered adaptive learning system in mathematics. The company's latest tool, LiveHint AI, uses a large language model trained on 25 years of student interaction data.
