High-Order Question Generation in a Multilingual Educational Context

AI summary

Overview

Research area: Natural Language Processing / educational technology — automatic question generation with large language models, applied in a multilingual classroom context.

Technical level: Intermediate. The paper assumes familiarity with large language models, prompting, and educational question-classification frameworks such as Bloom's Taxonomy, but the abstract presents the work at a conceptual rather than technical depth.

Scope: A single-sentence scope: the paper introduces prompts based on two questioning frameworks other than Bloom's Taxonomy and tests whether open-source and proprietary LLMs can use them to generate high-order questions in Basque, Spanish, and English.

What This Paper Is About

Critical thinking is a core educational skill, and one way to build it is by asking learners "high-order" questions that require reasoning rather than recall. Teachers find such questions hard to write, and classrooms therefore lean on low-order questions. Large language models have shown promise at generating higher-order questions, but research has concentrated almost entirely on one framework (Bloom's Taxonomy) and one language (English). This paper's goal is to test whether prompts built on two different frameworks — Claim-Evidence-Reasoning and Divergent Questioning — can produce high-order questions across three languages.

Key Contributions

1. Alternative prompting frameworks for question generation. The authors introduce prompts grounded in Claim-Evidence-Reasoning and Divergent Questioning, moving beyond the dominant reliance on Bloom's Taxonomy in prior work.

2. A multilingual evaluation setting. The study tests question generation in Basque, Spanish, and English rather than English alone, addressing a stated gap in existing research.

3. Comparison across model types. Both an open-source model and a proprietary model are used, allowing the authors to observe behavior across two different classes of LLM.

4. Teacher-grounded assessment of question quality. The generated questions are assessed by teachers, including a judgment of whether the answerable questions actually count as high-order.

Main Findings

• Generation works across languages and models: Both the open-source and the proprietary model generate questions "rather effectively" in all three languages (Basque, Spanish, and English).

• Teacher recognition falls short of the goal: Only about half of the answerable questions were recognized by teachers as high-order — the abstract's central caveat on quality.

• The alternative frameworks yield variety: Prompts based on Claim-Evidence-Reasoning and Divergent Questioning produced questions that were structurally and conceptually varied.

• The frameworks may be complementary: The authors suggest the two frameworks could complement each other and serve as viable alternatives to Bloom's Taxonomy.

Note that the abstract reports no model names, no counts of questions generated or evaluated, no per-language breakdowns, and no comparison figures beyond the "about half" teacher-recognition result. Those details are not given in the abstract.

Methodology in Plain English

The researchers designed prompts that instruct a language model to produce questions according to two established pedagogical frameworks — Claim-Evidence-Reasoning and Divergent Questioning — instead of the more commonly used Bloom's Taxonomy. They applied these prompts in three languages: Basque, Spanish, and English. Two models were used, one open-source and one proprietary, so the approach could be checked against more than one system. The resulting questions were then put in front of teachers, who judged whether the answerable ones qualified as high-order questions. The abstract does not describe how the prompt templates were constructed, how many questions were produced, how teachers were recruited, or what agreement procedure was used.

Why This Matters

Impact on research: The paper pushes the field past a near-exclusive focus on Bloom's Taxonomy and English, offering two alternative prompt frameworks and a trilingual testbed. It also highlights a gap between what a model generates and what teachers actually accept as high-order — a distinction that evaluation practices based purely on automatic metrics could miss.

Real-world applications:

• Classroom question banks that teachers can draw on across Basque, Spanish, and English.
• Teacher-training and lesson-planning tools that scaffold higher-order questioning in subjects where evidence-based reasoning matters (for example science and civics, where Claim-Evidence-Reasoning fits naturally).
• Curriculum and assessment design, where question "order" affects what a test actually measures.
• Multilingual education systems, particularly those serving regional or minority languages such as Basque that are under-represented in NLP tooling.

Industry relevance: EdTech vendors building question-generation, tutoring, or assessment products can use the findings to judge whether a single framework is enough (the abstract's evidence suggests not) and to calibrate expectations about how often generated questions will pass teacher scrutiny. Model providers also get a signal that multilingual educational generation is a use case worth supporting beyond English.

Future Directions

• Close the quality gap. Since only about half of answerable questions were accepted as high-order, follow-up work could refine prompts or add filtering to raise that rate.
• Combine the two frameworks. The abstract explicitly suggests Claim-Evidence-Reasoning and Divergent Questioning could complement each other, raising the question of what a merged prompting strategy would produce.
• Extend language coverage. The study covers Basque, Spanish, and English; whether the same prompt strategies transfer to other languages and educational systems is untested.
• Move from teacher judgment to learner outcomes. The abstract reports teacher recognition only — whether these questions actually improve critical thinking in students remains an open question.

Target Audience

Education researchers and practitioners interested in higher-order questioning, NLP researchers working on educational question generation, researchers in multilingual and low-resource language technology (particularly those working on Basque and other European regional languages), and EdTech developers or product teams building question-generation or assessment tools.

Authors’ abstract

Critical thinking is a fundamental skill that helps learners move beyond simple memorization. One way to develop this skill is through high-order questioning. However, crafting such questions remains a challenge for educators, and classroom practices tend to rely on low-order questions. Large Language Models have demonstrated strong capabilities in generating high-order questions, especially when guided by prompts based on Bloom's Taxonomy. Yet, existing research has largely centered on this framework and focused only on English. This study addresses these gaps by introducing prompts grounded in two alternative frameworks: Claim-Evidence-Reasoning and Divergent Questioning within a multilingual context using Basque, Spanish, and English. Results indicate that while both an open-source and a proprietary model rather effectively generate questions in all three languages, only about half of the answerable questions are recognized by teachers as high-order. A positive finding is that the alternative frameworks produce structurally and conceptually varied questions, suggesting they could complement each other and provide viable alternatives to Bloom's Taxonomy.