CLAWS:Creativity detection for LLM-generated solutions using Attention Window of Sections

AI summary

Overview

Research area: Natural Language Processing — LLM reasoning, interpretability, hallucination and creativity detection.

Technical level: Intermediate. The paper assumes familiarity with transformer attention, decoding, and standard uncertainty metrics, but the core idea (comparing how much attention a model pays to different parts of its prompt) is explainable without specialized background.

Scope (one sentence): The paper defines and evaluates CLAWS, a white-box method that classifies LLM-generated mathematical solutions as Hallucinated, Creative, or Typical by measuring the average attention weight each section of the input prompt receives.

What This Paper Is About

Large Language Models trained with reinforcement learning now solve math problems at high accuracy, but accuracy says nothing about whether a solution is creative — and creativity in reasoning has received far less attention than creativity in writing tasks. Assessing it normally requires human domain experts, which is costly and hard to standardize.

This paper asks whether a model's own internal attention patterns can reveal, without any human judgment, whether the solution it just produced is Hallucinated, Creative, or Typical.

Key Contributions

1. An evaluation framework that labels generated solutions as Hallucinated, Creative, or Typical using LLM evaluators instead of human annotators, applied to math reasoning tasks with reasoning language models.

2. CLAWS itself — a white-box detection method that splits the prompt and response into five sections (Guideline, Problem, reference Solutions, Instruction, Response), computes the average attention weight each section receives, normalizes those into ratios, and uses the resulting vector as the detection feature.

3. A comprehensive evaluation protocol consisting of five evaluation strategies (Threshold, Prototypes, XGBoost, MLP, TabM) and four metrics (weighted F1, macro F1, macro Average Precision, AUROC) for comparing detection methods.

4. Empirical validation across five 7–8B math RL models (DeepSeek, Qwen, Mathstral, OpenMath2, Oreal) and a dataset of 4,545 math problems collected from 181 math contests (A(J)HSME, AMC, AIME), showing CLAWS outperforms five existing white-box baselines.

Main Findings

• CLAWS beats all five baselines on creativity detection. Against Perplexity (PPL), Window Entropy (WE), Logit Entropy (LE), Hidden Score (HS), and Attention Score (AS), CLAWS was reported as outperforming every baseline on the test set across all four metrics. On the DeepSeek generator, CLAWS scored F1w 58.66, F1m 46.01, APm 41.17, and AUROC 62.09 on TEST, versus 48.09, 35.77, 37.07, and 56.49 for the closest baseline (PPL).

• CLAWS also dominates hallucination detection. When Creative and Typical were merged into a Non-hallucinated class, CLAWS again showed the strongest performance: on the DeepSeek TEST set it reached F1w 67.46, F1m 67.24, APm 55.73, AUROC 67.78; on Mathstral TEST it reached 72.99, 69.59, 49.42, 69.30.

• Strength is concentrated in macro-averaged metrics. The gains are most pronounced on F1m and APm, which weight all classes equally, and also appear on F1w, which weights larger classes more. The authors interpret this as evidence that CLAWS handles Creative solutions as well as the more frequent Typical and Hallucinated classes.

• Attention patterns differ by class. In the reference-set visualizations, Hallucinated solutions show relatively more attention on the Guideline and Problem sections, while Typical solutions attend more to the reference Solution, Instruction, and Response sections. Creative solutions fall between the two extremes. Hallucination focusing on the Guideline section appeared across all models, which the authors suggest means hallucinations may stem from over-focusing on part of the input prompt.

• Oreal is the exception on extended test sets. CLAWS performed well on most models across AMC, AIME, and A(J)HSME, but not on Oreal, which generated a much higher rate of Hallucinated solutions than the others, making a clean prototype hard to build. In that case, the authors report CLAWS improved substantially under the MLP, XGBoost, and TabM strategies.

• Class imbalance shapes baseline failure. Qwen had the most balanced Hallucinated/non-hallucinated distribution and all methods did consistently well on it. Mathstral and Oreal each produced roughly 1,200 Hallucinated solutions in the reference set — a hallucination rate of nearly 70% — and all five baselines performed poorly on them. On DeepSeek and OpenMath2, baselines were moderate on the easier AMC and A(J)HSME sets but dropped substantially on AIME.

• A balanced dataset helps. For each reference set, the authors kept all Creative samples and randomly sampled an equal number from Hallucinated and Typical — for example, the 206 Creative samples in DeepSeek's reference set were matched with 206 samples each from Hallucinated and Typical. CLAWS achieved the best overall performance there, performed best under the MLP strategy, and degraded least when trained on small amounts of data.

• All five sections matter. In an ablation on DeepSeek-Math-7B-RL using the Prototype strategy, removing the reference Solutions section hurt most (TEST F1w dropped from 58.66 to 50.01, F1m from 46.01 to 39.45, APm from 41.17 to 38.46, AUROC from 62.09 to 59.15). Removing the Instruction section also hurt (54.68 F1w). The full five-section configuration gave the best overall results.

• CLAWS is also the cheapest. In the runtime comparison of feature computation (excluding the shared response-generation time), CLAWS was the most efficient method. Among entropy-based methods, WE was least efficient and LE most efficient; among layer-level methods, HS was least efficient and AS was the most efficient apart from CLAWS.

Methodology in Plain English

The workflow has four steps. First, a reasoning model receives a prompt built from four sections: a Guideline describing how to judge whether two solutions differ creatively, the Problem itself, one or more reference Solutions (at most two in the test and extended test sets), and an Instruction telling the model to produce a novel solution unlike the references. The model generates a response, which becomes the fifth section.

Second, two frontier LLMs — GPT-o4-mini and Gemini-1.5-Pro — act as evaluators and label each response. If neither evaluator judged it correct, it is Hallucinated. If both judged it correct but at least one judged it creative, it is Creative (the authors call this an inclusive criterion). If both judged it correct and neither judged it creative, it is Typical. No humans are involved.

Third, CLAWS extracts the decoder attention weights from the last layer during generation. It records, at each decoding time step and for each attention head, how much weight went to each token of the prompt and the response so far. These vectors are stacked and padded to a fixed length, then averaged over heads, over time steps, and over the tokens belonging to each of the five sections. The five resulting average values are normalized into ratios that sum to one, giving a five-dimensional feature vector describing where the model "looked."

Fourth, those features are used for classification. The five baselines produce a single scalar each and are evaluated with a tuned threshold. CLAWS produces a vector, so it is evaluated using prototypes (Euclidean distance to each class's center from the reference set), and also with trainable classifiers: XGBoost, an MLP, and TabM.

The data comes from CreativeMath and HARP. CreativeMath contains 400 problems sampled and cleaned from 50 questions across eight contests (AMC 8, 10, 12, A(J)HSME, AIME, USAJMO, USAMO, IMO). HARP contains 5,409 problems from A(J)HSME, AMC, AIME, and USA(J)MO. After removing the 282 overlapping problems and excluding problems that were too difficult (for example, proof-based) or excessively long, the extended set contains 4,545 problems with solutions. 29 CreativeMath problems form the reference set (20 stochastic responses each), the remaining 371 form the test set (three responses each, reference solutions capped at two), and the HARP problems form the extended test set (one response each, reference solutions capped at two). Input length was limited to 2,048 tokens and output to fewer than 1,024 tokens.

Why This Matters

Impact on research. Most hallucination-detection work stops at catching errors. This paper treats creativity as a third, distinguishable state between "right but ordinary" and "wrong," and argues that aggressively suppressing hallucination may also suppress diversity. It also shows that a model's section-level attention distribution is a useful, cheap signal, which connects prompt-structure research with internal-state interpretability. The released code and the five-strategy, four-metric protocol give later work a comparable baseline.

Real-world applications:

• Reasoning-model evaluation pipelines that need to score not just accuracy but solution diversity, without paying for expert human raters.
• Math education platforms using LLM tutors, where distinguishing a student's or a model's creative method from a memorized template or a fabrication changes what feedback is useful.
• Reinforcement-learning training of reasoning models, where a creativity signal could be used to avoid collapsing generations into repetitive, typical outputs.
• Production monitoring of deployed reasoning assistants, where one cheap pass over attention weights can flag likely hallucinations without extra model calls.

Industry relevance. CLAWS requires no extra model calls, no majority voting over multiple generations, and no external evaluator at inference time — only summation and averaging over attention weights already produced during decoding. For companies serving reasoning models at scale, that means a detection layer with roughly the same overhead as simply reading hidden states, which the paper reports as faster than all five baselines tested.

Future Directions

• Fixing the Oreal failure case. CLAWS's prototype strategy struggled when a generator produced a heavily skewed class distribution (nearly 70% hallucinated). Determining the best strategy under severe imbalance — the balanced-dataset experiment is one attempt — remains open.
• Whether creativity definition can generalize. The Creative label depends on LLM evaluators and a Guideline that specifies what counts as creative. Whether this transfers to domains beyond competition mathematics, or to writing tasks where the Torrance framework is already used, is untested here.
• Cross-model consistency. Attention emphasis varied by model in the visualizations, yet one frozen CLAWS formulation was used throughout. How the five section weights should be adapted per model, and whether a universal configuration exists, is unresolved.
• Using CLAWS as a training signal rather than only a detector. The paper motivates creativity detection partly by the risk that hallucination mitigation suppresses diversity, but it does not test whether CLAWS features could steer generation toward more creative output.

Target Audience

This paper is most useful to NLP and LLM researchers working on interpretability, uncertainty quantification, or hallucination detection who want a method that extends to multi-class creativity labels; to researchers studying mathematical reasoning and evaluation benchmarks for RL-trained models; and to ML engineers building evaluation or monitoring layers for deployed reasoning systems who need a low-overhead feature extractor. Readers should be comfortable with attention mechanics and standard white-box detection metrics, though the framework itself is described in enough detail to follow without deep mathematics. The paper was published under CC BY 4.0, and code is available at https://github.com/kkt94/CLAWS.

Authors’ abstract

Recent advances in enhancing the reasoning ability of large language models (LLMs) have been remarkably successful. LLMs trained with reinforcement learning (RL) for reasoning demonstrate strong performance in challenging tasks such as mathematics and coding, even with relatively small model sizes. However, despite these improvements in task accuracy, the assessment of creativity in LLM generations has been largely overlooked in reasoning tasks, in contrast to writing tasks. The lack of research on creativity assessment in reasoning primarily stems from two challenges: (1) the difficulty of defining the range of creativity, and (2) the necessity of human evaluation in the assessment process. To address these challenges, we propose CLAWS, a method that defines and classifies mathematical solutions into typical, creative, and hallucinated categories without human evaluation, by leveraging attention weights across prompt sections and output. CLAWS outperforms five existing white-box detection methods (Perplexity, Logit Entropy, Window Entropy, Hidden Score, and Attention Score) on five 7-8B math RL models (DeepSeek, Qwen, Mathstral, OpenMath2, and Oreal). We validate CLAWS on 4545 math problems collected from 181 math contests (AJHSME, AMC, AIME).