ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

AI summary

Overview

• Research area: Natural Language Processing / multimodal large language model (MLLM) security, with a focus on automated chart generation and misleading visualization design.
• Technical level: Intermediate — the paper combines jailbreak-prompting concepts, chart question answering (QA) benchmarks, and parameter-efficient fine-tuning, but each component is explained in accessible terms.
• Scope: The paper introduces ChartAttack, a framework that automatically turns faithful chart specifications into misleading but data-consistent charts, and AttackViz, a labeled chart QA corpus used to measure how much those charts deceive MLLMs and human readers.

What This Paper Is About

Multimodal LLMs are increasingly asked to generate charts from data tables, and while this makes chart creation easier, the same capability can be abused to produce charts that look data-faithful yet push a reader toward a wrong conclusion. The authors ask whether MLLMs can be prompted at scale to apply "misleaders" — design choices such as truncated axes, 3D effects, or stacked bars — that leave the underlying data untouched but induce a target incorrect answer. Their goal is to quantify that attack, provide a labeled dataset of paired correct and misleading charts, and test whether prompting safeguards or fine-tuning can help.

Key Contributions

1. ChartAttack, described as the first framework for automatically generating misleading charts through reproducible, parameterizable misleaders that target a specific chart question and a specific wrong answer.
2. AttackViz, a multi-label chart QA dataset built from PlotQA and extended cross-domain with ChartQA and ChartX, containing simplified data-consistent chart annotations, questions, correct answers, and structured annotations for each applied misleader and the incorrect answer it induces.
3. An extensive evaluation of misleading chart attacks across 19 models (16 open-weight instruction-tuned MLLMs and three closed models), reporting relaxed accuracy and two conditional deception-rate metrics.
4. A controlled human study and mitigation experiments, including a between-subjects study with 48 participants and fine-tuning of Qwen2.5-VL-3B-Instruct on AttackViz using QLoRA.

Main Findings

• Accuracy degradation: ChartAttack reduces average MLLM QA accuracy by 17.2 percentage points (pp) in-domain and 11.9 pp cross-domain. In-domain drops range from 4.4 to 26.6 pp; cross-domain declines average 11.5 pp on ChartQA and 12.3 pp on ChartX, with per-model drops of 4.2–19.1 pp.
• Targeted deception, not random error: In-domain, correct answers shift to the attacker-intended answer 11.2% of the time on average, while originally incorrect answers change only 1.7% of the time. Cross-domain rates are 11.7% (ChartQA) and 14.9% (ChartX) for originally correct answers, and 2.7%/1.9% for originally incorrect answers.
• Stronger models are not immune: InternVL-3.5 (14B/38B) and Ovis-2.5 (2B/9B) reach 80–86% accuracy on correct charts but decline by 22–27 pp. Mid-range models including Claude-4.6, GPT-4o, Gemini-2.5, Qwen3-VL, and smaller InternVL variants (52–77% on correct charts) drop by 12–21 pp, while LLaVA-1.6 variants (28–44%) drop by 4–10 pp.
• Size does not determine robustness: InternVL-3.5 drops rise from 15.8 pp (1B) to 21.3 pp (8B) and 26.6 pp (38B), and LLaVA-1.6 drops grow from 4.9 pp (7B) to 9.8 pp (34B), but the trend is not monotonic. The 8B model shows smaller drops than the 4B model for dual axes (4.2 vs. 12.5 pp) and truncated axes (3.4 vs. 10.1 pp), yet a larger drop for inappropriate stacking (53.1 vs. 47.3 pp).
• Perceptual misleaders dominate in-domain: Stacked charts, 3D charts, and inappropriate log scales reduce accuracy to 24.6%, 34.2%, and 42.1% — drops of 41.5 pp, 30.6 pp, and 18.8 pp — with deception rates on originally correct answers of 20.0%, 10.7%, and 6.9%. Misrepresentation and inverted axes cause moderate declines of 18.8 pp and 19.4 pp with deception rates of 9.6% and 15.6%. Inappropriate line charts drop 9.0 pp (6.5% deception), and ineffective color schemes show a 0.4 pp increase with 1.2% deception. Dual axes show only a 1.4 pp overall decline but still produce 10.9% deception.
• Cross-domain misleader behavior: 3D remains most impactful, reducing accuracy to 27.9% and 22.7% on ChartQA and ChartX (drops of 55.1 pp and 61.4 pp) with deception rates of 4.2% and 5.0%. Misrepresentation yields 56.9% and 54.1% accuracy (-20.6 pp and -23.0 pp), and inappropriate stacked bars yield 59.3% and 55.9% (-17.2 pp and -20.5 pp). Dual axis and ineffective color scheme remain largely ineffective (+0.2 pp and 0.0 pp, and 1.8 pp and 2.6 pp drops), while line charts, log scales, axis ranges, and truncated axes show only modest drops despite working in-domain.
• Human readers are affected: In the between-subjects study, participants exposed to misleading charts had significantly lower odds of answering correctly (OR = 0.266, 95% CI [0.197, 0.357], p < 0.001), about 73% lower odds. Second-phase accuracy fell from 88.3% in the control group to 71.9% in the experimental group.
• Prompt-level guards fail: Adding a system-level guard instruction to the Misleader-generator did not reduce the effective attack success rate (ASR_eff) across three attacker settings: 0.988 with and without the guard for Qwen (line, zero-shot), 0.901 for Qwen (vertical bar, few-shot-5), and 0.727 for DeepSeek (horizontal bar, few-shot-5).
• Fine-tuning helps in-domain with trade-offs: On the in-domain PlotQA partition, the base Qwen2.5-VL-3B model scores 41.66% on the AttackViz test set versus 52.17% after fine-tuning (+10.51 pp), with PlotQA gains across all misleaders ranging from 11.73 to 48.78 pp. Cross-domain transfer is mixed: on ChartQA six misleaders improve (+1.34 to +2.85 pp) while others decline, most notably inverted axis (-14.87 pp); on ChartX nine of ten applicable misleaders decline, with only misrepresentation improving (+1.23 pp). Clean-chart accuracy also falls by 10.89 pp on PlotQA, 11.74 pp on ChartQA, and 23.89 pp on ChartX.
• Better than a random baseline, but only partly: On InternVL-3.5-14B, ChartAttack reduces relaxed accuracy by 25.3 pp versus 22.7 pp for random technique selection, and causes 13.2% targeted deception. The advantage concentrates in techniques requiring parameter selection, such as truncated axis and inappropriate axis range; single-flag techniques like 3D, log scale, and inverted axis often produce identical charts under both methods.

Methodology in Plain English

The researchers start from chart annotations — JSON files that specify the underlying data, axes, labels, and formatting — rather than from chart images. ChartAttack has two parts. A Demonstration Selection module retrieves similar examples from a large corpus using a fine-tuned SBERT model (trained with Multiple Negative Ranking Loss, where two examples count as similar if their sets of misleaders match exactly). A Misleader Generator, built on code-based instruction-tuned MLLMs selected via Human-Eval performance, then receives the chart annotations, the question, and the retrieved examples, and in a single call selects compatible misleaders, specifies minimal modifications, and produces a plausible but incorrect answer that preserves the correct answer's type and units. A separate retriever is trained for each chart type.

AttackViz is built from PlotQA, with annotations simplified (bounding boxes, label coordinates, and figure geometry removed; content reorganized into categories, values, legends, and colors) and verified against the CSV data tables. Phi-3.5-vision extracts formatting details such as grids, bands, and stacking. The authors subsample 400 images per chart type per partition and keep five questions per chart. Misleading variants come from a Python/Matplotlib rule-based system implementing 11 misleaders from the taxonomy of Lo et al. (2022), chosen by six criteria including that at least five real-world occurrences exist, that the correct answer remains recoverable, and that the underlying data table stays correct. Filtering uses relaxed accuracy with three instruction-tuned MLLMs (QwenVL 2.5-32B, InternVL 3.0-38B, KimiVL-A3B): instances are retained when the majority answer correctly on the original chart but incorrectly on the misleading one, with a consistency filter requiring numeric answer standard deviation below 0.5 or a majority identical textual answer. The same pipeline is applied to ChartQA and ChartX for cross-domain extension.

Evaluation uses relaxed accuracy (numeric answers within a 5% tolerance count as correct) plus two deception-rate metrics that use the attacker's misleading answer as the target: one for instances originally answered correctly, one for instances originally answered incorrectly.

Why This Matters

Impact on research. The paper reframes chart generation safety: data faithfulness is not sufficient, because a chart can be perfectly consistent with its data table and still systematically mislead. It also provides a reusable, structured corpus in which the underlying data is unchanged, isolating the effect of design-level misleaders from data falsification, and shows that robustness to misleading charts does not simply scale with model size.

Real-world applications:

• News and journalism: charts are named as a primary vehicle for communicating politics, climate, and healthcare information, and misleading design has been documented as a misinformation channel.
• Public health and crisis communication: the paper references the COVID-19 pandemic as a period when charts played a critical role in public understanding.
• Business dashboards and reporting tools: the attack targets chart JSON annotations, so it applies to automated chart generation pipelines rather than only image editors.
• Social media and content platforms: MLLMs can generate misleading content at scale, which is relevant to moderation and provenance decisions.

Industry relevance. Any product that lets users generate charts from tables — BI platforms, spreadsheet assistants, report builders, or visualization libraries — inherits this attack surface. The paper's negative result on prompt-level guards matters for teams currently relying on system instructions as a safety control, and its mixed fine-tuning result matters for teams considering robustness training: in-domain gains came with reduced clean-chart accuracy and poor cross-domain transfer. The released corpus is multi-label and annotation-level, which the authors note makes it compatible with other visualization libraries beyond Matplotlib.

Future Directions

• Reasoning misleaders. The study explicitly limits itself to design misleaders from the Lo et al. (2022) taxonomy. Misleaders that manipulate titles or annotations without breaking explicit design rules remain underexplored and are flagged as future work.
• Broader chart coverage. The framework covers horizontal bar, vertical bar, and line charts, which account for 64% of the taxonomy's misleading charts and 49% of real-world misleading visualizations in MisViz. Extending to other chart types is left open, with the note that pie charts are affected by two and map charts by three of the eleven techniques.
• Richer and larger human evaluation. The human study used a limited participant population and a controlled QA setting. The authors call for broader populations, additional chart-analysis tasks, and communication contexts such as news articles, dashboards, and social media, where source credibility, layout, interactivity, topic familiarity, and prior beliefs come into play.
• Defenses that actually work. Simple system-level guard prompts did not reduce attack success, and fine-tuning improved in-domain robustness while degrading clean-chart accuracy and transferring poorly cross-domain. The paper leaves robust detection, robustness-aware training, and generalizable mitigation as open challenges, and notes a compositional-misleader experiment (limited to AttackViz-PlotQA vertical bar charts, one generator model, one victim model, and three predefined combinations) as an initial step.

Target Audience

This paper is most useful to researchers working on multimodal LLM safety, jailbreak and prompt-injection attacks, and trustworthy visualization; to NLP and visualization practitioners evaluating chart QA models; and to product and security teams building automated chart-generation features who need to understand what design-level manipulations their systems can be induced to produce. Readers looking for concrete numbers on misleader effectiveness, per-model vulnerability, and the limits of prompt-based and fine-tuning-based defenses will find the evaluation and tables directly actionable.

Authors’ abstract

Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, improving efficiency but introducing new misuse risks. We present ChartAttack, a framework for evaluating how MLLMs use design misleaders to generate charts that induce incorrect interpretations. We also introduce AttackViz, a chart question-answering (QA) dataset labeled with effective misleaders and their induced incorrect answers. ChartAttack reduces MLLM QA accuracy by 17.2 points in-domain and 11.9 points cross-domain. Conditional deception rates show targeted effects: correct answers shift to attacker-intended answers 11.2\% of the time in-domain and 11.7-14.9\% cross-domain, while originally incorrect answers rarely change. A controlled human study shows that ChartAttack-generated charts also reduce human QA performance. Finally, fine-tuning on AttackViz improves in-domain MLLM robustness to misleading charts. Our findings highlight the need for secure, robust MLLM chart generation. Code and data are publicly available on the project website.