FirstAidQA: A Synthetic Dataset for First Aid and Emergency Response in Low-Connectivity Settings

AI summary

Overview

Research area: Natural Language Processing, specifically synthetic dataset construction for safety-critical, domain-specific question answering (first aid and emergency response).

Technical level: Intermediate. The paper sits at the intersection of dataset engineering, prompt design, and domain safety review; no novel model architecture or training method is proposed.

Scope: The paper introduces FirstAidQA, a synthetic corpus of 5,500 question–answer pairs for first aid and emergency response, generated with ChatGPT-4o-mini from a certified first aid manual and validated by medical professionals, intended to support fine-tuning of models that can run offline in low-connectivity settings.

What This Paper Is About

Large Language Models are powerful but rely on cloud connectivity and substantial compute, which makes them impractical in disaster zones, rural clinics, remote regions, or other bandwidth-constrained environments where first aid guidance is most urgently needed. Existing medical QA benchmarks such as BioASQ, MedQA, and PubMedQA target clinical diagnostics or biomedical literature rather than the instructional, situational knowledge a layperson needs mid-emergency, and the authors state that no dedicated first aid QA dataset exists. The goal of this work is to fill that gap with a publicly released, expert-validated synthetic dataset that can be used to instruction-tune or fine-tune LLMs and Small Language Models (SLMs) for offline emergency assistance.

Key Contributions

1. The authors propose and publicly release FirstAidQA, which they describe as the first synthetic QA dataset specifically tailored to first aid and emergency response, consisting of 5,500 question–answer pairs.
2. They conduct human validation on a randomly sampled subset of the dataset and outline a framework for quality assurance in synthetic data pipelines.
3. They highlight directions for using FirstAidQA in fine-tuning smaller, deployable models for real-time, offline medical assistance.
4. They publish flagged examples of unsafe or inaccurate QA pairs (Appendix A) so that downstream users handle those items cautiously.

Main Findings

• Dataset size and composition: FirstAidQA contains 5,500 high-quality question–answer pairs covering general emergency procedures, CPR, road traffic accidents, moving casualties, first aid equipment and techniques, family and community safety, patient examination and monitoring, specific medical conditions, and neck and spinal injuries.
• Human evaluation scores: Three medical professionals scored a random sample of 200 QA pairs on a 1–5 scale, producing mean ratings of 4.7 for Relevance, 4.2 for Clarity, 4.0 for Specificity & Completeness, and 3.7 for Safety & Accuracy.
• Safety is the weakest axis: Safety & Accuracy received the lowest mean score of the four criteria, and the appendix documents specific flagged answers containing unsafe advice.
• Specific failure modes identified: Flagged items include recommending pressure immobilisation for anaphylaxis from a bee sting (item 95), blind finger sweeps during choking CPR (item 102), vinegar for a stingray sting rather than hot water immersion (item 128), freezing products to kill a tick before removal (item 137), oral thermometry described as most reliable (item 143), diluted sugary drinks for a child with diarrhoea (item 146), rinsing the mouth after ingesting bleach or drain cleaner (item 180), and conflicting guidance on twisting during tick removal (item 184).
• Gap in existing resources: The authors state that prior work in this space consists of FAQ-based chatbots (such as Dr.FirstAider) or evaluations of general assistants like Siri, Alexa, and ChatGPT, which often miss key evidence-based steps or give incomplete guidance.
• Related dataset scale for context: The paper cites EHR-DS-QA (156K QA pairs from discharge summaries), MedQuAD (47K QA pairs from NIH websites), and COVID-QA (approximately 2,000 curated pairs) as comparable synthetic or curated healthcare QA efforts.

Methodology in Plain English

The researchers started from a single authoritative source, the certified Vital First Aid Book (2019), chosen for its structured coverage of emergency care and its alignment with international standards from the American Red Cross and ILCOR. They cleaned the text, split it into context-preserving chunks, and filtered out material that was not practically applicable to real emergencies.

Each chunk was fed to ChatGPT-4o-mini inside a prompt that cast the model as an expert in creating synthetic datasets for first aid and medical emergencies, and asked for 20 diverse question–answer pairs in JSON format, with answers that were detailed, medically accurate, and written from multiple perspectives such as bystanders, trained responders, or lone rescuers. To expand coverage, they repeatedly appended the instruction "Generate 20 more question–answer pairs. Ensure they are not repeated from the previous response" until roughly 100 QA pairs per topic existed. Each batch was reviewed for accuracy and diversity, with prompt adjustments when needed, for example explicitly requesting pediatric or elderly cases. Repeating this across topics produced 5,500 pairs.

For quality assurance, they constrained the model to generate only from the supplied chunks to keep responses in context, and diversified the chunks to reduce bias. They then randomly selected 200 QA pairs from different contexts and had three medical professionals rate each on clarity, relevance, specificity and completeness, and safety and accuracy using a 1–5 scale.

Why This Matters

Impact on research: The paper provides a public, domain-specific resource for a safety-critical area where the authors argue no dedicated dataset previously existed, and it demonstrates a quality-assurance pipeline for synthetic data that other safety-critical domains could reuse. It also shifts attention toward edge and offline deployment rather than cloud-dependent inference.

Real-world applications:

• Offline first aid guidance on low-tier devices for first responders working without network access.
• Decision support in disaster zones, rural clinics, and remote regions where high-speed internet and modern computing infrastructure are unavailable.
• Layperson assistance for civilians in socio-economically disadvantaged areas who have little or no formal first aid training.
• Instruction-tuning and fine-tuning of small deployable models that can answer procedural questions about burns, bleeding, animal bites, and other common emergencies.

Industry relevance: Developers building mobile or embedded emergency-response assistants, humanitarian technology organizations, and health agencies evaluating conversational tools all have a stake in a dataset that targets bandwidth-constrained deployment. The flagged safety failures are directly relevant to anyone shipping medical-adjacent AI, since they show where synthetic generation drifts from current guidelines.

Future Directions

• Fine-tuning and instruction-tuning SLMs on FirstAidQA and measuring whether the resulting models can run reliably offline on low-tier devices.
• Improving the safety and accuracy dimension, which scored lowest (3.7 of 5), including guidance on how to handle or correct the flagged QA pairs in Appendix A.
• Extending the safety framework beyond first aid to other safety-critical synthetic data pipelines, as the authors propose a general quality-assurance approach.
• Expanding or auditing the dataset further, since the human validation covers only a random sample of 200 of the 5,500 pairs; the accuracy of the remaining pairs is not reported.

Target Audience

Researchers and practitioners working on domain-specific language model fine-tuning, especially those focused on low-resource, offline, or edge deployment; NLP dataset builders interested in synthetic data generation and validation pipelines; medical and emergency-response professionals evaluating AI-generated guidance; and developers building emergency or humanitarian assistance tools for bandwidth-constrained environments.

Authors’ abstract

In emergency situations, every second counts. The deployment of Large Language Models (LLMs) in time-sensitive, low or zero-connectivity environments remains limited. Current models are computationally intensive and unsuitable for low-tier devices often used by first responders or civilians. A major barrier to developing lightweight, domain-specific solutions is the lack of high-quality datasets tailored to first aid and emergency response. To address this gap, we introduce FirstAidQA, a synthetic dataset containing 5,500 high-quality question answer pairs that encompass a wide range of first aid and emergency response scenarios. The dataset was generated using a Large Language Model, ChatGPT-4o-mini, with prompt-based in-context learning, using texts from the Vital First Aid Book (2019). We applied preprocessing steps such as text cleaning, contextual chunking, and filtering, followed by human validation to ensure accuracy, safety, and practical relevance of the QA pairs. FirstAidQA is designed to support instruction-tuning and fine-tuning of LLMs and Small Language Models (SLMs), enabling faster, more reliable, and offline-capable systems for emergency settings. We publicly release the dataset to advance research on safety-critical and resource-constrained AI applications in first aid and emergency response. The dataset is available on Hugging Face at https://huggingface.co/datasets/i-am-mushfiq/FirstAidQA.