ATCion: Exploring the Design of Icon-based Visual Aids for Enhancing In-cockpit Air Traffic Control Communication

AI summary

Overview

• Research area: Human-Computer Interaction, specifically aviation human factors, information visualization, icon design, and multimodal (audio + visual) communication interfaces.
• Technical level: Intermediate. The paper is readable without an aviation background, but it uses domain terms such as clearances, readback, VFR/IFR phraseology, and CPDLC.
• Scope: An iterative, four-phase user-centered design study that develops and evaluates ATCion, a set of icon-based visual aids displayed in the cockpit to supplement voice communication between general aviation pilots and air traffic control (ATC).

What This Paper Is About

Pilots have relied primarily on audio radio communication with ATC since World War II, and verbal exchanges are prone to missed, misheard, or misinterpreted messages, especially under high workload. Existing visual support such as Controller Pilot Data Link Communications (CPDLC) delivers ATC messages as plain text but is limited to non-urgent en-route instructions, leaving critical phases and single-pilot general aviation largely unsupported. This paper investigates how icon-based visual aids should be designed and integrated into the cockpit to improve ATC communication, and tests them against text-based aids and audio-only communication.

Key Contributions

1. The first design study on icon-based aids for in-cockpit ATC communication, conducted through an iterative, user-centered process that produced practical design guidelines and a set of artifacts named ATCion (30 icons).
2. Rich qualitative insights into pilot communication challenges, intended to inform future multimodal interface design for safer and more effective ATC interactions.
3. Preliminary empirical evidence from a VR simulation study with real pilots comparing audio-only, text-based visual aids, and icon-based ATC communication.
4. A validated set of design principles and visual icon components for ATC messages, derived from four phases involving 22 pilots and 1 ATC controller (N = 23 unique domain experts).

Main Findings

• Improved readback accuracy: In the comparative study, ATCion significantly improved pilots' readback accuracy compared with the other conditions, particularly for longer messages.
• Reduced memory workload: ATCion reduced memory workload, again especially for longer messages.
• No harm to flight operations: Flight performance remained stable across conditions, meaning the visual aids did not negatively impact flight operations.
• Lower reported cognitive workload: Participants reported lower cognitive workload with ATCion.
• Strong preference for icons over text: Most participants preferred ATCion over text-based aids, citing clarity, low cognitive cost, and fast interpretability.
• Perceived usefulness under pressure: Participants found the icons intuitive, effective to interpret, and helpful under pressure, especially during high-demand phases such as approach and landing.
• Challenge profile identified in the formative study: Pilots reported that controllers sometimes speak faster than pilots can copy, that note-taking is not always feasible, and that fatigue ("after a long day of flying or after a long flight, you just forget") worsens the problem. Expectations and assumption of clearances also caused errors, including responding to a clearance meant for another aircraft after hearing only the first part of a call sign.
• Placement matters: Expert questionnaire data showed most pilot focus centers on the middle and bottom regions of the windshield, making these areas the priority for positioning visual aids.
• Prioritized message types: Working with experts, seven ATC message types were identified (instructions, clearances, requests, acknowledgments, general information, advisories, emergency) and grouped into three priority levels: first-priority (emergency, shown in the pilot's field of view), high-priority (instructions, clearances, requests, advisories, shown in fixed cockpit locations), and medium-priority (general information, acknowledgments, shown less intrusively).

Methodology in Plain English

The researchers ran a four-phase, user-centered process with a total of N = 23 unique domain experts, mostly pilots plus one controller.

1. Formative study (N = 7). A 20.5-hour semi-structured interview plus a follow-up questionnaire with pilots whose average Pilot in Command (PIC) time was 72 hours. Two of the seven were instructor pilots. Two researchers independently coded transcripts and refined codes until consensus. This produced four needs: reduce cognitive workload, mitigate miscommunication, maintain situational awareness without distraction, and fit within physical and view constraints in the cockpit.
2. Co-design (N = 7). Six experienced pilots (average PIC = 700 hours) and one air traffic controller with over 10 years of experience at an international airport. The team analyzed ATC message content and structure from standard VFR and IFR phraseology guides, decomposed messages into key actions and variables (for example "taxi via Alpha" where "taxi" is the action and "Alpha" is the variable), and mapped where pilots look across flight phases. This produced three design goals and the ATCion icon set of 30 icons.
3. Validation (N = 5). Experts rated each icon across five dimensions (such as concreteness, complexity, and meaningfulness) using an established icon-rating approach, and the representations were refined based on that feedback.
4. Evaluation (N = 12). A functional ATCion prototype was integrated into a flight simulator. In a within-subjects VR study, experts experienced three communication conditions: audio-only, audio with text-based visual aids, and audio with ATCion. The study targeted approach and landing, described as the most critical flight phases.

Why This Matters

• Research impact: The work opens an HCI research thread on icon-based aids for real-time in-cockpit ATC communication, an area the authors describe as having few HCI studies despite the documented benefits of icons for recognition speed, cognitive processing, and error reduction.
• Aviation safety context: The paper reports that communication issues contribute to 70% of aviation accidents and incidents, that 73% of runway incidents are linked to misinterpreted ATC instructions and coordination failures, and that miscommunication occurs on average once per hour per radio frequency in high-traffic areas. The Tenerife airport disaster of March 27, 1977, which killed 583 people, is cited as the deadliest accident in aviation history and as a miscommunication example.

Real-world applications:

• General aviation cockpits: Single-pilot operations, where one person manages all tasks and currently relies on memory or handwritten notes.
• Approach and landing phases: High-workload phases where the study found the greatest benefits for longer messages.
• Future multimodal ATC systems: Combining audio with icon-based visual aids as a secondary, redundant information channel.
• AR, HUD, and mixed reality displays: Placing compact aids within the pilot's natural line of sight near the windshield or instrument panel.

Industry relevance: The work speaks directly to avionics and cockpit display design, suggesting icon-augmented formats that abstract instructions while retaining essential ATC terms as text, and pointing toward integration with existing efforts such as CPDLC and automated speech recognition (ASR) systems that convert voice messages into text.

Future Directions

• Integrating icon-based aids into multimodal ATC communication systems: The paper explicitly points to implications and opportunities for combining icons with audio (and potentially text) to improve both safety and efficiency.
• Combining with ASR and CPDLC pipelines: The paper discusses ASR systems that convert voice messages into text-based visual aids and notes CPDLC's limits to non-urgent en-route text; how icon representations would fit with these systems is left open.
• Deployment beyond VR and beyond general aviation: The study used VR flight simulation and targeted general aviation. The paper does not report testing on physical head-up displays, mixed reality hardware, or multi-pilot commercial operations, leaving those as open questions.
• Richer validation: The evaluation had N = 12 experts and the study is described by the authors as offering "preliminary empirical evidence," so larger and longer-term studies are implied, though specific future study designs are not reported.

Target Audience

HCI and visualization researchers working on icon design, multimodal interfaces, and mixed/augmented reality; aviation human factors researchers studying communication, workload, and situational awareness; cockpit display and avionics designers; and general aviation pilots or flight instructors interested in how visual aids could supplement radio communication.

Authors’ abstract

Effective communication between pilots and air traffic control (ATC) is essential for aviation safety, but verbal exchanges over radios are prone to miscommunication, especially under high workload conditions. While cockpit-embedded visual aids offer the potential to enhance ATC communication, little is known about how to design and integrate such aids. We present an exploratory, user-centered investigation into the design and integration of icon-based visual aids, named ATCion, to support in-cockpit ATC communication, through four phases involving 22 pilots and 1 ATC controller. This study contributes a validated set of design principles and visual icon components for ATC messages. In a comparative study of ATCion, text-based visual aids, and no visual aids, we found that our design improved readback accuracy and reduced memory workload, without negatively impacting flight operations; most participants preferred ATCion over text-based aids, citing their clarity, low cognitive cost, and fast interpretability. Further, we point to implications and opportunities for integrating icon-based aids into future multimodal ATC communication systems to improve both safety and efficiency.