Uncertain Pointer: Situated Feedforward Visualizations for Ambiguity-Aware AR Target Selection

AI summary

Overview

Research area: Human-computer interaction, specifically augmented reality target selection, uncertainty visualization, and feedforward interaction design.

Technical level: Intermediate. The motivating problem (ambiguous pointing and speech in AR) is intuitive, but the paper leans on a systematic literature review with explicit coding categories and two pre-registered online experiments with defined conditions.

Scope: The paper builds and evaluates a design space of 25 "Uncertain Pointer" visualizations that annotate multiple candidate targets in AR, either to convey uncertainty or to help users disambiguate before confirming a selection.

What This Paper Is About

When a person wearing AR glasses points at a shelf or asks "what does that sign say?", the system often cannot tell which of several objects is meant, because speech is linguistically ambiguous, gaze or hand pointing is noisy while moving, and real-world targets vary in distance and clutter. The paper asks a design question rather than a recognition question: what should the system show the user when multiple targets are plausible, so that the ambiguity is visible and resolvable? The authors propose "Uncertain Pointer," a set of visualization designs that annotate candidate targets, and evaluate how those designs affect identifiability, target visibility, perceived effort, confidence, and preference across different object distances and densities.

Key Contributions

1. A systematic literature survey and pointer space. The authors surveyed 30 years of relevant publications at CHI, UIST, DIS, VRST, SUI, ACM IUI, AutomotiveUI, IEEE TVCG, ISMAR, IEEE VR, and IEEE 3DUI (search covering 1990–2025, following PRISMA guidelines), coded the resulting dataset, and used it to generate a pointer space of 25 candidate designs organized along three dimensions: uncertainty complexity type, visual signifier, and pointer archetype.

2. A taxonomy of pointer designs by uncertainty complexity. Three pointer types are defined: Certain pointers annotate a single object (included as a baseline for high-certainty selection and implicit disambiguation), Identity pointers give each candidate a distinct non-hierarchical identity such as a color or text label to support verbal clarification, and Level pointers modulate visual intensity (for example opacity, size, or luminance-and-saturation) to convey graded system confidence.

3. Two pre-registered online experiments. Study 1 (n = 60) evaluated Certain and Identity pointers across four target-complexity scenes and three target-count levels, and was used to exclude low-performing or incompatible scene–archetype combinations. Study 2 (n = 40) then evaluated Level visualizations.

4. Design recommendations for choosing pointers. Based on the results, the authors derive recommendations and usage examples for selecting different Uncertain Pointers depending on AR context and the disambiguation technique being supported.

Main Findings

• Four archetypes describe most prior target visualizations. From 120 eligible papers, the authors summarized 220 visualization techniques: 16.36% (36) external, 26.37% (58) internal, 18.18% (40) boundary, and 39.09% (86) fill. A residual "others" category covered techniques such as embodied virtual characters and blinking or animated motion cues, which the authors judged overly distracting for on-the-go use.

• Color dominates as a visual signifier. Of the coded signifiers, 102 were color (46.36%), 35 size (15.91%), 26 opacity (11.82%), 17 text (7.73%), 12 texture (5.45%), 8 shape (3.64%), 6 position (2.73%), 6 resolution (2.73%), 5 orientation (2.27%), and 3 length (1.36%). The paper names 11 signifier categories in total; no count is reported for angle.

• The review funnel. The boolean query returned 721 records (288 TVCG, 285 CHI, 32 ISMAR, 30 DIS, 27 UIST, 24 IEEE VR, 13 AutomotiveUI, 8 SUI, 5 VRST). Screening titles and abstracts selected 299 and excluded 422; full-text eligibility review excluded a further 179, leaving 120 papers. Two authors coded the dataset with an initial inter-rater agreement of 93%, with no discrepancies remaining after discussion.

• The pointer space is deliberately constrained. It combines the three uncertainty complexity types with the four pointer archetypes and the four most prevalent signifiers in the dataset (color, text, size, opacity), plus a "none" uniform-color signifier in the Identity set to serve as a baseline and to show the mere presence of uncertainty. For Level color encoding, the design uses increasing luminance combined with decreasing saturation, citing prior results that this combination works well for uncertainty.

• Identity and Level pointers are not interchangeable. Because size and opacity changes inherently imply a ranked hierarchy, they are excluded as identifiers in the Identity category; Identity relies on distinct attributes such as color or text instead. The paper frames the pointer types as complementary rather than mutually exclusive, suggesting combinations such as Level pointers for a first coarse selection followed by Identity or Certain pointers for finer resolution.

• Explicit visualizations were under-evaluated. The authors note that disambiguation visualizations are commonly deployed as lassos, colored areas, or bounded regions but are seldom systematically compared, and that most prior visualizations were tested in plain scenes with tightly controlled object characteristics rather than at varying real-world distance and density.

• Study-level detail (outcome numbers not reported in the available text). Study 1 used 4 scenes (near/far × dense/sparse) and 3 target-count levels, with two embedded attention checks and a 5-minute break after every quarter of trials; 12 participants failed the attention check and were replaced until 60 eligible participants were reached, a sample size set by a pre-registered power analysis targeting 90% statistical power for comparisons across archetype, signifier, and archetype × scene. The specific condition-level results and the Study 2 findings are not reported in the portion of the paper available here.

Methodology in Plain English

The authors first looked backward before designing anything. They ran a structured search (not a keyword search) over the digital libraries of major HCI, VR, AR, and visualization venues, restricting to title or abstract matches on uncertainty, feedforward, in-situ or selection visualization, and covering full papers from 1990 to 2025. They screened titles and abstracts, then full texts, keeping only papers that (1) communicate feedforward or uncertainty information directly to users and (2) present a visualization for visible targets with area or volume that could plausibly be applied to real-world target selection. Abstract data graphics like density plots were excluded, as were uncertainty methods that use only audio or haptics and methods aimed purely at robot planning or internal model computation.

Each eligible paper was coded along two dimensions: the pointer archetype (where and how the annotation attaches to the referent) and the visual signifier (the visual property used to distinguish referents). Two authors coded independently and resolved disagreements by discussion. The coded patterns were then recombined with the three uncertainty complexity levels to produce the 25-design pointer space.

For evaluation, the team generated mock-up AR videos in Unity from Gaussian splat captures of real environments, manually annotating 3D objects to apply the visualization effects and adding simulated handheld camera shake through Cinemachine to mimic an AR wearer moving. Participants in an online Qualtrics study watched these videos and completed object-counting tasks plus subjective ratings — they did not perform actual target selection in the scene. The study was pre-registered and pre-planned with a power analysis. Study 1 handled the Certain and Identity designs to screen out weak scene–archetype combinations; Study 2 applied the surviving setup to Level designs.

Why This Matters

The work reframes AR disambiguation as a visualization and communication problem rather than purely a sensing problem. When sensing is degraded by motion, occlusion, or noise, implicit disambiguation can silently pick the wrong target; showing candidates explicitly brings the uncertainty into the user's awareness and gives them something to act on. This matters for research because the paper argues that disambiguation visualizations have rarely been systematically compared, and it supplies a shared design space, coding scheme, and comparison method for doing so.

Real-world applications include:

• Assistive and information queries on the go. A pedestrian asking about a distant traffic sign or storefront gets visually distinguished candidates and can clarify verbally, for example by naming a color.
• Retail and grocery shopping. A shopper pointing at a shelf where several items fall along the pointing ray sees the candidate set narrowed and can confirm the intended product.
• Remote collaboration and task guidance. Workers referencing equipment in cluttered environments can resolve which of several similar objects is meant before an instruction is carried out.
• Vehicle and navigation interfaces. Uncertainty-aware pointers could indicate system confidence when referencing road features or points of interest, echoing prior uncertainty displays for driving.

Industry relevance centers on smart glasses and AR headsets, which the paper names as Meta Orion, Snapchat Spectacles, and VIVE Eagle. Multimodal interactions on these devices — pointing, gesturing, and speaking — are exactly the inputs that produce the ambiguity this work addresses, so the findings feed directly into the design of selection and assistant interactions on lightweight, always-on hardware.

Future Directions

• How the pointer types combine in a full workflow. The paper states that Level, Identity, and Certain pointers can be layered, for example coarse Level selection followed by Identity refinement, but does not report evaluations of combined pipelines; testing combinations is a natural next step.
• Evaluation under real selection, not counting tasks. The reported studies use passive video viewing with object-counting and ratings rather than actual target selection, leaving open how these visualizations perform when users are actively choosing and confirming.
• De-emphasis approaches in AR. The authors deliberately avoid blurring or diminishing-reality techniques because they hide coarse-selected targets and generalize poorly to optical see-through devices; whether any de-emphasis scheme can be made viable remains open.
• Generalization beyond the four studied scenes. The evaluation covers near/far × dense/sparse with three target-count levels; behavior in scenes with moving targets, extreme clutter, or different lighting and occlusion conditions is not established here.

Target Audience

This paper is most useful to HCI and AR researchers and designers working on target selection, multimodal input, and uncertainty visualization, particularly those building selection or assistant features for smart glasses and headsets. It also serves interaction designers who need concrete guidance on when to use color-based identity labels versus graded intensity cues, and students or practitioners entering the field who want a structured overview of 30 years of target visualization techniques along with a reusable design space.

Authors’ abstract

Target disambiguation is crucial in resolving input ambiguity in augmented reality (AR), especially for queries over distant objects or cluttered scenes on the go. Yet, visual feedforward techniques that support this process remain underexplored. We present Uncertain Pointer, a systematic exploration of feedforward visualizations that annotate multiple candidate targets before user confirmation, either by adding distinct visual identities (e.g., colors) to support disambiguation or by modulating visual intensity (e.g., opacity) to convey system uncertainty. First, we construct a pointer space of 25 pointers by analyzing existing placement strategies and visual signifiers used in target visualizations across 30 years of relevant literature. We then evaluate them through two online experiments (n = 60 and 40), measuring user preference, confidence, mental ease, target visibility, and identifiability across varying object distances and sparsities. Finally, from the results, we derive design recommendations in choosing different Uncertain Pointers based on AR context and disambiguation techniques.