Characterizing Human Semantic Navigation in Concept Production as Trajectories in Embedding Space

AI summary

Overview

Research area: Natural Language Processing, specifically text-embedding–based modeling of human semantic memory, with connections to cognitive science, clinical neuropsychology, and cross-linguistic analysis.

Technical level: Intermediate. The paper is written to be readable for cognitive scientists and NLP researchers alike, but it assumes familiarity with word embeddings, cosine distance, and basic statistical modeling (mixed-effects models).

Scope in one sentence: The paper proposes and evaluates a framework that treats a participant's stream of generated concepts as a trajectory through a transformer embedding space, then extracts five geometry- and physics-inspired metrics to distinguish clinical groups and concept categories across four datasets and four languages/models.

What This Paper Is About

When people are asked to list properties of a concept or to name as many items as possible from a category, they are effectively navigating a mental "map" of meaning. The authors ask whether that navigation can be captured quantitatively by embedding each successive word a participant produces and tracing the resulting path through the vector space of a language model.

The goal is to replace labor-intensive manual linguistic coding (such as hand-labeling clustering and switching in verbal fluency data) with an automated pipeline that measures the fine-grained, step-by-step dynamics of semantic search. The framework is then tested on datasets that are deliberately hard for standard embeddings: a clinical dataset (Parkinson's disease and bvFTD patients), a swear-word fluency dataset, and parallel property listing datasets in Italian and German.

Key Contributions

1. A trajectory-based framework for semantic navigation. The paper represents concept production as a time-indexed path through embedding space, constructed with cumulative embeddings, where each point encodes the full prefix of items generated so far rather than an isolated word.

2. Five complementary metrics of navigation. The authors introduce and combine Distance to Next, Entropy, Velocity, Acceleration, and Distance to Centroid, capturing local kinematics, global variability, and global dispersion of the search process.

3. Validation across four datasets, four languages, and four embedding backends. The framework is evaluated on the Neurodegenerative (Chilean Spanish), Swear Fluency (U.S. English), Italian, and German datasets, using OpenAI text-embedding-3-large, Google text-embedding-004, Qwen3-Embedding-0.6B, and a non-cumulative fastText baseline.

4. Evidence of cross-model convergence and a cumulative vs. non-cumulative trade-off. Different embedding models produced highly similar trajectory metrics, and the paper shows that cumulative embeddings help for long trajectories while non-cumulative embeddings are better for short ones.

Main Findings

• Neurodegenerative dataset (76 Chilean Spanish-speaking participants, three groups: PD, bvFTD, HC). Significant effects of category appeared across all metrics. Healthy controls showed lower Distance to Next, Velocity, Acceleration, and Entropy than both patient groups, while bvFTD and PD did not differ from each other. Distance to Centroid showed the opposite pattern: controls were farther from the centroid than patients.

• Swear Fluency dataset (274 undergraduate native speakers of U.S. English). Letter categories and swear words produced larger Distance to Next than animals, with swear words longest and animals shortest, and letters in between. Velocity and Acceleration mirrored this ordering. Entropy was lowest for animals, higher for letters, and highest for swear words. Distance to Centroid reversed the pattern: animals were farthest from the centroid, letters intermediate, swear words markedly closest.

• Italian dataset (69 participants, 10 concept categories, Bird as reference). Most categories showed shorter Distance to Next than Bird, with Building and Vehicle least separated. Velocity and Acceleration followed the same ordering. Entropy differences were selective. Distance to Centroid was partially reversed, with some categories farther from Bird and others (Body Part, Clothing, Implement) closer.

• German dataset (73 participants, 10 concept categories, Bird as reference). Most categories produced shorter Distance to Next than Bird, with Vehicle among the most distinct and Vegetable showing little to no separation. Bird exceeded nearly all categories in Velocity, with the largest gap against Vehicle, and Acceleration followed the same pattern. Implement had higher Entropy than Bird. For Distance to Centroid, Fruit, Mammal, and Vegetable were farther from Bird, while Body Part, Building, Clothing, Furniture, Implement, and Vehicle were closer, with the most pronounced gaps involving Fruit compared to clothing and tool-like categories.

• Cross-model robustness. Pearson correlation matrices across all four datasets showed a block-diagonal structure, with strong positive correlations for each metric across OpenAI text-embedding-3-large, Google text-embedding-004, and Qwen3-Embedding-0.6B. Entropy showed near-perfect inter-model correlation because it depends on rank ordering; Distance to Centroid showed the weakest inter-model correlation, attributed to model-specific embedding geometry, anisotropy, and rogue dimensions.

• Cumulative vs. non-cumulative embeddings. On the Neurodegenerative dataset, all models benefited from cumulative representations (for example, OpenAI 003: 10 significant pairs, mean Cohen's d = 0.30 cumulative vs. 4 pairs, d = 0.19 non-cumulative). The Swear Fluency dataset showed a milder version of this effect, with non-cumulative variants sometimes giving larger effect sizes. The Italian and German datasets favored non-cumulative variants (German Google 004 non-cumulative reached 137 pairs and d = 0.37), consistent with their much shorter trajectories (4.96 and 5.49 mean properties vs. 19.53 and 20.69).

• Anisotropy correction. After ZCA-whitening, metrics for the Parkinson's and Swear Fluency datasets were nearly identical to the original, while Italian and German showed slight, non-directional fluctuations, likely due to their shorter trajectories.

Methodology in Plain English

The procedure works in four steps.

First, each participant's response stream is treated as an ordered list of words. Rather than embedding each word on its own, the authors embed the growing prefix of that list, so the representation at step t carries the memory of everything produced before it.

Second, a text encoder maps each cumulative string to a dense vector, producing a point-by-point path through a high-dimensional space.

Third, five metrics are computed on each path:

• Distance to Next is the cosine distance between consecutive points, giving an "N − 1" series of semantic jumps plus an average step size.
• Entropy binarizes those step sizes at the trajectory's median and computes normalized Shannon entropy, interpreted as how unpredictable the fluctuation pattern is.
• Velocity is the vector difference between consecutive embeddings, and Acceleration is the difference between successive velocity vectors, both assuming a unit time step.
• Distance to Centroid measures how far each item sits from the average position of all unique property embeddings for that participant and concept, indexing overall dispersion.

Fourth, each metric is analyzed with generalized linear mixed models (GLMMs) using the glmmTMB package in R version 4.3.1, with participants and concept as random factors, a lognormal distribution for Distance to Next, Entropy, Velocity, and Acceleration, and a Gaussian distribution for Distance to Centroid. Post-hoc pairwise comparisons use Tukey's HSD.

To check that results are not tied to one encoder, the pipeline is repeated with several embedding models and with a ZCA-whitening step to approximate isotropic conditions.

Why This Matters

Impact on research. The framework offers an automated, scalable alternative to manual linguistic annotation pipelines, which the paper identifies as labor-intensive and hard to compare across studies. It also bridges cognitive modeling of semantic foraging (clustering and switching) with learned representations from modern language models, giving a finer-grained, continuous view of the search process than binary cluster/switch counts. Because the same metrics work across four languages and multiple encoders, it provides a common measurement scaffold for cross-linguistic and clinical comparison.

Real-world applications:

• Clinical stratification of neurodegenerative conditions, such as separating Parkinson's disease and bvFTD patients from healthy controls based on the geometric signature of their verbal output.
• Cross-linguistic comparison of how semantic categories are organized, demonstrated on Italian and German property listing data.
• Behavioral research on taboo language, since swear-word fluency has been linked to substance use and to differential brain activity patterns in schizophrenia.
• Benchmarking the semantic navigation of generative language models against human participants, which the authors name as a target application.

Industry relevance. Any organization that needs to extract signal from free-form verbal or written responses without costly expert annotation could benefit: clinical trial screening, digital health and speech-based diagnostics, educational assessment, and evaluation of language model behavior in conversational agents. The availability of an open codebase (linked in the paper) lowers the barrier to adoption.

Future Directions

• Adding temporal information. The datasets used here did not include time stamps for individual words; the authors note that richer speech-based protocols with timing could yield more meaningful temporal dynamics.

• Moving beyond Euclidean assumptions. The paper acknowledges that treating embeddings as Euclidean overlooks their anisotropic structure. More mathematically robust, potentially non-Euclidean metrics are proposed as a next step, and the authors note that further work is needed to determine when explicit anisotropy correction is beneficial.

• Broadening the information-theoretic toolkit. The current work relies only on a basic Shannon entropy measure; the authors call for more advanced measures suited to systems with many interacting variables.

• Applying the framework to language models. A stated goal is to characterize how different LLMs navigate semantic space and to develop a unified account of trajectories that covers both humans and generative language models.

Target Audience

This paper is most useful for researchers working at the intersection of cognitive science and NLP, particularly those studying semantic memory, verbal fluency, and property listing tasks. It will also interest clinical researchers looking for automated markers of neurodegenerative disease from language, psycholinguists working on cross-linguistic concept representation, and NLP practitioners who want a concrete, reproducible pipeline for turning free-form word streams into quantified trajectories. Graduate students and advanced undergraduates with some background in embeddings and statistics will find the methods section accessible, while the applications sections are written to be useful for a broader cognitive-science readership.

Authors’ abstract

Semantic representations can be framed as a structured, dynamic knowledge space through which humans navigate to retrieve and manipulate meaning. To investigate how humans traverse this geometry, we introduce a framework that represents concept production as navigation through embedding space. Using different transformer text embedding models, we construct participant-specific semantic trajectories based on cumulative embeddings and extract geometric and dynamical metrics, including distance to next, distance to centroid, entropy, velocity, and acceleration. These measures capture both scalar and directional aspects of semantic navigation, providing a computationally grounded view of semantic representation search as movement in a geometric space. We evaluate the framework on four datasets across different languages, spanning different property generation tasks: Neurodegenerative, Swear verbal fluency, Property listing task in Italian, and in German. Across these contexts, our approach distinguishes between clinical groups and concept types, offering a mathematical framework that requires minimal human intervention compared to typical labor-intensive linguistic pre-processing methods. Comparison with a non-cumulative approach reveals that cumulative embeddings work best for longer trajectories, whereas shorter ones may provide too little context, favoring the non-cumulative alternative. Critically, different embedding models yielded similar results, highlighting similarities between different learned representations despite different training pipelines. By framing semantic navigation as a structured trajectory through embedding space, bridging cognitive modeling with learned representation, thereby establishing a pipeline for quantifying semantic representation dynamics with applications in clinical research, cross-linguistic analysis, and the assessment of artificial cognition.