Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations

AI summary

Overview

Research area: Natural Language Processing / computational semantics, at the intersection of distributional semantics and embodied cognition.

Technical level: Intermediate. The paper assumes familiarity with word embeddings (Word2Vec, GloVe, BERT), regression-style projection models, and correlation analysis, but its experimental logic is explained in accessible terms.

Scope: One sentence: the paper introduces SENSE, a projection model that predicts Lancaster sensorimotor norms from lexical embeddings, and validates it with a behavioral study of 281 participants on nonce words and a sublexical phonestheme analysis.

What This Paper Is About

Word embeddings learn meaning from co-occurrence patterns, while human language understanding is grounded in sensory and motor experience. This paper asks how much human-annotated sensorimotor information is already implicit in embeddings such as Word2Vec, GloVe, and BERT, and whether a learned projection can recover it. The authors build SENSE (Sensorimotor Embedding Norm Scoring Engine) to map word embeddings onto the 11 Lancaster sensorimotor dimensions, then test whether its predictions align with human judgments of made-up words and with character-level form-meaning patterns.

Key Contributions

1. SENSE, a learned projection model that maps lexical embedding vectors (e_w in R^d) to 11-dimensional sensorimotor norm predictions (f(e_w) in R^11), normalized to the [0, 1] range from the original [0, 5] Lancaster scale.
2. A systematic comparison of projection architectures and embedding types (baseline mean predictor, k-NN with k=5, and a feed-forward neural network) across Word2Vec, GloVe, and BERT CLS embeddings, evaluated with MSE on a held-out test set.
3. A behavioral study with 281 undergraduate participants using Wuggy-generated nonce words to test whether SENSE predictions generalize to novel, unfamiliar words (Hypothesis 2).
4. A sublexical analysis of character n-grams (length 2-4) that found systematic phonesthemic patterns for the interoceptive modality, suggesting a path toward computationally proposing candidate phonesthemes from text.

Main Findings

• Embeddings encode sensorimotor information (H1). Paired t-tests comparing per-word MSE between SENSE and the baseline revealed significantly lower errors for SENSE across all 11 modalities and overall (p < .001).

• Neural network and k-NN beat the baseline. Overall MSE averaged across the 11 modalities was 0.028 for the baseline across all three embeddings; k-NN scored 0.015 (Word2Vec), 0.018 (GloVe), and 0.020 (BERT CLS); the neural network scored 0.016 (Word2Vec), 0.017 (GloVe), and 0.016 (BERT CLS).

• Modality-specific error patterns. The lowest errors were observed for the gustatory and olfactory modalities across all three embeddings, while visual and auditory dimensions showed the highest errors (reported qualitatively in Figure 1).

• Human study supports generalization to nonce words (H2). SENSE predictions significantly correlated with human selection rates for 6 of the 11 modalities: interoceptive (r = 0.73, p < .001), auditory (r = 0.69, p < .001), torso (r = 0.57, p < .01), visual (r = 0.56, p < .01), gustatory (r = 0.54, p < .01), and hand/arm (r = 0.43, p < .05).

• Five modalities showed non-significant correlations: foot/leg (r = 0.25, p = .192), olfactory (r = 0.18, p = .349), haptic (r = 0.16, p = .408), head (r = 0.10, p = .618), and mouth (r = -0.14, p = .474).

• Partial support for character-level phonesthemes (H3). Interoception showed a significant positive correlation between the human selection rate P_H(w* → m | c ∈ w*) and SENSE's prediction f(e_c)_m (r = 0.630, p = 0.007); auditory showed no significant relationship (r = 0.267, p = 0.562).

• Chance level in the task. The forced-choice design used 3 target words out of 7 options (3/7), yielding a chance-level true positive rate of 42.9%.

Methodology in Plain English

The authors started with the Lancaster sensorimotor norms, which aggregate ratings from 3,500 participants for 39,707 English words across six perceptual modalities (auditory, gustatory, visual, haptic, olfactory, interoceptive) and five action effectors (hand/arm, foot/leg, head, torso, mouth/throat). They kept words and phrases present in Word2Vec, GloVe, and Lancaster vocabularies (averaging constituent vectors for multi-word phrases), leaving 34,110 aligned entries, and split them 70/15/15 into train, development, and test sets. For BERT, they used the CLS representation of each word or phrase passed through the model alone, without sentential context.

They trained three predictors: a baseline that always outputs the mean training sensorimotor vector, a k-NN model (k=5, cosine similarity, weighted averaging), and a one-hidden-layer feed-forward network with 64 or 128 neurons tuned on the development set, ReLU activation, Adam optimization at learning rate 0.001 for 10 epochs, and batch size 128. All were scored by MSE on the held-out test set. The neural network with BERT CLS embeddings was chosen as the SENSE model because it could embed nonce words and arbitrary character sequences.

For the human study, they used the Wuggy Pseudoword Generator, seeded by 63,975 words from Wuggy's lexicon (sourced from the 306,128-word Moby Word List), generating 10 candidates per seed with a 2/3 sub-syllabic segment overlap ratio. They excluded nonce words within Levenshtein distance 1 of, sharing stems with, or homophonous to Moby Dictionary entries. They selected the 12 nonce words with the highest SENSE score per modality (all above 0.5) and ran a Qualtrics survey asking participants to pick which 3 of 7 nonsense words most related to a given modality; each question mixed 3 targets with 4 distractors. Each participant answered two questions per modality, and the survey drew 281 undergraduate respondents.

For the sublexical analysis, they extracted character n-grams of length 2-4 from the 28 nonce words in each of the interoceptive and auditory modalities, keeping only n-grams appearing in at least 3 words, removing redundant substring n-grams and those whose selection rate was at or below the modality's overall rate. This left 13 character sets for interoceptive and 19 for auditory. They embedded each character set, computed SENSE's prediction, and correlated it with the human selection rate via Pearson correlation.

Why This Matters

Impact on research. The work bridges distributional semantics and grounded cognition theory, providing evidence that co-occurrence-based representations already carry sensorimotor information. It also extends the validation of such projections beyond small word sets (prior work evaluated roughly 500 words) and beyond a primary focus on emotional information, and it adds human behavioral validation that earlier studies lacked.

Real-world applications:

• Grounding language models in embodied meaning for robotics or multimodal agents that must connect words to perception and action.
• Lexicography and language pedagogy: automatically proposing sensorimotor associations, or candidate phonesthemes, for new or unfamiliar vocabulary.
• Psycholinguistics and cognitive science: generating testable hypotheses about form-meaning correspondences (phonesthemes) from text data.
• Assistive or educational tools that surface sensory information about words, particularly for the interoceptive dimension relevant to abstract concepts.

Industry relevance. Any system that must align text with physical or sensory context—embodied conversational agents, virtual/augmented reality interfaces, accessibility and sensory-description tools, and search or recommendation systems sensitive to perceptual attributes—could benefit from embedding-level sensorimotor signal that is already present, without new annotation.

Future Directions

• Cross-linguistic validation. The evaluation is restricted to English; the authors state that morphologically rich or logographic languages, where the form-to-meaning relationship differs substantially, may behave differently and warrant native-speaker behavioral studies across typologically diverse languages.
• Explaining the missing modalities. It remains an open question why five modalities (haptic, olfactory, foot/leg, head, mouth/throat) showed non-significant correlations; preliminary analysis found no clear link to training corpus sparsity or inter-subject agreement.
• Addressing sampling and representation limitations. Future work should reduce the WEIRD bias from the 281 participants at a single university and move from orthographic forms to phonological representations, since phonesthemes are traditionally phonological.
• Direct behavioral validation of phonesthemes. The authors note that time constraints prevented a targeted study testing whether model-proposed interoceptive character sets are psychologically real; such a study remains a natural next step.

Target Audience

NLP researchers working on lexical semantics, grounding, and embedding interpretability; cognitive scientists and psycholinguists studying embodied cognition and phonesthemes; and applied researchers who want to inject sensory information into language systems. Readers need some background in word embeddings and statistical evaluation to follow the architecture comparisons and correlation results, but the paper's core ideas are accessible to graduate students entering either field.

Authors’ abstract

While word embeddings derive meaning from co-occurrence patterns, human language understanding is grounded in sensory and motor experience. We present $\text{SENSE}$ $(\textbf{S}\text{ensorimotor }$ $\textbf{E}\text{mbedding }$ $\textbf{N}\text{orm }$ $\textbf{S}\text{coring }$ $\textbf{E}\text{ngine})$, a learned projection model that predicts Lancaster sensorimotor norms from word lexical embeddings. We also conducted a behavioral study where 281 participants selected which among candidate nonce words evoked specific sensorimotor associations, finding statistically significant correlations between human selection rates and $\text{SENSE}$ ratings across 6 of the 11 modalities. Sublexical analysis of these nonce words selection rates revealed systematic phonosthemic patterns for the interoceptive norm, suggesting a path towards computationally proposing candidate phonosthemes from text data.