Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions

AI summary

Overview

Research area: Natural Language Processing — interpretability and fairness of large language models, drawing on social psychology (person perception, the Stereotype Content Model) and sociolinguistics (language attitudes, African American Language).

Technical level: Intermediate. The paper assumes familiarity with linear probing of hidden states, open-weight LLM families, and regression modeling, but its central framing (impressions along warmth and competence) is explained from first principles.

Scope: A one-sentence summary — the paper introduces "artificial impressions," shows they are more reliably decodable from LLM hidden states than reportable via prompting, and demonstrates that these impressions predict response quality and hedging and vary with dialect.

What This Paper Is About

People form rapid impressions of others and hold stereotypes about groups, and those impressions shape behavior. Because LLMs are trained on text written by many authors, the authors hypothesize that models encode analogous associations between linguistic patterns and traits. The goal is to measure those "artificial impressions" of prompt authors using the two-dimensional Stereotype Content Model (SCM) — warmth (perceived intent) and competence (capacity to act) — and to test whether they relate to model behavior and to features of the prompt.

Key Contributions

1. Introduces and operationalizes "artificial impressions." The authors define artificial impressions as patterns in LLM internal representations of prompts that resemble human impressions and stereotypes based on language, and measure them along the SCM's warmth and competence dimensions. They frame evaluation around two psychometric concepts: reliability and validity.

2. Builds a synthetic impression dataset at scale. Using 131 warmth traits and 104 competence traits, they form impression specifications, then sample 10 generations per specification at temperature 0.9, yielding 274,830 prompts per model for three open-weight LLMs (Llama-3.1 8B, Llama-3.2 1B, OLMo-2 7B).

3. Develops linear impression probes over hidden states. Instead of asking models to self-report impressions, the authors train separate warmth and competence probes on MLP activations from each layer, evaluated with 5-fold cross validation (F1 and accuracy with 95% confidence intervals) against bag-of-words baselines.

4. Links impressions to behavior and prompt features. They connect probe outputs to response quality and hedging in real conversations, and to content, style, and language-variety (AAL vs. WME) features of prompts.

Main Findings

• Prompted self-reports are unreliable (Finding 1). Self-consistency between provided traits and reported impressions is near random in the 1st-person setting for all models (warmth 47.67, 51.67, 47.68; competence 51.82, 51.85, 56.65 for Llama-3.2 1B, Llama-3.1 8B, OLMo-2 7B respectively), driven by a strong tendency to report positive traits (warm, competent) over negative ones. Third-person performance improves but remains low: Llama-3.1 (8B) reaches 80.77 for warmth and 65.06 for competence, while Llama-3.2 (1B) reaches 61.54 and 59.89, and OLMo-2 (7B) 74.01 and 61.28. An analysis of non-instruction-tuned models showed low self-consistency but not necessarily the same positive bias, suggesting post-training may draw out this behavior.

• Human annotators validate the trait dictionary (Finding 2). Across 81 low/high warmth pairs and 81 low/high competence pairs per model, four annotators agreed with the original trait specifications at Cohen's κ = 0.68 and Spearman r = 0.68 overall, close to annotator-to-annotator agreement of r = 0.76 for average ratings. Inter-rater reliability was Krippendorff's α = .71 on raw ratings and α = .78 on binary message choices.

• Impressions are linearly decodable (Finding 3). Probe F1 scores exceed the bag-of-words baseline at most model depths and at every training data percentage tested. Peak F1 falls between 75–90 for warmth probes and 75–85 for competence probes; accuracies were 80–90% for warmth and 75–85% for competence, exceeding the third-person self-consistency scores. Performance rises quickly, peaks around or before the midpoint of the model, then declines slowly.

• Warmth is more clearly encoded than competence (Finding 4). Warmth probes outperform competence probes across models and layers, warmth bag-of-words baselines outperform competence ones by nearly 20% F1, and self-reports are also more consistent for warmth. The authors relate this to the warmth-primacy effect observed in human impressions.

• Impressions predict response quality (Finding 5). In an ordered logistic regression on quality scores rated by Llama-3.1 (405B) for 2,000 real prompts, warmth probe probability had coefficients of 1.07, 0.49, and 0.76 for Llama-3.2 (1B), Llama-3.1 (8B), and OLMo-2 (7B) respectively, and competence 0.90, 0.39, and 0.35, all statistically significant.

• Low competence predicts hedging (Finding 6). In a negative binomial regression on hedge-term counts, competence coefficients were −1.06, −1.18, and −0.69, significant for all three models. Warmth results were mixed, significant for only Llama-3.2 (1B) (−0.46) and OLMo-2 (7B) (0.40). Prompt and response length effects were near zero.

• Content and style track impressions (Finding 7). Using LIWC with log-odds ratio and an informative Dirichlet prior on Llama-3.1 (8B) prompts, high-warmth prompts are associated with categories such as Affiliation (z = 163.44) and Drives (141.16), and qualitatively with tentative and discrepancy terms indicating politeness; low-warmth prompts with Negate (−132.65) and Adverb (−107.13). High-competence prompts are associated with Preposition (122.74) and Adjective (111.76); low-competence prompts with Adverb (−114.54), Informal (−96.78), and Netspeak (−74.09).

• AAL prompts receive more negative impressions (Finding 8). For Llama-3.1 (8B), tweets from the TwitterAAE corpus generally score low on both dimensions, and AAL tweets score significantly lower than WME tweets. The posterior probability of AAL correlated negatively with warmth (r = −0.32, p ≤ 0.001) and competence (r = −0.52, p ≤ 0.001). On the parallel counterparts dataset, AAL texts were predicted significantly less competent (t = −24.78, p ≤ 0.001) and, to a lesser extent, less warm (t = −3.89, p ≤ 0.001).

Methodology in Plain English

Rather than asking a model "what do you think of me?" — which the authors show is unreliable — the researchers look inside the model.

First, they take a published list of trait terms that vary in warmth and competence, combine them into impression specifications such as "friendly and meticulous," and ask each model to generate sample user messages expressing those traits. This produces a large set of prompts with known trait labels (274,830 per model) and captures the fact that different perceivers can form different impressions of the same target.

Second, they feed those prompts back into the model and record the internal activations at every layer ("hidden representations"). They train simple linear probes to map those activations onto the two SCM dimensions, one probe for warmth and one for competence, using 5-fold cross validation with bag-of-words classifiers as a difficulty baseline.

Third, they check the probes against human judgment: annotators compared pairs of generated messages differing only in one dimension and rated which was warmer or more competent, using a 4-point scale and instructed to judge how the messages would be viewed by others to reduce social desirability bias.

Fourth, they test the probes on real data — the first user prompt from 2,000 randomly sampled English, non-code conversations from LMSysChat (a corpus of 1 million LLM-user conversations) — and fit ordered logistic regression for quality (rated 1 to 9 by Llama-3.1 405B) and negative binomial regression for hedge-term counts based on terms from Vincze (2013).

Finally, they test whether prompt features predict impressions, using LIWC category analysis with log-odds ratios, and comparing probe outputs on 400 stratified tweets from the TwitterAAE corpus plus the parallel AAL/WME counterparts dataset.

Why This Matters

Impact on research. The paper offers an alternative to prompting-based evaluation of model attitudes: probes on hidden states were more reliable than self-reports, suggesting that introspection-based measurement of model "opinions" may systematically understate or distort what models encode. It also connects NLP interpretability work to a long-standing psychological framework (SCM), and it links probe outputs to concrete behavioral outcomes — quality and hedging — rather than only to classification accuracy.

Real-world applications (implied by the paper's findings and discussion):

• Bias auditing of deployment systems. The negative AAL findings give a measurable audit signal: models associate lower competence, and to a lesser extent lower warmth, with AAL text, which the authors note risks representational and quality-of-service harms for speakers of historically marginalized varieties.
• Response-quality monitoring. Because warmth and competence predictions significantly predict quality ratings, probes could act as an early-warning indicator for prompts that receive systematically worse responses.
• Chatbot evaluation and benchmarking. The self-consistency results caution against building evaluation suites on models' own reports of their impressions and point toward representation-level measurement instead.
• Personalization decisions. The authors note that different users have different needs and that adapting behavior (for example, to a user's level of knowledge in educational settings) may sometimes be desirable — making it important to distinguish legitimate adaptation from stereotype-driven behavior.

Industry relevance. Any product where a model responds to user text — assistants, support bots, educational tools — is affected by the paper's claim that prompt style and dialect influence downstream quality and hedging. The authors warn that such behaviors "can pose allocational harms (e.g., lower quality LLM responses) to users from historically marginalized groups … and must be avoided."

Future Directions

• Move beyond the first turn. The paper restricts itself to the initial messages of English conversations; the authors note that person perception research documents how impressions change over the course of one or many interactions, and they leave this to future work.
• Use finer-grained or alternative stereotype models. The SCM has been subdivided into morality and sociability (for warmth) and agency and ability (for competence), and the Power-Benevolence framework captures more complex, group-dependent stereotypes; the authors call exploring these "promising directions."
• Trace where these impressions come from. It remains unknown what factors in pretraining data or procedures produce the observed phenomena, and the authors note training processes may draw out the positive-report bias seen in instruction-tuned models.
• Cross-cultural and broader behavioral coverage. The authors flag that impression formation differs among countries and cultures, and that they could not exhaustively study all prompt factors or behaviors related to artificial impressions.

Target Audience

Researchers in NLP interpretability, fairness, and bias evaluation; computational social scientists and psychologists studying language attitudes and stereotype content; and practitioners who build or audit LLM applications and need to know how prompt wording, register, and dialect may change model behavior. Readers unfamiliar with probing techniques will still follow the argument, but the paper is most immediately useful to those already working with hidden-state analysis or bias measurement.

Authors’ abstract

We introduce and study artificial impressions--patterns in LLMs' internal representations of prompts that resemble human impressions and stereotypes based on language. We fit linear probes on generated prompts to predict impressions according to the two-dimensional Stereotype Content Model (SCM). Using these probes, we study the relationship between impressions and downstream model behavior as well as prompt features that may inform such impressions. We find that LLMs inconsistently report impressions when prompted, but also that impressions are more consistently linearly decodable from their hidden representations. Additionally, we show that artificial impressions of prompts are predictive of the quality and use of hedging in model responses. We also investigate how particular content, stylistic, and dialectal features in prompts impact LLM impressions.