LLM-Guided Exemplar Selection for Few-Shot Wearable-Sensor Human Activity Recognition

AI summary

Overview

Research area: Few-shot Human Activity Recognition (HAR) using wearable inertial sensors, combined with Large Language Model (LLM) reasoning for data-efficient training.

Technical level: Intermediate. The paper assumes familiarity with standard machine-learning baselines (k-NN, Logistic Regression, Random Forest, SVM, Naïve Bayes), feature engineering, and evaluation by macro F1, and it also uses graph-centrality and submodular-optimization terminology. The LLM component is described at the prompting level rather than the model-internals level.

Scope (one sentence): The paper tests whether structured semantic knowledge generated by an LLM can be combined with geometric and graph-based cues to pick a small, representative set of training exemplars for few-shot wearable-sensor HAR on the UCI-HAR dataset.

What This Paper Is About

Human Activity Recognition models usually need large labeled sensor datasets, and when labels are scarce the field typically selects a small subset of training samples using purely geometric criteria such as random sampling, herding, or k-center. Those criteria cannot tell apart activities whose accelerometer and gyroscope patterns overlap, such as walking, walking upstairs, and walking downstairs.

The paper proposes an LLM-Guided Exemplar Selection framework that adds semantic reasoning to that selection step: an LLM produces feature weights per activity, an inter-class confusability matrix, and per-class exemplar budget multipliers, which are then combined with validation margins, PageRank centrality, a hubness penalty, and facility-location optimization to choose a compact exemplar set.

Key Contributions

1. A method for deriving LLM-based semantic features from numerical sensor data, expressed as linear combinations of standardized base channels and intended to capture activity-relevant dimensions beyond statistical descriptors (an example given is a "verticality" axis defined as a weighted sum of z-axis acceleration statistics).
2. A mechanism that integrates structured LLM knowledge — label–feature weights, inter-class confusability, and exemplar budget multipliers — into a single semantic scoring formulation.
3. A hybrid exemplar selection algorithm that combines the validation margin, class-conditional PageRank centrality, a hubness penalty, and the LLM semantic prior inside a greedy facility-location framework.
4. An evaluation on UCI-HAR under subject-disjoint few-shot conditions, plus an ablation study isolating the semantic prior, graph structure, hubness penalty, and facility location.

Main Findings

• Best per-model score: Under the reported table, Logistic Regression reaches a macro F1 of 88.79% with the LLM-Guided method, the highest value in the comparison. The abstract states 88.78% as the framework's macro F1, which differs slightly from the 88.79% reported in the results table and conclusion.
• Mean across models: The LLM-Guided approach has a mean macro F1 of 67.02%, versus 62.72% for K-Center, 61.49% for Herding, and 56.66% for Random Sampling, a +10.36 point gain over Random Sampling.
• Largest per-model gains: kNN improves by +27.24 points (85.88 vs. 58.64 for Random Sampling) and Logistic Regression by +19.99 points (88.79 vs. 68.79).
• Gains are not uniform across classifiers: HistGradient Boosting reports 16.99% for Random Sampling, Herding, K-Center, and the LLM-Guided method alike (+0.00), and Linear SVC reports 72.85% for the proposed method versus 74.45% for Random Sampling (−1.60), a decrease. Gaussian Naïve Bayes improves by +3.39 points to 58.03%. This conflicts with the paper's statements that the method "consistently outperformed all baselines."
• Facility location is the most important component: Removing the Facility-Location greedy step drops the mean macro F1 from 67.02% to 47.34% (−19.68 points).
• The LLM prior contributes measurably: Removing the semantic prior (LLM knowledge) reduces the mean macro F1 to 64.97% (−2.05 points).
• Graph and hubness components also contribute: Removing class-conditional PageRank gives 65.44% (−1.58), and removing the hubness penalty gives 65.70% (−1.32).
• Runtime is reported as unchanged: The paper states, citing its Appendix G, that inference runtime is comparable to baseline exemplar-selection methods, so the LLM-guided selection adds no additional latency, though specific timing values are not given in the main text.
• Data split: The subject-disjoint split uses 6,012 training samples from 17 subjects, 1,340 validation samples from 4 subjects, and 2,947 test samples from 9 subjects.

Methodology in Plain English

The pipeline has three stages: exemplar selection, training, and inference.

First, raw UCI-HAR inertial signals are windowed into 2.56-second segments (128 samples) with 50% overlap, and basic features are computed — the mean and standard deviation of six channels (acc_x, acc_y, acc_z, gyr_x, gyr_y, gyr_z). Additional feature sets are built: six PCA components, Spectral PCA on log-amplitude rFFT spectra projected until 95% of variance is retained, and LLM-generated semantic features. All features are z-score standardized.

An LLM (GPT-4o-mini, temperature 0, up to 1200 max tokens, accessed via LangChain) is prompted with training-set class-wise statistics only, and returns a JSON knowledge object with three parts: per-class feature weights, an inter-class confusability matrix, and per-class exemplar budget multipliers (around 1.2× for dynamic activities such as walking and walking upstairs, around 0.8× for static ones such as laying and sitting). A separate prompt asks the LLM to design six semantic features as linear combinations of the base channels with weights in [−2, 2].

Each candidate exemplar then receives a hybrid score combining four terms: a validation-margin term (cosine similarity to same-class versus different-class validation samples), a class-conditional PageRank centrality term computed on a mutual-kNN graph with cosine edge weights, a hubness penalty counting how often a sample appears in others' neighbor sets, and the LLM semantic score. The weights are set to α = 1.0, μ = 0.10, τ = 0.10, and β = 0.15. The semantic score for a sample subtracts the maximum weighted score of any other class scaled by confusability, so that a sample scores highly if it is representative of its own class and distinct from confusable classes.

Finally, exemplars are chosen greedily to maximize a facility-location objective — the sum over all samples of the maximum cosine similarity to any selected exemplar — until the per-class budget is reached (for example, 8 for dynamic classes and 2 for static classes). Other reported parameters include k_graph = 10, λ_red = 0.25, α_prior = 0.20, μ_cent = 0.10, τ_hub = 0.10, and seed = 42.

In the training stage, a lightweight Machine Learning Gate first separates static activities (laying, sitting, standing) from dynamic ones (walking, walking upstairs, walking downstairs) so each classifier is trained only on its relevant domain. Six classical models are trained on the selected exemplars: k-NN, Logistic Regression, Random Forest, Histogram Gradient Boosting, Linear SVC, and Gaussian Naïve Bayes. At inference, the gate restricts the candidate label set before the appropriate classifiers predict, and the outputs are merged into a six-class result. The authors state that no validation or test information is used during gating or LLM prompting, to prevent leakage.

Why This Matters

Impact on research. The paper argues that exemplar selection for few-shot HAR has been almost entirely geometric, and that LLM-derived semantic priors can act as a knowledge layer that describes why activities differ, not just how close their feature vectors are. It also positions LLMs as active components in a sensor-learning pipeline — shaping the feature space, setting class priors, and influencing which samples are selected — rather than as passive scorers, which the authors present as an advance over earlier work such as HED-LM.

Real-world applications (as listed in the paper's motivation):

• Healthcare monitoring
• Physical rehabilitation
• Elderly care and monitoring
• Human–computer interaction

Industry relevance. Because the method relies only on basic statistical features and class-wise summaries, the authors state it can be applied to other wearable-sensor datasets, naming WISDM, PAMAP2, and RealWorld HAR as examples, and that LLM priors are regenerated from each dataset's own training data so the approach can adapt to different activity sets or sensor configurations. The paper also reports that inference runtime stays comparable to baseline selection methods, which it presents as suitable for real-time and wearable-sensor environments, while noting that initial LLM generation still requires access to an external API.

Future Directions

• Evaluate the framework on larger or noisier datasets, since the current study is limited to a single dataset and the quality of LLM priors may vary across datasets.
• Replace the current statistical and PCA-based features with richer time-series features, which the authors expect may further improve results.
• Explore multimodal fusion to extend the approach beyond inertial signals.
• Develop lightweight or local LLM implementations for edge devices, to improve adaptability, efficiency, and real-time applicability.
• Broader open question: the paper does not report a per-component runtime breakdown or cost analysis of LLM calls, so the practical overhead of generating priors at scale is not quantified in the main text.

Target Audience

Researchers and practitioners working on wearable-sensor activity recognition with limited labels, especially those interested in few-shot learning, exemplar or instance selection, and knowledge-guided machine learning. It is also relevant to applied engineers building data-efficient monitoring systems for healthcare, rehabilitation, or elderly care, and to readers tracking how LLMs are being integrated into non-text sensing pipelines.

Authors’ abstract

In this paper, we propose an LLM-Guided Exemplar Selection framework to address a key limitation in state-of-the-art Human Activity Recognition (HAR) methods: their reliance on large labeled datasets and purely geometric exemplar selection, which often fail to distinguish similar wearable sensor activities such as walking, walking upstairs, and walking downstairs. Our method incorporates semantic reasoning via an LLM-generated knowledge prior that captures feature importance, inter-class confusability, and exemplar budget multipliers, and uses it to guide exemplar scoring and selection. These priors are combined with margin-based validation cues, PageRank centrality, hubness penalization, and facility-location optimization to obtain a compact and informative set of exemplars. Evaluated on the UCI-HAR dataset under strict few-shot conditions, the framework achieves a macro F1-score of 88.78%, outperforming classical approaches such as random sampling, herding, and k-center. The results show that LLM-derived semantic priors, when integrated with structural and geometric cues, provide a stronger foundation for selecting representative sensor exemplars in few-shot wearable-sensor HAR.