Large Language Model-Driven Context-Aware Eco-Feedback Generation and Evaluation

AI summary

Overview

• Research area: Human-Computer Interaction (cs.HC), at the intersection of building energy management (BEM), eco-feedback design, and large language model (LLM) prompt/context engineering. The accepted manuscript appears in Energy and Buildings 370 (2026) 118038.
• Technical level: Intermediate. Readers need basic familiarity with LLM prompting concepts (chain-of-thought, self-consistency) and with household energy terms such as time-of-use rates, load shifting, and appliance-level metering.
• Scope (1 sentence): The paper proposes and empirically tests a context-engineering framework in which an LLM (OpenAI GPT 4o) generates household-specific "context-aware eco-feedback" from pre-computed appliance energy metrics, utility rate structures, and synthesized household personas, then evaluates that output against reference interventions and across 400 combinatorial scenarios.

What This Paper Is About

Existing eco-feedback systems—energy bills, smart-meter usage charts, and personalized dashboards—generally rely on household energy use patterns alone and fail to account for household characteristics such as personas, comfort limits, and non-negotiable routines, which makes the advice generic, infeasible, or superficial. The authors' goal is to demonstrate that an LLM-integrated framework, built on principles of contextual engineering, can generate eco-feedback that is simultaneously grounded in the household's energy data and adapted to its lived context. They then test both the accuracy of that feedback (against independently derived reference interventions) and its adaptability (across a large combinatorial scenario space).

Key Contributions

1. A context-aware eco-feedback conceptual framework with four interconnected components—Household Data, Energy Data Analysis, Prompt, and LLM—where a deliberate design principle separates deterministic computation from language-based reasoning so that quantitative energy metrics remain reproducible across runs.
2. A five-metric energy use analysis (total energy and mean power use, usage frequency, usage variability, solar power alignment, solar power coverage) that is computed outside the LLM and passed to it as a structured "household energy analysis report" rather than as raw interval data.
3. An SC-CoT (self-consistency with chain-of-thought) prompt encoding five explicit reasoning steps for eco-feedback generation, with outputs retained only when a strategy appeared at least three times across five independent runs.
4. A multi-dimensional evaluation framework covering accuracy (appliance validation rate, reference alignment rate), data citation (citation frequency, citation accuracy across five data categories), and token efficiency (input tokens, mean output tokens, average output tokens per reference-aligned intervention).

Main Findings

• Reference alignment: The proposed framework generated eco-feedback that aligned with reference interventions at a mean rate of 92.0%.
• Data grounding: Recommendations were grounded in the provided household data with 95.7% citation accuracy, meaning citations were correct when a data category was cited rather than being generic or fabricated.
• Adaptability: The framework "proved highly adaptive," shifting both the appliances targeted and the energy-saving strategies recommended in response to utility rate structure and household context, as examined across 400 scenarios spanning 50 households, two utility rate structures, and four behavioral personas.
• Solar and rate differentiation: The empirical validation included homes with and without solar and with standard versus time-of-use rates, enabling assessment of whether the framework recognized opportunities arising from on-site solar generation; the paper states it did so effectively.
• Reproducibility rationale: Because LLM stochasticity makes repeated runs on identical inputs non-identical—an unacceptable property for energy metrics—all quantitative computations (usage frequency, temporal variability, solar alignment) were delegated to a deterministic module, restricting the LLM to synthesis and reasoning.
• Not reported (in the available content): Per-household appliance validation rates, the token efficiency results (input tokens, mean output tokens, average output tokens per reference-aligned intervention), and confidence intervals or statistical significance tests are not shown in the provided text; the equations for these metrics are presented but their numeric outcomes are not.

Methodology in Plain English

• Data: Three actual appliance-level home energy datasets from Austin, Texas (selected from a 50-household pool purchased from Pecan Street Inc.) were used for validation. Selection was based on two criteria: the number of appliances (which increases framework complexity) and whether the home had solar panels. Power was recorded at 15-minute intervals in kilowatts, with timestamps, household identifiers, and per-appliance readings; the solar home also had household-level photovoltaic generation at the same resolution. Data covered the summer months of June through August.
• Household compositions: Home #1 (7 appliances, no solar, standard rate, temperature-sensitive occupants), Home #2 (7 appliances, no solar, time-of-use rate, occupants prioritizing environmental sustainability), Home #3 (14 appliances, solar included, standard rate, occupants who always have dinner together at 6 pm on weekdays).
• Utility rates: A standard rate of 12.10 cents per kWh applying uniformly, and a TOU rate charging 26.10 cents per kWh from 2 pm to 8 pm and 9.31 cents during all other hours.
• Personas: Constraints (thermal comfort, environmental sustainability, non-negotiable routines) were not directly observed but were incorporated as plausible, evidence-informed assumptions grounded in prior studies.
• Metric computation: Raw time series were never passed to the model. A deterministic Energy Data Analysis component computed five metrics using fixed algorithms—mean power as a double average over hours and days, usage frequency via an indicator function against an appliance-specific power threshold, usage variability via coefficient of variation of normalized hourly use frequency, solar power alignment via the fraction of active intervals coinciding with generation above a threshold, and solar power coverage via proportional apportionment of solar generation relative to instantaneous demand.
• Reference interventions: A five-step reasoning process (home type assessment, appliance inventory and energy assessment, load curtailment and shifting evaluation, behavioral intervention design with context, eco-feedback finalization) produced the reference interventions independently. Appliances with high use were flagged at a threshold of 0.5% or more of total energy usage. These steps were reviewed and validated through discussion with an external expert for third-party adjudication.
• Generation: The same reasoning steps were restructured into an explicit SC-CoT prompt, and OpenAI GPT 4o was used as the generation engine. Eco-feedback was generated 50 times per household.
• Adaptability testing: A combinatorial scenario-based analysis swept 400 scenarios across 50 households, two rate structures, and four personas to test how the framework's appliance targeting and strategy recommendations respond to context.

Why This Matters

• Research impact: The paper argues that personalized eco-feedback has been limited by reliance on predefined features and usage patterns alone. By framing contextual engineering as the methodological bridge between raw household data and adaptive feedback, it positions LLMs as context-aware generators rather than generic advisors, and identifies two previously unaddressed requirements: embedding household-specific information (appliance-level profiles, structured behavioral personas) into generation, and evaluating output quality across multiple households against defined behavioral benchmarks.
• Real-world applications:
  • Utility customer engagement programs that need to explain TOU rate implications in household-specific terms.
  • Smart home and smart meter platforms that already have interval data but produce only charts or pre-framed text.
  • Solar-equipped households where advice must account for when generation is available, not just how much energy is used.
  • Households with non-negotiable routines (for example, fixed dinner times) where generic curtailment advice is infeasible.
• Industry relevance: Token usage in API-based LLM deployments correlates directly with computational cost and response latency, which is why the paper treats token efficiency as a practical consideration. The separation of deterministic analysis from LLM reasoning also matters operationally: it reduces tokens passed to the model and ensures reproducibility across evaluation runs, a property energy metrics require. The paper also notes load disaggregation methods such as non-intrusive load monitoring could extend the framework to homes that only have aggregated data.

Future Directions

1. Validation with real households: The three personas and constraints were synthesized from literature rather than observed; testing with real occupant personas and non-negotiable routines is the obvious next step.
2. Scaling beyond the current evidence base: The analysis covers 50 households in Austin over summer months (June–August) with two rate structures and four personas. Extending to more households, other climates, other seasons, and additional rate structures remains open.
3. Disaggregation for aggregated-data homes: The framework supports load disaggregation as an option, but the paper does not report results from applying it, leaving the aggregated-data pathway untested here.
4. Multi-LLM and verification architectures: The framework allows a single or multiple LLMs with distinct roles (generation, verification, constraint checking), but only OpenAI GPT 4o was used as the engine, so multi-model orchestration effects are not reported.

Target Audience

Researchers and practitioners in human-building interaction, building energy management, and occupant-facing interfaces; utility program designers exploring individualized energy advice; and HCI or applied-LLM researchers interested in context engineering and evaluation methodologies for domain-specific generation tasks. The paper is most useful to readers who want a template for combining deterministic domain analytics with LLM reasoning, plus a concrete evaluation scheme (reference alignment, data citation, token efficiency) that others could adopt or critique.

Authors’ abstract

The objective of this study is to demonstrate the potential of generating context-aware eco-feedback - eco-feedback that reflects a household's contextual characteristics alongside its energy use patterns - through a large language model-integrated framework. Previous studies have introduced personalized eco-feedback, mostly relying on household energy use patterns; however, they frequently did not reflect distinct household characteristics, including their persona or non-negotiable routines, leaving eco-feedback ineffective and sometimes superficial. To address these limitations, we introduce a contextual engineering framework that generates eco-feedback using a self-consistency with chain-of-thought prompt, leveraging household energy analysis data, utility rate structures, and household characteristic information. We conducted a rigorous empirical validation and a combinatorial evaluation analysis to assess this framework systematically. The former tested the framework's ability to generate accurate and contextually grounded eco-feedback for three households by comparing its output against reference interventions independently derived from the same household data. The latter examined the framework's adaptability across 400 scenarios spanning 50 households, two utility rate structures, and four behavioral personas. Our framework generated eco-feedback that aligned with reference interventions at a mean rate of 92.0% and grounded its recommendations in the provided household data with 95.7% citation accuracy. It also proved highly adaptive, shifting both the appliances targeted and the energy-saving strategies recommended in response to rate structure and household context. Ultimately, this study contributes to realizing the next level of context-aware interactions between occupants and buildings which paves the way for higher occupant living quality and sustainability.