ValuePilot: A Two-Phase Framework for Value-Driven Decision-Making

AI summary

Overview

Research area: Artificial Intelligence — personalized, value-driven decision-making and human-AI alignment.

Technical level: Intermediate. The paper combines a language-model data-generation pipeline with a neural scoring network and a multi-criteria decision-making method (PROMETHEE), so some familiarity with LLMs, encoders, and ranking methods helps, but the core ideas are described conceptually.

Scope: The paper introduces ValuePilot, a two-phase framework (a Dataset Generation Toolkit and a Decision-Making Module) for training agents to select actions that match an individual user's stated value priorities, and evaluates it against strong LLM baselines plus a 40-participant human study.

What This Paper Is About

Most AI decision-making systems are task-oriented: they optimize a predefined external reward, and value alignment methods like RLHF or DPO typically learn from aggregated, collective human feedback rather than individual preferences. This paper argues that human decisions are guided by stable but individually weighted "value dimensions" (such as curiosity, safety, intimacy, fairness), and that explicitly modeling both the dimensions and a person's priorities over them produces more interpretable, personalized choices — including in scenarios the model never saw during training. The goal is to build the data generation pipeline and the decision module needed to make that kind of value-driven choice work in practice.

Key Contributions

1. A multi-stage LLM generation pipeline (DGT) that turns a specified set of value dimensions into structured decision scenarios, each with candidate actions annotated with numerical value scores, followed by automatic re-evaluation filtering and human review.
2. A personalized decision-making module (DMM) that combines an objective Value Assessment Network (which estimates how each action affects each value dimension) with a user's subjective value preference vector, producing individualized action rankings.
3. Integration of the PROMETHEE multi-criteria decision-making method into the DMM, so the agent can handle trade-offs across multiple value dimensions and output a ranked, interpretable action list.
4. An evaluation combining benchmark comparison against LLMs and a human study with 40 participants and 11 formal scenarios, using order-sensitive ranking similarity (OS-Sim) and first-choice accuracy (First-Acc) as metrics.

Main Findings

• Value recognition beats open-source LLMs: The Value Assessment Network achieves 66.70% average accuracy at threshold t = 0.2 and 40.00% at t = 0.05, with a mean absolute error of 0.19. This is an improvement of 15.09 and 14.36 percentage points respectively over the strongest baseline, Gemini-1.5-Flash, and a 36.7% relative reduction in MAE.
• Baseline value-recognition scores: llama-3.5-70b scored 40.90% (t = 0.2) and 17.74% (t = 0.05) with MAE 0.30; llama-3.5-405b scored 41.62% and 18.00% with MAE 0.29; mixtral-8x22b scored 42.71% and 18.39% with MAE 0.29; gemini-1.5-flash scored 51.61% and 25.64% with MAE 0.24.
• Improved alignment with human action choices: On unseen scenarios, DMM reaches a mean OS-Sim of 73.16% (± 0.43%), versus 69.23% (± 0.71%) for the strongest LLM baseline, GPT-5 — a gain of +3.93%.
• Better first-choice accuracy: DMM's First-Acc is 46.14% (± 4.09%) versus GPT-5's 38.01% (± 3.81%), a +8.13% relative improvement.
• Every component contributes: Ablations show the full DMM (73.16% OS-Sim, 46.14% First-Acc) outperforms "Only Action" (60.23%, 32.27%), "w/o Preference" (61.07%, 31.82%), "w/o Subjective" (68.93%, 43.45%), and "w/o Scenario" (69.99%, 43.64%).
• Dataset scale: After generation and screening, the curated dataset contains 11,938 scenarios and 100,255 actions in total. The hierarchical breakdown reports 9,899 training scenarios (83,473 actions) and 2,039 test scenarios (16,782 actions) across six complexity tiers (1-D through 6-D).

Methodology in Plain English

The researchers built the framework in two halves.

First, generating data (DGT). Because no existing dataset links everyday decision scenarios to fine-grained value dimensions, they used GPT-4 to create domestic multi-agent scenarios that implicitly exercise selected values, deliberately avoiding the value keywords themselves so the recognition task stays non-trivial. For each scenario, GPT-4 generates 10 possible actions, and each action receives a score from −1 to +1 on each value dimension: near −1 means the action strongly contradicts that value, near 0 means neutral, and near +1 means strong alignment. GPT-4 is then re-prompted in a separate session to re-identify the values in each sample; if the re-identified dimensions diverge from the target set, the sample is discarded. Finally, a four-member team with AI and psychology backgrounds reviews the samples for realism, action coherence, scenario diversity, and value alignment.

Second, choosing actions (DMM). A T5 encoder converts the scenario and each action into representations; a four-head self-attention mechanism captures the relationship between them, the outputs are averaged and passed through a two-layer MLP with hidden size 128, and a tanh activation produces a continuous score in [−1, 1] per value dimension. These objective scores are combined with the user's self-reported 0-to-1 value preferences. Because people tend to avoid extreme ratings even for strong preferences, the preference vector is pushed apart with a sigmoid transform using a scaling factor of 10. For each dimension the system computes how closely an action's value impact matches (or diverges from) the user's preference, mixes that discrepancy with the objective score using a weight w = 0.3, and then scales the action's score by how relevant that dimension is to the scenario. Finally, PROMETHEE compares actions pairwise across all value dimensions, weighting each comparison by the user's preferences, and ranks actions by their net outranking flow — the difference between how much an action dominates others and how much it is dominated.

Evaluation. Value recognition was tested against Llama-3.5-70b, Llama-3.5-405b, Mixtral-8x22b, and Gemini-1.5-flash using thresholded average accuracy and MAE. Decision alignment was tested with a human study: 40 subjects rated the importance of six value dimensions on a 0-to-1 scale, completed three pilot questions, could rescale their ratings, and then ranked candidate actions in 11 formal domestic scenarios. Those personal preference vectors were fed to DMM and to baseline LLMs (Llama-3.1-70b, DeepSeek-R1, Claude-Sonnet-4, Gemini-2.5-flash, Kimi-K2, GPT-4o-mini, and GPT-5), which also received dimension definitions and two real participants' preferences and rankings as few-shot examples. Rankings were scored with OS-Sim (a prefix-based variant of Jaccard similarity that rewards getting early positions right) and First-Acc (whether the model's top-ranked action matches the person's first choice).

Why This Matters

Research impact: The work argues that explicit value dimensions plus individual preference vectors are a more interpretable and generalizable route to personalization than learning implicit preferences from collective feedback, and it supplies the dataset-generation machinery and an ablation-validated architecture to test that claim.

Real-world applications:

• Home and domestic assistants that must choose between, for example, a familiar safe option and an exploratory one, depending on the resident's priorities.
• Personalized recommendation or planning agents where the trade-off between competing goals (efficiency vs. enjoyment, novelty vs. security) differs per user.
• Decision-support tools for care, education, or personal finance, where the system should surface why an option ranks highly in terms of the user's stated values.
• Human-robot or multi-agent coordination in shared spaces, where fairness and intimacy considerations must be balanced against individual goals.

Industry relevance: The framework is presented as an "engineering pathway" — a modular pipeline that a product team could populate with its own value dimensions, generate training data with an LLM, and serve interpretable action rankings. The paper also notes the framework is model-agnostic across encoders (T5 slightly outperformed alternatives in accuracy), which lowers the barrier to swapping in different backbone models.

Future Directions

• Scaling beyond the six dimensions: The paper selected Curiosity, Energy, Security (referred to as Safety in the appendix), Happiness, Intimacy, and Fairness for this study, and frames the design as flexible enough for system designers to specify other value sets — an untested extension.
• Synthetic data risks: The authors cite work showing overreliance on unfiltered synthetic data can degrade model robustness, and rely on human review to mitigate this; how well the pipeline holds up at much larger scale, or with lighter human oversight, is unresolved.
• Richer evaluation: The human study uses 40 subjects and 11 scenarios, and the paper describes itself as preliminary and "an initial step," leaving open whether results hold across broader populations and more varied, real-world decision contexts.
• Deeper integration with existing alignment methods: The related-work discussion positions ValuePilot against RLHF and DPO, which learn from collective feedback; how explicit value modeling might combine with those approaches rather than replace them is not addressed.

Target Audience

Researchers and practitioners working on human-AI interaction, personalization, value alignment, and agent decision-making will get the most from this paper, along with engineers building assistant or planning systems who need an interpretable, preference-aware ranking mechanism. Readers interested in multi-criteria decision-making will find the PROMETHEE integration notable, and those who need a benchmark connecting LLM value reasoning to human choices will find the DGT pipeline and evaluation protocol the most directly reusable parts.

Authors’ abstract

Personalized decision-making is essential for human-AI interaction, enabling AI agents to act in alignment with individual users' value preferences. As AI systems expand into real-world applications, adapting to personalized values beyond task completion or collective alignment has become a critical challenge. We address this by proposing a value-driven approach to personalized decision-making. Human values serve as stable, transferable signals that support consistent and generalizable behavior across contexts. Compared to task-oriented paradigms driven by external rewards and incentives, value-driven decision-making enhances interpretability and enables agents to act appropriately even in novel scenarios. We introduce ValuePilot, a two-phase framework consisting of a dataset generation toolkit (DGT) and a decision-making module (DMM). DGT constructs diverse, value-annotated scenarios from a human-LLM collaborative pipeline. DMM learns to evaluate actions based on personal value preferences, enabling context-sensitive, individualized decisions. When evaluated on previously unseen scenarios, DMM outperforms strong LLM baselines, including GPT-5, Claude-Sonnet-4, Gemini-2-flash, and Llama-3.1-70b, in aligning with human action choices. Our results demonstrate that value-driven decision-making is an effective and extensible engineering pathway toward building interpretable, personalized AI agents.