Creator Incentives in Recommender Systems: A Cooperative Game-Theoretic Approach for Stable and Fair Collaboration in Multi-Agent Bandits

AI summary

Overview

• Research area: Multi-agent bandit learning, cooperative game theory (transferable utility games, Shapley value), and incentives for content creators in recommender systems.
• Technical level: Advanced. The paper assumes familiarity with stochastic linear bandits, pseudo-regret, core/balanced games, convex (supermodular) games, and the Shapley axioms.
• Scope: The paper models collaboration among content creators as a multi-agent stochastic linear bandit whose induced transferable utility game is analyzed for stability and fairness, with numerical simulations on MovieLens-100k.

What This Paper Is About

When a recommender platform learns from user feedback on one creator's content, that learning changes what gets shown to users for other creators, so creators' fates are entangled. The paper asks how this coupling should be modeled and whether creators can be given payouts that make full collaboration both stable (no group wants to break away) and fair (each gets credit for what they contribute). It answers by defining a game where a coalition's value is the negative sum of its members' cumulative regrets, then characterizing the game's core and a simple payout rule.

Key Contributions

1. Homogeneous agents with fixed action sets. For a symmetric setting where all agents share one fixed, finite action set across time, the authors prove that the induced collaboration game is convex under a "mild" assumption on the underlying single-agent algorithm (Assumption 1), which implies a non-empty core that contains the Shapley value, making the grand coalition stable and equitable.
2. Heterogeneous agents with diverse action sets. For agents with differing action sets (e.g., content specializations), they prove the game still has a non-empty core under an algorithmic assumption (Assumption 3), though convexity and Shapley-value core membership are no longer guaranteed.
3. A regret-based payout rule. They propose the allocation that pays each agent the negative of its regret in the grand coalition, and prove it satisfies three of the four Shapley axioms (efficiency, dummy-player, symmetry) while also lying in the core; the linearity axiom is argued not to hold in general.
4. Empirical validation. Simulations on MovieLens-100k with four user-attribute splits and two algorithms (LinUcb-M and Greedy) show when the regret-based payout aligns with, and when it diverges from, the empirical Shapley value.

Main Findings

• Convexity for identical agents (Theorem 1): With the meta-algorithm Mul and a single-agent black-box Sin obeying Assumption 1, the collaboration game on fixed-action-set instances is convex for a sufficiently large horizon T.
• Regret correspondence (Lemma 1): For a coalition of size m over an action set of size K, the coalition's total regret is bounded above and below by a single agent's regret run for mT rounds, up to an additive term of order mK that is independent of T.
• Stability of the grand coalition (Corollary 1 and Theorem 2): Convexity gives a non-empty core containing the Shapley value in the fixed-action-set case; with heterogeneous action sets, the "the more the merrier" assumption (Assumption 3) alone guarantees a balanced game and hence a non-empty core.
• Fair payout without counterfactual computation (Theorem 3): The allocation p_a = −R_a^M satisfies efficiency, dummy-player, and symmetry, and belongs to the core. The paper argues it cannot satisfy linearity in general.
• Symmetry from anonymized data: The symmetry axiom holds because Assumption 2 requires that agents' pooled data be anonymized, so play trajectories and regrets are unchanged under relabeling of agents.
• MovieLens-100k, occupation split (I_occ): Payouts and empirical Shapley values are reasonably close for almost all agents under both LinUcb-M and Greedy, with a positive correlation across agents that is more prominent under Greedy.
• MovieLens-100k, geography split (I_geo): Greater disparity appears. Agents far above or to the left of the identity line have much higher payout than Shapley value (they benefit more from others than they contribute), while agents on the bottom-right contribute more than they benefit. These mismatched agents are predominantly users from New York and Pennsylvania under LinUcb-M, and New York and Maryland under Greedy, in this example.
• Greedy versus LinUCB: The Greedy algorithm performs better at minimizing regret than LinUCB in both the I_geo and I_occ instances. The paper attributes this to heterogeneity in action sets producing inadvertent exploration, backing recent work showing greedy can achieve non-trivial regret bounds under sufficient heterogeneity.

Methodology in Plain English

The authors set up a bandit problem where each agent picks an action vector, receives a noisy linear reward, and has its own action set. Agents are allowed to join coalitions beforehand; inside a coalition they pool the action-reward pairs they observe, and outside it they share nothing. Using a chosen multi-agent algorithm, the paper defines each coalition's value as the negative of the sum of its members' regrets, then treats this as a cooperative game and asks whether the "grand coalition" of everyone is stable and how payouts should be split.

For the simple case where all agents share the same fixed action set, they analyze a meta-algorithm (Mul) that repeatedly feeds an ordinary single-agent bandit algorithm (Sin) from a shared buffer of rewards collected by all coalition members. Assuming the single-agent algorithm's regret curve is strictly concave but cannot improve faster than logarithmic, they show the coalition's regret tracks a single agent's regret run for a number of rounds proportional to coalition size, which makes the game convex. For richer settings where agents' action sets differ, they do not fix an algorithm; instead they assume the algorithm consumes anonymized pooled data and that adding agents never increases an agent's regret, and then prove the game's core is non-empty using the Bondareva-Shapley characterization of balanced games.

Finally, they test on MovieLens-100k, building four bandit instances by splitting users by gender, age group, geographic location, and occupation, with one agent per attribute value. For each instance they run the multi-agent algorithm for all 2^M − 1 coalitions, compute the empirical Shapley value from those coalitional regrets, and scatter-plot each agent's grand-coalition payout against its Shapley value.

Why This Matters

Impact on research. The paper connects multi-agent bandit regret bounds to cooperative game theory, showing that algorithm-level assumptions (concavity of the regret curve, monotone regret improvement with coalition size) translate directly into game-theoretic properties like convexity and a non-empty core. It also gives a payout rule that needs no counterfactual regret computation across all coalitions, sidestepping the exponential cost of exact Shapley values.

Real-world applications (bullets):

• Recommender platforms such as the video platform example given in the paper, where feedback on one creator's content determines whether another creator's content is shown.
• Designing creator payout or revenue-share policies so that payouts reflect marginal contribution rather than raw engagement alone.
• Deciding whether to offer data-sharing or collaboration agreements among content creators, where the core answers whether all parties would prefer to stay together.
• Auditing engagement-based compensation for equity, since the paper shows specific user segments (New York and Pennsylvania under LinUcb-M; New York and Maryland under Greedy) can be systematically under- or over-compensated relative to Shapley value.

Industry relevance. Co-authored by a researcher at FAIR, Meta AI alongside NYU Courant and IIT Bombay, the work speaks directly to two-sided recommendation markets where platform learning couples creators' visibility and monetization.

Future Directions

• The linearity gap. The paper explicitly leaves open why the regret-based payout cannot satisfy the linearity axiom in general, and whether another practical rule could satisfy all four Shapley axioms.
• Convexity for heterogeneous agents. It remains unclear whether collaboration games with diverse action sets are ever convex or whether the Shapley value can lie in their core.
• Computational scale. Exact Shapley values are exponential in the number of agents and require regrets under all coalitions, so the experiments are limited to small instances; scaling the analysis to larger creator populations is unresolved.
• Broader empirical validation. The current simulations cover MovieLens-100k plus synthetic instances (deferred to Section E.2), with two algorithms and four attribute splits; testing more datasets, algorithms, and settings would clarify when payout and Shapley value diverge.

Target Audience

Researchers and graduate students working on multi-agent bandits, cooperative game theory, or mechanism design; recommender-system and platform-economics practitioners interested in creator incentives and fair revenue allocation; and applied scientists at platforms who need payout rules that are both theoretically grounded and computable. Readers without a background in bandit regret analysis or the Shapley axioms will find the theoretical sections demanding.

Authors’ abstract

User interactions in online recommendation platforms create interdependencies among content creators: feedback on one creator's content influences the system's learning and, in turn, the exposure of other creators' contents. To analyze incentives in such settings, we model collaboration as a multi-agent stochastic linear bandit problem with a transferable utility (TU) cooperative game formulation, where a coalition's value equals the negative sum of its members' cumulative regrets. We show that, for identical (homogenous) agents with fixed action sets, the induced TU game is convex under mild algorithmic conditions, implying a non-empty core that contains the Shapley value and ensures both stability and fairness. For heterogeneous agents, the game still admits a non-empty core, though convexity and Shapley value core-membership are no longer guaranteed. To address this, we propose a simple regret-based payout rule that satisfies three out of the four Shapley axioms and also lies in the core. Experiments on MovieLens-100k dataset illustrate when the empirical payout aligns with -- and diverges from -- the Shapley fairness across different settings and algorithms.