Floor It, Then Fill It: Simulating Creator Ecosystems for Fair Video-Content Recommendation Systems at Billion User Scale

AI summary

Overview

Research area: Recommender systems / machine learning — specifically fairness in video-content recommendation, creator-side equity, and user-simulation methodology for evaluating debiasing policies.

Technical level: Advanced. The paper assumes familiarity with multi-armed bandits, explore-exploit trade-offs, ranking-system feedback loops, and A/B testing.

Scope: The paper builds a grounded simulator of ~42K creators and their viewers over 12 weeks, and uses it to evaluate a budgeted multi-armed-bandit view-allocation policy designed to stop large creators from capturing a platform's distribution budget — plus an honest accounting of where such a simulator may mislead.

What This Paper Is About

Engagement-optimized recommenders create a rich-get-richer dynamic: established creators get exposure, generate reliable engagement signal, get more exposure, while new creators have little signal and stay buried. Testing fixes for this normally requires long, expensive, ethically fraught live experiments on real users. The authors instead build a simulated creator-viewer ecosystem, calibrated to real platform distributions, and use it to test a "floor it, then fill it" allocation policy — guarantee every creator a minimum share of views, then spend the leftover budget where it does the most good. They also deliberately interrogate whether conclusions drawn from such a simulator can be trusted.

Key Contributions

1. Diversity — a heterogeneous, grounded population. A fixed cohort of 42,436 creators spanning four behaviorally distinct archetypes (T1 large under-parity, T2 small under-parity, T3 small over-parity, T4 large over-parity), preserving real platform distributions of follower counts (14,254 creators with fewer than 1,000 followers), engagement rates (EWTR, LTR, PSR), breakout rates (1d1K, 1d10K, 1d100K) and production patterns — with explicit representation of the under-served tail.

2. Fidelity — two coupled state machines. A Creator State Machine (Exploration/cold-start → Growth → Graduation, with regression back to earlier stages) coupled to a Content State Machine that stages individual videos by VPV targets ranging from thousands to ~1M based on retention and engaged watch time. Together these model cold-start loops, engagement accumulation, organic compounding, and graduation to self-sustaining reach.

3. An equity-constrained allocation policy. A budgeted multi-armed bandit that is batch (all arms pulled weekly), budgeted, hard-floor constrained per creator, and non-stationary — combining a non-negotiable parity floor with reward-greedy water-filling of the discretionary remainder, plus a tunable small-creator reserve fraction ρ.

4. Validity — an explicit Sim2Real analysis. Identification of where the simulation aligns with production (population distributions, behavioral trends, robustness of policy rankings) and where it may diverge (7-day snapshots vs. hourly pacing, organic attribution, population drift, strategic creator adaptation), plus a concrete validation protocol.

Main Findings

• Floor + parity-gap reward meets the coverage promise. A floor-protected per-content-view-target combined with a view-parity-gap reward guarantees 90% of view targets are met by creators every week, and under-VPV-parity creators are funded 100% of the time.

• Small-creator organic reach compounds. The deployed configuration grows small-creator organic reach by ~+33% per week.

• Stable allocations. Funded-views Jaccard = 0.819; the parity-gap reward achieves the highest rank-order consistency (Spearman = 0.645) and lowest turnover (0.144) among the four reward functions. Comparable values: vpv_only (0.520 Spearman, 0.816 Jaccard, 0.700 top-10% Jaccard, 0.190 turnover), parity_x_eng (0.524, 0.819, 0.725, 0.195), hybrid (0.635, 0.817, 0.715, 0.153).

• Delivered views grow over the horizon. Total system-delivered D-VPVs grow from 173M at week 0 to 244M at week 11 under the deployed configuration (graduated target, ρ=0.7, parity-gap reward).

• The organic-to-allocated ratio splits sharply by type. Approximately 0.02 for T1, ~0.6 for T2, ~2 for T3, and 125 to 200 for T4 — i.e., large over-parity creators are overwhelmingly organic, while large under-parity creators are almost entirely boost-dependent.

• Engagement metrics rise monotonically across growth stages, consistent with real platform observations: EWTR 22.93% (Exploration) → 24.25% (Growth) → 30.54% (Graduation); 1d1K rate 83.33% → 87.63% → 92.32%; 1d10K rate 27.44% → 30.01% → 37.71%. Active creators per stage: 10,775 / 8,488 / 6,148. LTR is non-monotonic (3.118 / 2.612 / 3.472), as is the 1d100K rate (1.99 / 1.37 / 2.18).

• Growth trajectories are large and uneven. Over the 12-week simulation on the balanced cohort of 26,958 continuously active creators, 47.6% grew their weekly VPV. Mean growth ratios (week 11 / week 0): T1 170.98× (median 10 → 14, with the breakout tail driving the mean), T2 23.03× (median 8,997 → 30,920), T3 11.57× (median 14,872 → 34,104), T4 1.46× (median 115,809 → 139,824).

• Net parity crossings are slightly positive. 3,222 creators climb from under- to over-parity versus 3,125 falling from over- to under-parity (net +97). Individual trajectories show creators scaling from fewer than 100 views to over 2.7M within 12 weeks.

• The graduated target unlocks discretionary optimization. It shifts budget from the mechanical floor into the reward-steered pool (11% → 31%), and 24–32% of budget remains reward-steered.

• Hybrid reward weight shows a monotonic trade-off. Sweeping w lowers week-over-week stability and shifts over-funding to large creators as w rises, saturating near w ≈ 0.8; a balanced split occurs near w = 0.5, and w = 0.4 was deployed to lean toward small creators.

• The population is strongly bimodal. Each week most creators sit either far below (T1) or far above (T4) their per-content target; the weekly cohort grows from 26,959 creators at week 0 to 42,317 at week 11.

Methodology in Plain English

The authors build a world, not an experiment on real users. They take a fixed set of 42,436 creators drawn to match real platform statistics, enroll them in weekly batches over 12 weeks, and give each one a state that evolves: a brand-new creator is in a cold-start "exploration" stage where the system can only match them to viewers by similarity rather than engagement history; as they accumulate real engagement they move to "growth"; then to a "graduation" trial where support tapers and organic performance is checked — and they can fall back if it doesn't hold. Individual videos have their own lifecycle, staged by view targets from thousands up to about a million.

Each week, the allocator works in two phases. Phase 1, the floor: every creator is guaranteed at least 90% of their target views, but organic views count toward that — if organic already clears 90% of target, no funded views are spent. Phase 2, water-filling: whatever budget remains after the floor is handed out greedily by a reward function, ranked high to low, until the budget is exhausted, with a per-creator cap of three times their target.

The target itself comes in two flavors: a flat step target (3,000 views if under 1,000 followers, else 10,000) and a graduated target that scales with follower count via a square root, floored at 3,000 and capped at 100,000. The weekly budget is set to whichever is larger — half the sum of all targets, or the full cost of the floor — so the floor is never underfunded.

To stop large creators from swallowing the discretionary pool, a reserve fraction ρ carves off part of it exclusively for under-parity creators; the deployed value is ρ = 0.7. Four reward functions are compared: reward existing reach (vpv_only, which by construction favors the already-popular), reward the distance below target (parity_gap), reward that distance weighted by content quality (parity_x_eng), and a tunable blend of reach and gap (hybrid).

Finally, the authors audit their own simulator: they compare weekly allocated views and organic-to-allocated ratios against production, enumerate the assumptions most likely to break, and lay out a validation plan.

Why This Matters

Impact on research. Simulators are becoming infrastructure for training and evaluating recommenders, but the paper adds needed skepticism: an overly cooperative or poorly calibrated simulator inflates fairness metrics and the conclusions don't transfer. By pairing a concrete fairness mechanism with an explicit Sim2Real gap analysis, the paper provides a template for "when can I trust a simulation-derived fairness claim?" It also shows that single-step evaluation systematically underestimates both the problem and the solution, because engagement signals compound over weeks.

Real-world applications.

• Creator-marketplace platforms (short video, streaming, music) designing cold-start and newcomer-promotion programs without running months of live experiments.
• Fairness auditing and compliance — producing longitudinal, pre-deployment evidence that an allocation policy meets coverage commitments to small creators.
• Budget and inventory planning — sizing weekly view budgets that guarantee floors while leaving room for optimization, using the ρ lever and the target definitions as policy knobs.
• RL environment research — the simulator doubles as a non-stationary, bimodal, population-scale environment for training bandit agents.

Industry relevance. The authors are at Meta Platforms, and the setting is explicitly a billion-scale video platform with weekly batch decisions, non-monetary view budgets, and per-creator equity floors — a combination the paper argues is not covered by prior bandit work such as LinUCB or BanditQ. The ρ lever is the practical artifact: a single dial that monotonically shifts discretionary budget toward small creators, tunable per segment.

Future Directions

1. Close the Sim2Real gap empirically. The proposed protocol — a production holdout group that attributes organic growth and graduation rates to the policy (reporting incremental lift rather than raw endpoints), per-segment A/B tests of ρ and reward choice against simulated predictions, and concordance metrics like ranking agreement and directional accuracy — is proposed but not yet executed at scale.

2. Model strategic creators. Real creators may optimize toward the floor rather than for viewers, violating the simulator's non-strategic assumption; the paper flags this as an open validity threat with no current mitigation.

3. Increase temporal resolution and fix attribution. Simulation uses 7-day snapshots while production paces hourly, and organic reach in production is entangled with the boosted path, making clean causal attribution of creator growth difficult — the paper notes creator growth estimates can be far from reliable.

4. Handle population drift. The cohort is fixed, whereas real populations drift weekly; the bimodal four-archetype structure would need to be re-estimated as new creators enter and others churn.

Target Audience

Researchers and applied scientists working on recommender-system fairness, creator-economy incentives, or simulation-based evaluation of ML systems. It is most valuable to practitioners who need to justify a debiasing policy before an online launch and want a concrete example of how to design the mechanism, ablate its reward function and reserve lever, and interrogate the simulator's own credibility. Readers without a bandits or ranking-systems background will find the technical sections dense, though the allocation logic and the Sim2Real caveats are accessible in outline.

Authors’ abstract

Engagement-optimized content recommendation systems exhibit a well-documented rich-get-richer dynamic that structurally disadvantages emerging creators on large-scale platforms. Evaluating debiasing interventions at scale demands faithful simulation of creator-viewer ecosystems, yet the fidelity, diversity, and validity of such simulators remain underexplored. We present a grounded user-simulation framework that models a heterogeneous population of ${\sim}42$K creators and their coupled viewer interactions over 12~weeks, and use it to evaluate equitable view-allocation policies formulated as a budgeted multi-armed bandit (MAB). The simulator captures three behavioral dimensions: (i)~\textbf{creator diversity}, modeled via a clustering approach grounded in real platform creator-level distributions; (ii)~\textbf{behavioral fidelity}, captured through coupled creator-content state machines that represent growth trajectories, engagement accumulation, and graduation dynamics; and (iii)~\textbf{population-level validity}, measured by calibration against observed platform metrics. We evaluate simulative-configurations that vary content-viewership target definitions, reward functions, and a small-creator reserve lever to maximize creator-content recommendation equity. A floor-protected per content-view-target with a view-parity-gap reward guarantees 90\% of view targets being met by creators every week, grows small-creator organic reach ${\sim}{+}33\%$ per week, and achieves high distribution stability (set of creators with funded-views Jaccard $= 0.819$). We analyze the Sim2Real gap and propose validation protocols, contributing evidence for when simulation-derived fairness conclusions can be trusted.