HumanCentric-Scenes-AI: A Human-Curated Multimodal Dataset of AI-Generated Human-Centric Scenes Across Surveillance, Occupational Portraits, AI-Edited Images, and Short-Form Video

AI summary

Overview

Research area: Computer vision and generative media — specifically the construction of a human-curated stimulus dataset for studying how people perceive AI-generated depictions of other people, with secondary use for AI-detection calibration and bias auditing of image/video generators.

Technical level: Intermediate. The dataset description is accessible to newcomers, but the accompanying analyses use nonparametric statistics (Mann–Whitney U, exact binomial tests, sign tests, Spearman correlation with permutation tests) and zero-shot LLM classification.

Scope: The paper introduces and characterizes HumanCentric-Scenes-AI, a release of 296 AI-generated stimuli spanning four domains — surveillance-style imagery, occupational portraits, partially AI-edited photographs, and short-form video — each with per-stimulus prompts and structured metadata.

What This Paper Is About

Existing detection benchmarks are dominated by face swaps in otherwise authentic video or by large whole-image generation sets aimed at training classifiers. This paper instead asks what people actually encounter when they see AI-generated depictions of people, and builds a small, manually curated stimulus set covering four such contexts: low-quality CCTV footage, occupational portraits that carry gender associations, photographs where only a region was altered, and social-media-style short video. The goal is a reusable, metadata-rich stimulus collection for human-perception research, supplemented by per-domain characterizations against outside references such as U.S. Bureau of Labor Statistics occupational gender shares.

Key Contributions

1. A four-domain multimodal release of 296 AI-generated stimuli. The Surveillance subset contains 85 Midjourney CCTV-style images; Occupations contains 128 Midjourney portraits; AI-edited contains 27 photographs with a region altered or replaced by Midjourney inpainting; Video contains 56 clips (Veo 35, Sora 18, Kling 1, 2 unattributed).
2. A balanced paired design for occupation–gender studies. The Occupations subset covers 64 occupations with one female and one male version of each, so gender balance is guaranteed by construction rather than sampled.
3. A separate audit of the generator's default. An initial set of 134 generations without a specified gender, spanning 67 occupations prompted twice (once by occupation name, once by paraphrased description), exposes the unconstrained gender prior that the balanced release set by design cannot show.
4. Per-domain characterization against third-party references — image-quality comparisons against real CCTV frames from the CCTV Human Pose Estimation Dataset, gender shares against U.S. BLS Current Population Survey (2025 annual averages, Table cpsaat11), edit-region size against zero-shot LLM detection accuracy, and duration/resolution statistics for video — plus a datasheet following the Gebru et al. framework.

Main Findings

• AI-generated "CCTV" is still sharper than real CCTV. Despite degradation-oriented prompts ("grainy low-resolution CCTV footage, 1990s video compression"), Midjourney outputs on the 25-image characterization subset had a median Laplacian variance of 649 versus 212 for 26 real CCTV frames (Mann–Whitney U = 482, z = 3.29, p = 0.001). Median file size was 128 KB [77–203] versus 98 KB [67–157]; all stimuli were normalized to 600 × 600 px.
• Midjourney defaults to depicting men for occupational prompts. Across the 134 gender-neutral generations, 99 (73.9%) depicted a man and 33 (24.6%) a woman, with 2 unclear. The split was 55/67 male (82.1%) when the prompt named the occupation (binomial p = 3.6 × 10⁻⁸) and 44/67 (65.7%) when the role was described instead (p = 9.2 × 10⁻³).
• Prompt phrasing shifts the outcome by 16.4 percentage points. The paper reports this phrasing effect between the occupation-name and descriptive conditions, and finds 15 of 67 occupations changed depicted gender between the two styles — 13 flipping male → female under descriptive phrasing and 2 the other way (two-sided sign test, p = 0.007). Flips toward female included actor, architect, astronaut, author, baker, biker, biologist, firefighter, ice-cream vendor, journalist, painter, singer and student; flips toward male were florist and interior designer.
• The female renders it does produce track real labor statistics. Of the 11 of 67 occupations rendered female under occupation-name prompting, 7 of the 8 with a published BLS share are female-majority occupations (e.g., dental hygienists 95.0%, hairdressers 92.0%, registered nurses 87.3%), all above the 47.1% baseline share of women among the employed. Newscaster is the exception at 41.1%; model, florist and cheerleader have no published BLS share.
• Larger AI edits are easier for zero-shot models to catch. Across 25 characterized real–edited pairs, edit-region size correlated positively with mean zero-shot accuracy across three LLMs (GPT-4.1, Claude Opus, Gemini Pro): Spearman ρ = 0.55, permutation p = 0.005. Accuracy spanned 0 to 3 of 3 models correct.
• Most edits are person-to-person swaps, and they touch relatively few pixels. Person-replaced-with-person was the most common edit type (9 of 25, median 12.7% of pixels altered), followed by background replacement (6, 23.2%), object insertion or replacement (5, 9.8%), person replaced with an object or animal (4, 26.3%), and local attribute re-rendering (1, 1.3%). Overall median altered share was 14.3% (range 1.3–53.9%), median edit bounding box covered 60.4% of the image, and median global SSIM between source and edit was 0.788.
• AI video durations match the real reference clips. In the 20-video characterization subset spanning six subject categories, median duration was 8.0 s for both AI-generated videos and real references ([5.7–10.0] vs [6.6–8.5]). AI clips spanned 10 distinct resolutions (528–1280 px wide) against 2 for real clips (960 × 540 and 360 × 360), and all 20 in each group carried an audio track.

Methodology in Plain English

The authors generated candidate images and videos with commercial models and then reviewed every candidate by hand over multiple rounds, keeping only those without visible artifacts that would give them away as synthetic. Surveillance and Occupations images came from Midjourney; the CCTV set used prompts deliberately asking for noise, blur, VHS artifacts and low frame rates. The Occupations set was generated as matched female/male pairs for each of 64 occupations. For the AI-edited subset, they took existing public-source photographs and used Midjourney inpainting to remove, change or add content in one region, leaving the rest of the scene untouched.

Before building the balanced set, they had already generated the same 67 occupations with no gender specified, once by name and once by description, giving 134 generations used only for the bias analysis. They counted depicted gender in those and compared the results to BLS occupational shares, using exact binomial tests against a 50/50 expectation and a sign test for the phrasing comparison.

For the edited images, they coded each of the 25 characterized pairs by hand into one of five edit types after inspecting the source, the edit, and a pixel-difference map. They measured the share of altered pixels with a per-pixel absolute-difference threshold of 25/255, computed global SSIM, and asked three LLMs zero-shot to judge whether each image was manipulated, then correlated accuracy with edit size. For video they measured technical properties with ffprobe and compared AI clips to real reference clips from the same subject categories. All analysis scripts are released so the characterization subsets can be extended to the full subsets.

Why This Matters

Detection benchmarks are usually optimized for training classifiers on large volumes of generated media; this release is optimized for asking what human viewers notice, in the specific formats where they actually encounter synthetic people. That distinction matters because the paper's own results show that degradation-oriented prompting does not reproduce real CCTV statistics, and that the generator's gender prior is strong enough that a balanced dataset must be built deliberately rather than sampled.

Real-world applications:

• Designing and auditing human-perception studies on whether viewers can spot synthetic people in surveillance, professional, and social-media contexts.
• Calibrating and stress-testing AI-image and AI-video detectors, particularly on small, localized edits where the paper finds detection is hardest.
• Measuring demographic representation in text-to-image models, using the balanced portrait pairs and the occupation-name-versus-description comparison as a prompt-design probe.
• Providing realistic synthetic stimulus material for media-literacy training without depicting real identifiable people in the Surveillance, Occupations and Video subsets.

Industry relevance: Teams building or evaluating generative image and video products can use the release to quantify default demographic priors and how prompt phrasing moves them, and trust-and-safety groups working on provenance and manipulation detection get a curated, small-scale test set with edit-region ground truth. The licensing is CC BY-NC 4.0 for the AI-generated assets, metadata and curation, with the AI-edited subset released as derivative works under a fair-use rationale for academic research, and the datasheet explicitly prohibits using the dataset to train generative models.

Future Directions

• Scaling the characterizations to the full release. Several statistics are computed on 25-stimulus subsets (Surveillance and AI-edited) or a 20-video subset (Video), not the full 85, 27 and 56; the authors state the same scripts apply to the full subsets and that extending them is a one-line change.
• Tracking how quickly these regularities decay. The authors note that every difficulty estimate ages as generators improve, and that recording the generator per stimulus is what makes future decay measurement possible. Re-running the CCTV sharpness and edit-detection analyses on newer model versions is a direct follow-up.
• Extending the demographic audit beyond gender and beyond Midjourney. The bias analysis covers depicted gender only, on one generator; the same paired design could be applied to other attributes and to Veo, Sora and Kling outputs, where the release currently provides video but no bias characterization.
• Repairing and expanding the metadata. The maintenance section flags that the Video subset's Veo stimuli are pending restoration to the repository's main branch and that their generator labels in metadata.csv read gemini and need correction to veo; the Midjourney version is recorded for only 32 of the 85 Surveillance images.

Target Audience

Researchers in human perception and media psychology who need controlled, photorealistic stimulus sets; computer-vision researchers working on AI-generated media detection who want a small, curated, metadata-rich evaluation set rather than a training corpus; and fairness/bias researchers auditing demographic priors in text-to-image models. It is also useful to media-forensics practitioners and policy-oriented readers who want concrete, quantified examples of how generators deviate from the real-world conditions they are prompted to imitate. Readers seeking a large-scale detection benchmark should look elsewhere, since the authors describe the release as small by design.

Authors’ abstract

We introduce HumanCentric-Scenes-AI, a human-curated dataset of 296 AI-generated stimuli that depict people across four contexts in which people encounter AI-generated media. The Surveillance subset contains 85 AI-generated CCTV-style images. The Occupations subset contains 128 AI-generated occupational portraits in a balanced paired design covering 64 occupations with one female and one male version of each. The AI-edited subset contains 27 real photographs in which a region was altered or replaced by a generative model. The Video subset contains 56 AI-generated short videos. We curated the images and videos manually over multiple rounds to make them as photorealistic as possible. For each stimulus we release the file, the generation prompt where recorded, and per-stimulus metadata. We report depicted-gender distributions from an initial set of 134 generations without a specified gender and compare them with U.S. Bureau of Labor Statistics occupational gender shares. We report edit-region size and its association with zero-shot detection difficulty for the AI-edited subset. We report image-quality distributions for the Surveillance subset and subject-category coverage for the Video subset. HumanCentric-Scenes-AI is intended first for human-perception research, and also for calibration of AI-detection classifiers and bias analysis of generative models.