CCQ: A Multi-State Child Care Quality Dataset to Support AI for Children's Health Research

AI summary

Overview

Research area: Applied machine learning for children's health, with a focus on dataset construction, LLM-based data curation, and benchmarking of predictive models on child care quality rating (QR) prediction. The paper's CCS categories are Applied computing (Health informatics), Computing methodologies (Machine learning), and Information systems (Data cleaning).

Technical level: Intermediate. The paper combines dataset engineering, LLM-based curation pipeline design, and standard supervised learning benchmarks (ordinal classification, cross-domain transfer), so it is accessible to readers with working knowledge of machine learning evaluation but does not require deep domain expertise in early childhood policy.

Scope in one sentence: The paper introduces CCQ, a de-identified, multi-state, provider-level child care quality dataset built with an LLM-assisted curation pipeline, and benchmarks a representative set of machine learning and language models on within-state and cross-state quality rating prediction.

What This Paper Is About

Research on how child care quality affects children's health and development has been limited by datasets that are fragmented across states, published in incompatible schemas, and bound by privacy constraints, leaving no unified provider-level resource that links quality ratings to the features that shape quality. The authors build CCQ to fill that gap, assembling 59,372 child care provider records from 12 U.S. states and curating them into two AI-ready releases. They then ask how well current models can predict a provider's quality rating, both when trained on a state's own rated providers and when a state with few or no ratings must borrow models trained elsewhere.

Key Contributions

1. A new multi-state dataset. The authors construct CCQ, a provider-level dataset that combines child care quality ratings with provider features across 12 U.S. states, totaling 59,372 provider records with 29,073 rated providers. They describe it as the first publicly available, multi-state, AI-ready dataset of real-world child care quality ratings.

2. An automated LLM-based curation pipeline. The pipeline leverages LLMs to collect, anonymize, and clean raw state-specific records into two releases: a cleaned textual release (raw) and a strictly numerical preprocessed release (preprocessed). The pipeline is adapted from a hand-implemented Georgia reference and is designed to reduce the time and labor that manual curation of multi-dimensional child care data typically requires.

3. Human verification of privacy, leakage, and values. Every output was checked by hand for private information and data leakage, and values were spot-checked against the source portals. Two annotators judged 96.8% and 97.6% of released values correct, or 1,438 of 1,480 combined checks (97.2%).

4. A benchmark of 10 models on two prediction tasks. The authors benchmark 10 representative models spanning classical machine learning classifiers, tabular foundation models, and finetuned language models on within-state and cross-state QR prediction, then use feature attribution to identify which provider attributes drive predictions and measure how cross-state transferability changes as more target-state labels become available.

Main Findings

• Within-state prediction favors tabular classifiers on the preprocessed tables. Among the model families tested, tabular classifiers operating on the preprocessed (strictly numerical) release perform best within a state.

• Zero-shot cross-state transfer is near chance. When a model trained on other states is applied to a target state with no target-state labels, performance is near chance, reflecting the mismatch in feature schemas and rating systems across states.

• A modest amount of target-state supervision recovers most of the within-state performance. Adding a small amount of labeled data from the target state restores most of the accuracy achievable by within-state training. The cross-state protocol evaluates supervision levels from zero-shot up to 80% of the target state's providers.

• Pretraining on other states helps finetuned language models. For language models that are finetuned, pretraining on data from other states benefits prediction accuracy, even though schemas differ.

• Coverage of quality ratings is low, motivating prediction. Only 29,073 of 59,372 providers (49.0%) carry an assigned QR score. The rated share of a state's providers runs from 7.9% in California to 99.9% in Oklahoma. The paper also cites that, among QRIS reporting participation in 2017, the median system rated only 60% of licensed centers and 25% of family child care programs, and that New Jersey rated only 1% of its centers.

• Rating distributions are skewed and unevenly so across states. Every state's distribution except Oklahoma's is skewed, but not in a shared direction. Five states concentrate at the bottom of their scale: Kentucky and Maryland place 61.23% and 43.60% of providers at one star, and Nebraska, South Carolina, and Colorado place 76.55%, 75.31%, and 63.51% of providers in their bottom two levels. Two states concentrate at the top: California assigns four- and five-star ratings to 87.36% of providers, and North Carolina assigns five stars to 53.93%. Four states concentrate in the middle: Washington places 80.24% at exactly three stars, Wisconsin places 52.08% at two stars, Montana places 65.19% at two or three stars, and Georgia places 65.86% at two stars on its three-level scale. Some classes are extremely small: Washington, Wisconsin, Montana, and California report 0, 2, 6, and 6 providers respectively at the one-star level.

• Feature schemas vary widely between states. Schemas range from 10 attributes in South Carolina to 203 in Georgia, with a median of 62.5 attributes. Provider counts per state range from 181 rated providers in Montana to 5,005 in Maryland.

• Feature missingness also varies widely. In the raw release, South Carolina is nearly complete (only capacity missing, at 5.81%), followed by Nebraska (5.93%) and North Carolina (20.75%), then Kentucky (38.66%), Colorado (39.12%), and Georgia (55.37%). Montana (88.95%), Washington (93.88%), Wisconsin (94.53%), California (95.75%), Maryland (98.38%), and Oklahoma (99.02%) have nearly empty columns for their median field. Maryland shows the sharpest split: only one of its 30 feature columns is fully populated, nine more are missing for at most 15% of providers, and 19 of the remaining 20 are empty for more than 86%.

• The code-writing curation agent was adopted over the per-sample agent. Two zero-shot agents were tested on twenty curation tasks grouped into five types. On column removal the per-sample agent (Qwen3-8B, "E2E") reached 97.12 accuracy and the code-writing agent (Qwen3-Coder-30B, "Coder") 99.28. On de-identification the E2E agent reached 35.2 ± 42.8 and the Coder agent 100.0 ± 0.0. On text-to-numeric cleaning the E2E agent reached 93.3 ± 13.4 and the Coder agent 99.98 ± 0.02. On text-to-multi-label the E2E agent reached 97.2 ± 3.8 and the Coder agent 88.9 ± 22.1. On compound field splitting the E2E agent reached 95.8 ± 3.0 and the Coder agent 90.2 ± 11.9. The authors state the Coder agent's accuracy is comparable to the per-sample agent except on rare edge cases, at a fraction of the cost.

• Verification agreement is high but not perfect. Pooled across states, Cohen's kappa (× 100) is 98.1 for quality rating, 99.4 for provider type, 95.9 for licensed capacity, and 99.4 for ages served. Agreement is lowest for licensed capacity in Kentucky (68.0). The two annotators agreed with each other on 730 of 740 values, and 16 values (2.2%) were flagged by both. The authors attribute the mismatches to records updated between collection and verification, but state that they did not explicitly determine the cause of each mismatch.

• Benchmark result tables are not reported in the available content. The specific per-model accuracy, QWK, and balanced accuracy values for the benchmark, and the feature attribution results identifying which provider attributes drive predictions, are not included in the paper content provided here.

Methodology in Plain English

The authors started with the observation that every U.S. state runs its own Quality Rating and Improvement System (QRIS), and each publishes its findings in its own format. There is no federal registry and no common schema, which is why no national provider-level dataset existed.

Their construction pipeline has four steps: collection, anonymization, cleaning, and verification. Georgia was chosen as the reference state because it has the most published fields of the 12 states (203) and publishes them on easily accessible portals. The authors hand-wrote the collection pipeline for Georgia, then prompted a coding agent (Claude) to adapt that reference to each remaining state. For each state they collect a single table where rows are providers and the first two columns are always the provider identifier and the quality rating.

Cleaning produces two releases. The raw release applies minimal preprocessing and preserves the tabular text of source records so records can be fed as natural language into text-based models. The preprocessed release is strictly numerical for out-of-the-box analysis. Both releases remove columns that identify a provider or person (names, addresses, emails, phone numbers, websites), free-text narratives such as inspection, complaint, and violation reports, and columns that leak the rating (such as award dates or alternate formats of rating scores). Coarse geography, county, and region are retained. Missing values are left empty in raw and encoded explicitly in preprocessed. Unrated providers are removed.

To choose between two zero-shot LLM agents for anonymization and cleaning, the authors tested a per-sample agent (Qwen3-8B, prompted once per record) and a code-writing agent (Qwen3-Coder-30B, prompted to write one reusable cleaning function per field) against the Georgia ground truth on twenty tasks. They adopted the code-writing agent and included the Georgia pipeline functions as an in-context code example in every prompt; no provider record from any state is ever placed in the prompt. No private information was exposed to the proprietary model, because anonymization was performed locally with open-source models beforehand and sensitive columns were removed by hand before any cleaning script was generated.

The benchmark has two tasks mirroring deployment. Within-state prediction trains on a state's own rated providers and predicts ratings for the rest, evaluated with stratified five-fold cross-validation and scored with quadratic weighted kappa (ordinal-aware) and balanced accuracy (to account for label imbalance). Cross-state prediction uses a leave-one-state-out protocol: each state is the target once and the remaining states are pooled as the source training set, with a stratified 20% target test set held out and fixed across all supervision levels, and stratified draws of p% of the target state's providers ranging from zero-shot up to 80%. Because the preprocessed schemas differ across states, that release is used only for within-state prediction.

Why This Matters

Impact on research. Child care quality research has been constrained by fragmented, non-research-friendly, and privacy-bound datasets, and prior predictive modeling studies each fit and evaluated models on a single program with a limited number of labeled providers. CCQ provides a pooled resource across 12 states and defines the cross-state transfer setting that single-system studies could not examine, where a model must handle feature schema mismatch, label space mismatch, and distribution shift at the same time. It also supplies reference baselines for both QR prediction tasks and shows where current models break down under realistic, heterogeneous conditions.

Real-world applications:

• Helping state agencies estimate ratings for unrated providers. Since only 49.0% of providers in CCQ carry a rating, and QRIS participation is voluntary in most states, accurate within-state prediction could assign approximate ratings to unrated providers, lower the cost of rating, and extend ratings to more providers.
• Improving family decision-making. QRIS ratings are intended to inform parents' choices, but coverage gaps mean the provider a family is considering may simply have no rating. More complete coverage would make published ratings more useful.
• Supporting subsidy and quality-improvement policy. States use QRIS to help providers improve care quality, and many pay higher subsidy rates to programs with higher ratings; more complete coverage would make these efforts more effective.
• Enabling health and development research at scale. Higher-quality care has been associated with stronger cognitive and language skills by age three, better cognitive and social development through second grade, and higher academic achievement and less externalizing behavior at age 15 (Vandell et al., 2010). The paper cites the NICHD Study of Early Child Care and Youth Development, which followed 1,364 children from birth and directly observed their child care settings, and NIH ECHO's work on early-life environmental exposures, as lines of research that a larger linked dataset could support.

Industry relevance. The pipeline itself is a demonstration of LLM-assisted data curation for heterogeneous, privacy-sensitive public records, showing that a code-writing agent can replace hand-written, domain-specific preprocessing rules that are otherwise impractical to maintain. The release model is gated: the dataset is hosted on Hugging Face as a gated dataset requiring a data use agreement, a stated purpose, and acknowledgment of the non-commercial restriction, under a CC-BY-NC-SA 4.0 license. Code for curation and for reproducing the benchmarks is available on GitHub.

Future Directions

• Closing the cross-state gap more efficiently. Zero-shot transfer is near chance and a modest amount of target-state supervision recovers most within-state performance. How little target-state labeling is actually needed, and whether better schema alignment or representation transfer can reduce that requirement, remains open.

• Improving the curation pipeline at scale. The code-writing agent matched or beat the per-sample agent on most task types except text-to-multi-label and compound field splitting, and verification flagged 16 values (2.2%) that both annotators judged incorrect. The authors did not explicitly determine the cause of each mismatch, leaving room for more systematic error analysis and improved cleaning.

• Extending coverage beyond 12 states and beyond QRIS. The paper documents that no unified, multi-state provider-level QRIS dataset was previously available. Expanding to more states, and to sources such as the Canadian province of Alberta's inspection records or international registers, would further test generalization.

• Handling extreme class imbalance and sparse features. Some rating levels hold only a handful of providers (Washington, Wisconsin, Montana, and California report 0, 2, 6, and 6 providers at one star), and six states have nearly empty columns for their median field. Methods robust to extremely small classes and heavily missing features are a natural next step.

Target Audience

This paper is most useful to machine learning researchers working on tabular data, domain generalization, and cross-domain transfer; to health informatics and applied data science researchers studying children's health and early childhood development; to practitioners building LLM-based data curation and anonymization pipelines for heterogeneous public records; and to policy analysts, state agency staff, and early care and education researchers who want provider-level quality data across multiple rating systems. Readers primarily interested in the benchmark results should note that the detailed per-model scores and feature attribution results are not present in the excerpt summarized here.

Authors’ abstract

High-quality child care in early life is a critical determinant of children's growth and development. Research on child care quality has been constrained by fragmented, non-research-friendly, and privacy-bound datasets. We present CCQ (Child Care Quality), a large-scale, de-identified dataset for applied data science research at the intersection of AI and early childhood health. CCQ integrates 59,372 child care provider records across 12 U.S. states, covering diverse provider types as well as data schemas. To ensure research utility while protecting privacy, we implement an automated, LLM-based curation pipeline that anonymizes, cleans, and standardizes raw state records into two complementary releases: a cleaned textual release and a fully preprocessed tabular release. We also benchmark traditional machine learning models, tabular foundation models, and language models on quality rating prediction and important features analytics. Within a state, tabular classifiers on the preprocessed tables perform best. Across states, zero-shot transfer is near chance, but modest target-state supervision recovers most of the within-state performance, and pretraining on other states benefits finetuned language models. We release both datasets with all code to accelerate AI-driven research on child care quality and ultimately improve children's health and development.