Cost, Performance, and Data Platform Design

The bill for a dataset can include source fees, storage, scans, shuffles, network transfer, online replicas, backfills, labeling, review, incident response, and governance. How small a share the model itself can be was drawn on one page in 2015. D. Sculley and nine co-authors put the ML code in a small black box in the middle of the central figure of “Hidden Technical Debt in Machine Learning Systems”, and captioned it: “Only a small fraction of real-world ML systems is composed of the ML code, as shown by the small black box in the middle. The required surrounding infrastructure is vast and complex.” Later in the same paper they put a ratio on that picture. A mature system “might end up being (at most) 5% machine learning code and (at least) 95% glue code”.

That is not one team's impression of its own codebase. A 2022 survey in ACM Computing Surveys reviewed published deployment case studies and found Sculley's anti-patterns, “such as glue code or pipeline jungles”, to be “currently widespread in machine learning software”. If at most a twentieth of a mature system is the model, a budget that counts training and calls the rest overhead is measuring the black box. It is ignoring the diagram drawn around it.

Cost per successful release, cost per fresh feature, and time to recover from a bad dataset therefore reveal more than monthly warehouse spend. Human waiting and duplicated pipelines are part of the 95%, not an accounting nuisance outside it. Count them. Optimization should also preserve correctness evidence: removing lineage, snapshots, or validation to save money can raise the cost of the next incident.

Profiling the workload shape helps teams choose layout and platform boundaries. Each of the five shapes below has a different price, and for three of them the price is published.

Scan-bound work is billed by the byte. Google charges $6.25 per TiB of data processed for BigQuery on-demand queries, with the first 1 TiB per month free. The BigQuery pricing page puts it plainly: “By default, queries are billed using the on-demand (per TiB) pricing model, where you pay for the data scanned by your queries.” AWS bills Athena per terabyte scanned, and works the arithmetic through on its own “Amazon Athena pricing” page. A query over one column of a 3 TB uncompressed text table scans all 3 TB and costs $15. GZIP compression at 3:1 brings it to $5. Compression plus conversion to Apache Parquet lets Athena read only the one required column, scanning 0.25 TB: “This query would cost: $1.25. There is a 3x savings from compression and 4x savings for reading only one column.” That is a 12x reduction from layout alone. Nobody touched the query text.

Google published the scan-bound case in 2010. Dremel combined “multi-level execution trees and columnar data layout” to run “aggregation queries over trillion-row tables in seconds”. The same paper reports the system scaling “to thousands of CPUs and petabytes of data”, with thousands of users. Layout, not engine choice, made those queries interactive.

Shuffle-bound and small-file-bound work has been measured in production at two companies. LinkedIn's Spark clusters shuffle a few petabytes per day across tens of billions of blocks. The average block is only tens of kilobytes. LinkedIn's 2020 paper on Magnet, its push-based shuffle service, prices the delay that causes: “Around 15% of the total Spark computation resources on our clusters are wasted due to this latency.” Merging those blocks cut end-to-end production job runtime by nearly 30%. Riffle, a 2018 paper from Princeton and Facebook, reports the same failure mode at Facebook. Over 50% of daily batch analytics jobs there contain at least one shuffle. Request counts rise from 30K to 100M as average request size shrinks from 1.7MB to 50KB. Merging gave up to a 10x reduction in shuffle I/O requests and 40% faster job completion on clusters of over 1,000 executors. Neither team changed engine. Both changed block size.

Five shapes are worth profiling for. Scan-bound work: large column reads, repeated full-table profiles, broad historical rebuilds. Shuffle-bound work: joins, group-bys, deduplication, repartitioning, entity resolution. Small-file-bound work: metadata overhead, task scheduling, listing, inefficient object-store access. State-bound work: streaming windows, deduplication, feature materialization, long lateness retention. External-dependency-bound work: APIs, label providers, source databases, and network transfer dominating latency or cost.

A risk team changes how device sessions are grouped, and every training and monitoring dataset has to be restated. The scenario is a worked hypothetical. Every line of its bill is priced by the figures above. The rebuild is scan-bound at $6.25 per TiB unless the tables are already columnar. The regrouping is a shuffle, and its block sizes decide whether it costs like LinkedIn's clusters before merging or after.

The platform should remove work every team would otherwise repeat, without erasing domain ownership. Six questions decide it. Count the repeated pain: how many teams rebuild the same capability, and how often it causes incidents. Separate mechanism from meaning: centralize generic execution, leave domain semantics with accountable owners. Define the paved road: defaults, templates, and observability, without blocking justified exceptions. Price migration: adoption, dual running, support, and deprecation, not only implementation. Set platform SLIs: reliability, lead time, user success, cost, and escape rate to custom paths. Plan reversibility: portable data and contracts, so teams can exit or replace the service.

The last of the six is no longer a matter of taste in the EU. The EU Data Act makes exit portability a legal obligation rather than a negotiating point. It was published in the Official Journal on 22 December 2023 and has applied since 12 September 2025. Its Chapter VI governs switching between data processing services, and Article 29 phases out switching charges. During a transitional period from 11 January 2024, providers may recover only the costs directly linked to switching. Germany's Bundesnetzagentur, the national competent authority, states the cut-off in one line: “From 12 January 2027, no switching charges at all may be imposed (Article 29(1) of the Data Act).” The European Commission's own explainer says the same thing in its own words: “The Data Act will also entirely remove switching charges, including charges for data egress (i.e. charges for data transit), from 12 January 2027.” Bundesnetzagentur adds the Article 29(4) duty to disclose the reduced transitional charges before contracting — that is, before the customer is inside the platform, not when they try to leave it.

A reversibility plan with a statutory date on it changes the shape of step 4 as well. Migration is priced against a deadline that is already running, not against a hypothetical future decision to move.

Compare architectures when everything is running normally, at peak load, during a source outage, a backfill, a schema migration, and an incident recovery. The cheapest steady state may be expensive to change or explain.

One migration has a published price. TSB moved its corporate and customer data onto a new IT platform in April 2018. The data itself migrated successfully. The platform failed immediately anyway. All TSB branches and a significant proportion of its 5.2 million customers were affected, and the bank did not return to business-as-usual until December 2018. On 20 December 2022 the UK Financial Conduct Authority and the Prudential Regulation Authority jointly fined TSB Bank plc £48,650,000 — £29,750,000 by the FCA and £18,900,000 by the PRA, after a 30% settlement discount from an undiscounted £69,500,000. TSB paid a further £32.7m in customer redress on top of the penalties. Mark Steward at the FCA set out why: “The firm failed to plan for the IT migration properly, the governance of the project was insufficiently robust and the firm failed to take reasonable care to organise and control its affairs responsibly and effectively, with adequate risk management systems.” The PRA's own Final Notice to TSB Bank plc records a £27,000,000 penalty for breaches of PRA Fundamental Rules 2 and 6 between 16 December 2015 and 10 December 2018, reduced by 30% to £18,900,000. A correct data migration and a failed platform migration are not the same event. Only one of them was on the project plan.

Document unit economics: cost per million examples, per online lookup, per retained snapshot, per label, and per recovered incident. Use published units where they exist. $6.25 per TiB scanned on BigQuery on-demand is a real line item, and so is the 12x swing between a text table and a Parquet one. Include staff time and vendor constraints.

Object stores publish the unit that a cost model needs. Amazon documents “at least 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per partitioned Amazon S3 prefix”. There are “no limits to the number of prefixes in a bucket”, the same page adds. Ten prefixes read in parallel scale to “55,000 read requests per second”. Key design is therefore capacity planning, not just a naming convention.

Finish with a decision memo stating assumptions, sensitivity, rejected alternatives, and the conditions that would justify redesign. Cost numbers without a workload model age quickly. Eight months of remediation, £48,650,000 in fines and £32.7m in customer redress is what the model looks like when it is written after the fact.

Cost must be counted across building, backfills, serving, incidents, and the hours people spend coordinating. Four options recur, and each buys down one cost by taking on another. Local optimization reduces one job's runtime or storage footprint; it is easy to measure, and it can shift cost downstream or reduce observability. A reusable platform capability spreads fixed cost and improves consistency, but it can become a bottleneck and needs clear product ownership. A managed service reduces infrastructure burden while introducing lock-in and limits, with pricing that may scale nonlinearly. A manual process is flexible early and appropriate for rare exceptions, but it hides queue and coordination cost and is hard to reproduce.

The lock-in line in the managed-service column stopped being abstract in 2024. Both hyperscalers removed their exit data-transfer charges — the clearest public sign that egress pricing had been functioning as a lock-in cost. Google Cloud went first, on 11 January 2024, applying the change “to all customers globally”: “Starting today, Google Cloud customers who wish to stop using Google Cloud and migrate their data to another cloud provider and/or on premises, can take advantage of free network data transfer to migrate their data out of Google Cloud.” AWS followed on 5 March 2024: “That’s why, starting today, we’re waiving data transfer out to the internet (DTO) charges when you want to move outside of AWS.” The waiver sits on top of the standing 100 GB per month free allowance. It is granted as credits on request through AWS Support, and the account does not have to be closed. AWS states that it “follows the direction set by the European Data Act and is available to all AWS customers around the world and from any AWS Region”. AWS updated the post on 30 September 2025. Eligible customers now have 90 days to complete the move, and the previous “more than 100 GB per month” precondition for contacting Support was removed.

Both announcements arrived before the 12 January 2027 prohibition, and one of them names the regulation as its reason. An exit plan whose cost model still assumes egress charges is pricing a barrier that two providers have withdrawn and a statute is due to abolish. An exit plan that assumes portability is therefore free is ignoring the dual running, support, and re-validation that no waiver covers.