What Makes Fine-Tuning Slower Than The FLOPs Suggest?
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: What Makes Fine-Tuning Slower Than The FLOPs Suggest? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Hugging Face reports that a CatBoost model predicted runtime for 272 supervised fine-tuning jobs with a 1.71-hour mean absolute error in five-fold cross-validation. The best tested FLOP-based estimate had a 12.41-hour error, but the results come from one production GPU pool and do not show whether improved predictions save time or money in a live scheduler.

Hugging Face reports that a CatBoost model predicted the runtime of 272 supervised fine-tuning jobs with a 1.71-hour mean absolute error in five-fold cross-validation, compared with 12.41 hours for the best tested FLOP-based estimate. The result could help operators schedule scarce GPUs, but it comes from one corporate production pool and has not yet been tested in a live scheduling trial.

The runs accounted for about 7,382 GPU-hours on shared NVIDIA H200 machines. Job durations ranged from 1.6 to 172.4 hours and covered models up to 15 billion parameters, LoRA and full fine-tuning, different dataset sizes, and allocations of one, two or four GPUs. The pool contained between eight and 32 H200s across one to four nodes, with other training and inference workloads running at the same time.

The target was wall-clock time from GPU allocation to job completion; queue time was excluded. The predictor used 24 selected features, including submitted configuration and model metadata available before dispatch. Those inputs included evaluation cadence, checkpoint settings, per-GPU token volume and fine-tuning regime. A FLOP-based duration estimate was also included as one feature.

In stratified five-fold cross-validation, the CatBoost model scored an R² of 0.98 and a root mean squared error of 3.52 hours. Its median absolute error was 0.51 hours. The FLOP estimate performed best at an assumed 30% Model FLOPs Utilization, or MFU, with R² of 0.31 and a 12.41-hour mean absolute error. Its error rose to 13.01 hours at 45% MFU and 14.62 hours at 60%; the preview says testing MFU values from 25% to 65% did not resolve the gap.

At a glance
reportWhen: Study preview published; full paper for…
The developmentHugging Face shared a preview reporting that a CatBoost model predicted fine-tuning job durations more accurately than a tested FLOP-based estimate on 272 production runs.
AI Training · Briefing

Why Fine-Tuning Takes Longer Than FLOPs Say

Hugging Face tested runtime predictions on 272 production fine-tuning jobs. Configuration-aware estimates were more accurate in this dataset, but results from one production pool do not establish performance elsewhere.

1.71 hours
CatBoost mean absolute error
12.41 hours
Best tested FLOP baseline error
272
Production jobs evaluated
7,382
Approximate GPU-hours

What the estimates capture

The target is time from GPU allocation to job completion; queue time is excluded.

Compute

The training loop

FLOP estimates approximate compute using an assumed Model FLOPs Utilization, or MFU.

Configuration

The whole job

CatBoost uses job settings and model metadata, including evaluations, checkpointing and token volumes.

Shared systems

Other runtime costs

Setup, data handling, network and storage can add time alongside training.

Prediction error in the tested data

Lower error means predicted duration was closer to observed duration on average.

CatBoost1.71 hours
FLOP baseline, 30% MFU12.41 hours
FLOP baseline, 45% MFU13.01 hours
FLOP baseline, 60% MFU14.62 hours

What this could mean for GPU scheduling

More useful duration estimates could help teams plan scarce GPU capacity.

When jobs finish early
  • GPUs may sit idle until another job starts.
  • Operators can misjudge when capacity will be free.
When jobs run over
  • Other work can be delayed.
  • Delivery dates may shift.

How far the result reaches

The preview reports cross-validation on one corporate production environment.

1
Tested on one pool

Runs used shared NVIDIA H200 GPUs and varied configurations.

2
Generalization remains open

Accuracy on other clusters, software stacks and workload mixes is not established.

3
Operational impact remains unmeasured

The preview reports prediction error, not savings in GPU time or scheduling delays.

What to watchThe full paper is forthcoming; validation on independent runs and a live scheduler trial would show whether the result carries over.
The takeaway
Configuration-aware prediction beat the tested FLOP estimates on these jobs, but teams need broader validation before relying on the same accuracy for their own clusters.

More Reliable GPU Job Scheduling

Runtime estimates affect when cluster operators dispatch work and how they assign shared accelerators. If a job finishes sooner than expected, GPUs may sit idle until another job is ready; if it runs long, it can delay other jobs and shift delivery plans. A predictor that better reflects the full job lifecycle could give operators a more useful basis for planning than a compute estimate alone.

The reported error difference is large within this dataset, but it is not a measured reduction in idle GPU time, cost or delays. The study evaluates prediction accuracy, not the operational effects of deploying the model. Its practical value will depend on whether similar accuracy holds across other clusters and whether schedulers can act on the predictions effectively.

Products Worth Considering

Bose QuietComfort Ultra Wireless Noise Cancelling Earbuds, 2nd Gen, with 20W 2-Port Wall Charger

Bose QuietComfort Ultra Wireless Noise Cancelling Earbuds, 2nd Gen, with 20W 2-Port Wall Charger

Jackery Explorer 1000 v2 Portable Power Station, 1024Wh LiFePO4 Battery

Jackery Explorer 1000 v2 Portable Power Station, 1024Wh LiFePO4 Battery

As an affiliate, we earn on qualifying purchases.

Why FLOP Counts Miss Runtime

A common training-time estimate approximates dense Transformer compute as six times the parameter count and training-token count, then divides by GPU count, peak throughput and an assumed MFU. MFU represents the share of theoretical peak throughput achieved in practice; the preview describes typical values as ranging roughly from 30% to 60%.

That calculation estimates training compute, while Hugging Face’s target covers time after GPUs are allocated until the job finishes. Model setup and loading, tokenization, scheduled evaluation, checkpoint uploads and final storage can all add time. The preview says these tasks vary by configuration and can compete for network and storage resources on multi-tenant clusters. It also says researchers included evaluation tokens in the baseline comparison and tested a range of MFU assumptions.

“Everything is known before dispatch.”

— Hugging Face, in its study preview

How Far the Results Extend

The results come from 272 jobs in one corporate production environment. Cross-validation tests performance on held-out portions of that dataset; it does not establish accuracy on independently collected clusters, GPU types, software stacks or workload mixes. The preview also does not provide a live scheduler trial or report whether using the predictions reduced GPU idle periods, delays or costs.

The full paper has not yet been posted, and the preview gives no publication date. Further details on the dataset and evaluation design may appear there. It is also unclear how frequently the model would need retraining as workloads and shared-cluster conditions change, or whether performance would hold on runs collected in a later period.

Full Paper and External Testing

Hugging Face says a full version will be posted on arXiv, but has not specified when. That paper may give readers more information about feature selection, validation and the underlying production data.

To establish whether the approach generalizes, it would need testing on jobs from different time periods or environments, followed by a trial in an operational scheduler. Such results could show whether the low cross-validation error persists beyond the original GPU pool and whether more accurate estimates lead to fewer scheduling mismatches.

Key Questions

What did the Hugging Face model predict?

It predicted wall-clock runtime for supervised fine-tuning jobs, measured from GPU allocation until completion. Queue time was not included.

How did CatBoost compare with the FLOP estimate?

Across five-fold cross-validation on the reported dataset, CatBoost had a 1.71-hour mean absolute error. The best tested FLOP-based setting, at 30% MFU, had a 12.41-hour error.

Does this show that GPU costs or delays were reduced?

No. The preview reports prediction errors, not results from a live scheduling deployment or measured savings in GPU time, cost or delays.

Can the results be expected on other clusters?

That is not established. The evaluation used one production pool of shared NVIDIA H200 GPUs, so independent testing on other hardware and workloads is needed.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

The Impact Of AI On Our Decision Regarding Cursor And SpaceX Partnership

OpenAI announced its decision on how it will handle Cursor after its acquisition by SpaceX, impacting AI coding tools and developer workflows.

Explore The Top 10 AI Mini PCs For 2026

Discover the top 10 AI mini PCs for 2026, featuring powerful processors, expandability, and connectivity to handle demanding AI workloads.

Mobilisiert, nicht ausgegeben: Was von Europas €200-Milliarden-KI-Offensive übrig bleibt

Die EU plant, €200 Milliarden für KI zu mobilisieren, doch nur ein Bruchteil ist garantiert. Die tatsächliche Investitionskraft bleibt unklar.

2026 AI Automation Software Checklist For Business Teams

A 2026 checklist spans six AI software categories, but incomplete product names and unclear listings mean teams should verify details before buying.