LLM Training Cost Estimator

• The core estimate uses the common 6 x parameters x training tokens FLOPs approximation.
• GPU utilization or MFU is a major cost driver; lower utilization means more GPU hours.
• Provider prices change frequently and differ by region, reservation, commitment, and availability.
• The calculator estimates pre-training-like compute, not full R&D cost, hyperparameter search, RLHF, data curation, or evaluation.
• Fine-tuning, LoRA, QLoRA, and RAG are often far cheaper than training a model from scratch.

Use Cases

Frequently Asked Questions

How does this estimate LLM training cost?

It estimates training FLOPs from parameters and tokens, converts FLOPs to GPU hours using selected GPU throughput and utilization, then multiplies by approximate GPU-hour rates plus overhead assumptions.

Are the cloud GPU prices guaranteed?

No. They are sample planning rates. GPU pricing changes often and varies by provider, region, commitment, availability, and contract.

What is MFU or GPU utilization?

MFU measures how much of the GPU's theoretical compute your training run actually uses. Poor data loading, communication overhead, and inefficient kernels reduce MFU and increase cost.

Does this include fine-tuning cost?

The main calculator is aimed at pre-training-style estimates. Fine-tuning existing models is usually much cheaper and may require separate assumptions.

Does this include post-training and evaluation?

No. It does not fully model RLHF, DPO, evaluation, red-teaming, data curation, engineering labor, or failed research runs.