MLToolslabel · build · tune · ship

Fine-tuning

GPU Memory (VRAM) Calculator

Out-of-memory is discovered after the download, the environment and the first step. This calculator estimates memory for weights, gradients, optimizer states, activations and the KV cache from the numbers you know before you start, for full fine-tuning, LoRA and QLoRA, and for inference, and shows which common GPUs (8 to 80 GB) the run fits on.

Loading the tool…

How it works

01

Describe the model

Parameter count (presets for common sizes), layers, hidden size and heads if you want an activation estimate, precision of the weights.

02

Describe the run

Full, LoRA or QLoRA; LoRA rank and targets; optimizer; batch size, sequence length, gradient checkpointing.

03

Read the estimate

A breakdown per component, a total with a safety margin, and a fits/doesn't-fit table across GPUs.

Formats, limits and the numbers that matter

Formulas

Weights: params x bytes per parameter. Full fine-tune with AdamW in mixed precision: about 16 bytes per parameter (weights, grads, two optimizer states, fp32 master copy). LoRA: trainable params are rank x (in + out) per target matrix; base weights stay frozen at their precision. QLoRA: 4-bit base at about 0.5-0.6 bytes per parameter.

Activations

Roughly batch x sequence x hidden x layers x a constant (about 34 bytes per element without checkpointing, a fraction with it). This is the least certain term; the tool shows it separately.

KV cache at inference

2 x layers x heads x head_dim x bytes x tokens per sequence; for a 7B model at fp16 that is about 0.5 MB per token.

Common questions

Why does my run use more than the estimate?

CUDA context (0.5-1 GB), fragmentation, and framework buffers. The default 15 percent margin covers typical cases; long sequences and large vocabularies (logits are batch x seq x vocab in fp32) need more.

Does gradient checkpointing halve memory?

It cuts activation memory by roughly the square root of the layer count at about 30 percent more compute; the toggle applies that model.

What about multi-GPU?

Enter the number of GPUs; with FSDP or DeepSpeed ZeRO-3 weights, grads and optimizer states divide by that count, activations do not.

Next in the workflow

Training Time & Cost EstimatorEstimate hours and cloud cost from dataset size, epochs, throughput and GPU price, for LLM and vision runs.Model Size & Quantization CalculatorFile size and inference memory for a model at fp32, fp16, int8, int4 and GGUF quantisation levels, plus KV cache per token.Fine-tuning Script GeneratorGenerate a runnable training script and config for YOLO detection, Hugging Face LoRA SFT, or an image classifier from your paths and settings.