FP16 and BF16

Also known as: bfloat16, half precision

16-bit formats and the standard for training and running LLMs. BF16 keeps FP32's range with less precision, which makes training more stable; it needs Ampere, CDNA 2 or newer.

GPUs with FP16 and BF16, best value first

Ranked by $ per PFLOP-hour of dense FP16 / BF16 at each GPU's lowest on-demand price.

1Instinct MI355X288 GB$2.59/h$1.03/PFLOP·h
2Instinct MI300X192 GB$1.85/h$1.42/PFLOP·h
3B200 SXM180 GB$3.20/h$1.42/PFLOP·h
4RTX A500024 GB$0.16/h$1.44/PFLOP·h
5Instinct MI325X256 GB$2.00/h$1.53/PFLOP·h
6RTX 309024 GB$0.11/h$1.57/PFLOP·h
7RTX 409024 GB$0.29/h$1.78/PFLOP·h
8RTX 6000 Ada48 GB$0.65/h$1.79/PFLOP·h

Compare every GPU →

Related terms

PrecisionHow many bits each number uses. Fewer bits mean more throughput and less memory, at some cost in accuracy. The same GPU has a different peak for each precision.
FP88-bit floating point for training and inference, with roughly twice the throughput of FP16. Native on Hopper, Ada, Blackwell and AMD CDNA 3 and newer.
TF32NVIDIA's 19-bit format that runs FP32 matrix math on tensor cores. PyTorch can use it for FP32 matmuls on Ampere and newer.

Compute

FLOPS, TFLOPS, PFLOPSFloating-point operations per second: how much arithmetic a chip can do. 1 TFLOPS is a trillion per second, 1 PFLOPS (PetaFLOPS) is a thousand TFLOPS. PetaPrice shows all compute in PFLOPS.
Dense vs. sparse FLOPSVendors often quote tensor throughput with 2:4 structured sparsity, which doubles the headline number. Almost no training or inference uses it, so PetaPrice always shows dense figures.
$ per PFLOP-hourThe price of one GPU-hour divided by the GPU's dense peak PFLOPS at a chosen precision. It shows how much compute a dollar buys, so fast expensive GPUs and slow cheap ones can be compared directly. Lower is better.
FP44-bit floating point, used for quantized inference. Only Blackwell and AMD CDNA 4 run it natively.
INT8 and TOPS8-bit integers for quantized inference. Throughput is counted in TOPS (trillions of operations per second) rather than FLOPS.
Tensor cores / matrix coresUnits that multiply small matrices in one step, where nearly all of a GPU's AI throughput comes from. NVIDIA calls them tensor cores, AMD matrix cores.
MFU (model FLOPS utilization)The share of peak FLOPS a real workload achieves. Large training runs typically reach 30–60%, so datasheet peaks are for comparing GPUs, not for predicting run time.

← All terms

…