Transformer Parameter and FLOPs Estimator

Estimates parameter count and training FLOPs for decoder-only Transformers (like OPT/GPT). Shows formulas and per-component breakdown.

Batch Size

Sequence Length

Number of Layers

Hidden Size (d_model)

Vocabulary Size

Show FLOPs Breakdown

Estimates