Token Economics Calculator

We created this Tokenomics calculator to help companies deploying LLMs understand the costs and energy involved, and compare different hardware and models.

Start here

Model Selection

Included models

  • Llama 3 70B - 1k_1k
  • Llama 3 70B - MLPerf®
  • Llama 3 405B - 1k_1k
  • Llama 3 405B - MLPerf®

Parameters (millions)

Quantization (model parameter precision)

  • 1 byte - INT8, FP8...
  • 2 bytes - FP16, BF16...

Quantization (math/KV-cache precision)

  • 1 byte - INT8, FP8...
  • 2 bytes - FP16, BF16...

Hardware Selection

Reference Hardware

  • AMD MI325X
  • Cerebras CS-3d-Matrix
  • Corsair
  • Furiosa RNG
  • GroqRack
  • Nvidia L40S
  • Nvidia DGX H200
  • Nvidia NVL72
  • Nvidia RTX Pro 6000
  • Rebellions REBEL-Quad
  • SambaNova SN40L
  • Tenstorrent Galaxy Rack

Memory per Chip (GB)

Chips per Rack

Rack Power (kW)

Rack Cost (USD)

Concurrent Users (est)

Operating Parameters

Depreciation (years)

Weighted Average Cost of Capital (%)

Colocation Cost ($/kW per month)

Learn about colocation cost

Energy Cost ($/kWh)

Learn about US energy cost
Learn about EU energy cost

Colo/Datacenter PUE

Expected System Utilization (%)

Default Operating Parameters

Results

Waiting for inputs

0 Rack Comparison

Total Tokens per Second

Waiting for inputs

Cost per million total tokens

Waiting for inputs

Total Tokens per USD

Waiting for inputs

Total Tokens per kWh

Waiting for inputs

Total Expenses per Month

Waiting for inputs

Total Rack Cost (USD)

Waiting for inputs

0 Racks Needed

Reference Hardware Cost Breakdown

Learn more about why Tensordyne performs so well

Disclaimer

The GenAI Token Economics Calculator was developed to help users consider how variation in certain parameters may impact the performance, costs, and energy for deploying LLMs using different hardware and models.

Tensordyne has attempted to provide estimates for a limited number of substantially similar scenarios by compiling information from publicly available sources including MLCommons and normalizing the data. Where public information is limited or missing for certain parameters, Tensordyne has included its best estimates, e.g. concurrent users supported, quantization, etc.

This calculator is for instructional purposes only. The calculator makes many assumptions and relies on publicly available data that may be incorrect or outdated, which may introduce significant error into results. Tensordyne does not make factual representations about any of the calculator’s inputs or outputs. Users are encouraged to observe the relative sensitivity of outputs to changes in inputs.