Inference workload

RAG over internal knowledge, multi-turn Q&A, and audit logging.

Model Quantization

Hardware profile

Custom label Memory / GPU (GB)

Tokens/sec/GPU Cold read GB/s Accelerated read GB/s

GPU count 8

8 x H100 80GB | 640 GB HBM

Concurrent users 2,000 Turns per session 20

Sessions per user/day Batch size

2.45B tokens/day

Prefill

System prompt User input Context/RAG

Decode

Max output tokens GPU hourly rate

KV cache estimate pending

Tenants Corpus size (TB) Growth %/mo

System logs (weeks) User data (weeks)

Storage estimate pending

Results for

AI Factory Cost Calculator

Configure your deployment, then run a simulation.

Ready

Memory fit -- Awaiting simulation

Annual GPU cost -- Right-sized estimate

3-year data -- Knowledge + usage + logs

Cost per 1M tokens -- Steady-state inference

Configure your deployment to see results Select inputs on the left and click Run Simulation.