Will another graphics card actually help?
LLM Calculator compares hardware configurations for generating text locally. It separates two questions: will the model fit in memory, and what limits the speed of generating the next token? More combined VRAM can answer the first while leaving the second unchanged.
The calculator runs as a static JavaScript page in the browser. It does not run a model or require an account, inference server or API key.
Inputs and outputs
Start with model parameters: parameter count, quantized storage size, layers and KV cache requirements. For MoE models, parameters stored in memory are distinguished from parameters active during token generation.
The editor opens with fully editable custom values. Every GPU has independent settings, and all parameters remain editable after loading an example. Presets are an optional shortcut; you can save, export and share a configuration built entirely from your own values.
Then describe each GPU: VRAM capacity, memory bandwidth, effective compute and its PCIe connection. CPU and RAM settings describe offload capacity when part of the model does not fit on the GPUs.
The output estimates tokens per second, per-token latency, and memory, compute and transfer constraints. Each GPU has its own VRAM reservation, and CPU offload is limited by usable RAM. If complete layers do not fit, the calculator reports the missing capacity instead of producing a throughput number.
Your model and its context budget
You can start with the Llama 3.1 8B/70B architecture presets or illustrative Dense/MoE examples. Weight profiles include a stated storage allowance for quantization metadata. Independent hardware examples configure one or two GPUs or a CPU-only setup; their bandwidth and compute values remain editable and are not measured benchmarks.
The KV cache planner uses the number of layers, KV heads, head dimension, context tokens and cache precision. It distinguishes decimal GB from GiB. For example, 32 layers, 8 KV heads, a head dimension of 128 and 8192 tokens at FP16 require 1 GiB, or about 1.074 GB, for one sequence’s key/value data.
A sensitivity table compares contexts from 2K to 128K tokens on the current model and machine. It recalculates cache storage, estimated decode speed and placement at each point without replacing the active inputs. Applying a cache preview is an explicit action. The memory table then shows weights, cache, usable capacity and free memory for each device.
How multiple GPUs are modeled
The computation model assigns consecutive blocks of layers to individual cards. During token generation, activations pass through this GPU chain, including transfer costs between stages. Offload adds CPU and RAM bandwidth constraints.
The live visualization shows the exact layer ranges in execution order, the weights and KV cache stored on each device, and activation payloads crossing each link. Step controls expose processing and transfer stages, their bottlenecks and cumulative elapsed time. It also makes the distinction visible: weights and KV cache stay with their assigned layers while activations move between blocks.
This distinction matters: layer splitting differs from splitting the tensors of one layer across several cards. Communication specific to tensor parallelism is not fully modeled here. The calculator’s built-in guide explains these assumptions.
Using the estimate
A practical comparison keeps the model and quantization fixed, then tries one GPU, two GPUs, and a configuration with some layers in RAM. It shows whether extra VRAM removes offload and whether the connection between cards becomes the next constraint.
Up to six named configurations can be saved and compared in the browser. Loading one restores its full numeric configuration. JSON import/export and shareable links make a setup reproducible; a downloadable report includes the current results, per-device memory placement, assumptions and JSON inputs. No account is required.
The result is an estimate rather than a measurement of a particular computer. Effective bandwidth, compute and KV cache size can be calibrated against your benchmarks. The inference engine, drivers, quantization format and connection topology affect actual performance.
The application runs at llm-calculator.piotrzmudadev.com as a separate Cloudflare Worker with static assets. The GitHub repository remains public and contains the complete source, README, methodology and tests. Runtime calculations run locally in the browser without external scripts or API calls.