AI Inference Energy Cost Calculator
Estimate the direct electricity footprint of AI inference serving
AI inference is often described through tokens, latency, and model quality, but the serving work also has a physical cost. GPUs operate for a measurable time, consume power while processing requests, and use electricity whose price and carbon intensity depend on the supplying grid. This calculator connects those quantities in one estimate. Enter a token workload, per-GPU serving speed, active GPU count, and GPU power draw to estimate direct runtime, electricity use, cost, and emissions without building a separate spreadsheet.
An AI inference energy estimate can support several decisions. A product team can size the electricity component of a projected launch. Infrastructure engineers can compare a high-throughput deployment with a lower-watt alternative. Sustainability teams can translate anticipated serving volume into a transparent operational carbon estimate. The same inputs are also useful for testing planning cases, such as a larger token volume, a different GPU count, a new electricity rate, or a grid with a different carbon intensity.
The AI inference calculation follows a visible chain rather than an opaque score. Tokens establish the work to be served. Throughput establishes the elapsed time. GPU runtime and watts establish direct electricity use. Electricity use then determines the price estimate and the grid-based emissions estimate. Keeping those links explicit makes it easier to check assumptions and explain why one scenario differs from another.
Choosing inputs for an AI inference energy estimate
Tokens to Process should represent the full token workload for the period you are modeling. For a large language model, that commonly includes prompt tokens and generated tokens across all requests, rather than generated output alone. For a batch run, use the tokens in that run; for a daily traffic plan, use the day-level total. The calculation uses the aggregate workload, so the planning period should be clear before comparing results.
Throughput (tokens/sec per GPU) is specifically a per-GPU value, not total cluster throughput. If a four-GPU benchmark reports 480 tokens per second in aggregate, the corresponding per-GPU input is 120 tokens per second. Entering the aggregate number in this field would overstate serving speed by a factor of four. Use a benchmark that resembles the intended model, batch size, sequence length, quantization, and precision whenever possible.
Number of GPUs is the number of devices actively serving the token workload. It primarily changes the AI inference completion time: more GPUs increase total throughput when per-GPU throughput remains constant. In the calculator's idealized direct-energy model, adding GPUs does not necessarily reduce kWh because increased total wattage can offset shorter runtime. Communication overhead, utilization, memory pressure, and batching may make real-world scaling depart from that baseline.
Power per GPU should approximate sustained inference power, not automatically the card's advertised maximum. Monitoring data from a comparable deployment is most useful. A rated maximum can serve as a conservative upper-bound assumption, while idle power would understate serving energy. Match the wattage as closely as possible to the operating point that produced the throughput figure.
Electricity Price and Grid Carbon Intensity convert direct AI inference energy into financial and emissions estimates. Price is entered in dollars per kilowatt-hour, and carbon intensity is entered in grams of CO₂e per kilowatt-hour. Where rates or grid conditions vary by location or hour, comparing several input cases is more informative than treating one value as universal.
AI inference runtime, energy, cost, and CO₂e calculations
This AI inference calculator reports four connected values. Runtime in seconds equals the token workload divided by total serving throughput. Direct GPU energy in kilowatt-hours comes from runtime, active GPU count, and watts per GPU. The entered electricity rate converts kWh to cost. Finally, the entered grid intensity converts kWh to kilograms of CO₂e. Cooling, networking, storage, CPUs, and facility overhead are outside these direct GPU results.
The page applies the following direct AI inference relationships:
These equations also show an important property of the model: when per-GPU throughput and power remain unchanged, GPU count cancels out of the direct-energy calculation. GPU count still changes runtime, but the practical energy result can change when actual throughput or actual wattage changes with scale.
Worked AI inference energy example
Suppose an inference workload contains 10,000,000 tokens. A benchmark provides 120 tokens per second per GPU, the deployment uses 4 GPUs, and each GPU averages 300 watts during serving. With an electricity price of $0.12 per kWh and grid intensity of 400 g CO₂e per kWh, total throughput is 480 tokens per second. Runtime is 10,000,000 ÷ 480 = 20,833.33 seconds, or about 5.79 hours. Direct GPU energy is 300 × 4 × 20,833.33 ÷ 3,600,000 = 6.9444 kWh. The electricity estimate is 6.9444 × 0.12, or about $0.83, and the emissions estimate is 6.9444 × 400 ÷ 1000, or about 2.78 kg CO₂e.
This inference-serving example provides a useful check on GPU scaling. Holding per-GPU speed and per-GPU power constant, doubling the GPU count halves runtime while doubling instantaneous GPU power, leaving direct kWh approximately unchanged. Actual deployments may not cancel so neatly, but faster completion should not be assumed to mean lower electricity use without checking measured throughput and power.
AI inference GPU-count scaling comparison
This AI inference comparison holds workload size, per-GPU throughput, and watts per GPU constant. It illustrates why additional GPUs chiefly shorten completion time in the calculator's linear model, while direct GPU kWh remains the same.
| Scenario | GPUs | Estimated time | Energy used | Electricity cost | Emissions | Interpretation |
|---|---|---|---|---|---|---|
| Smaller cluster | 2 | 11.57 hours | 6.9444 kWh | $0.83 | 2.778 kg CO₂e | Slower completion, but ideal direct energy remains similar because lower total power runs for longer. |
| Baseline | 4 | 5.79 hours | 6.9444 kWh | $0.83 | 2.778 kg CO₂e | A reference deployment for comparing alternative inference configurations. |
| Larger cluster | 8 | 2.89 hours | 6.9444 kWh | $0.83 | 2.778 kg CO₂e | Faster turnaround, without lower ideal energy unless real scaling or power behavior improves. |
Reading AI inference energy calculator results
After calculation, the result panel separates four AI inference planning measures. Estimated time is the period for which the active GPUs serve the entered workload. Energy used is direct GPU electricity based on the entered wattage. Electricity cost applies the selected rate to that direct energy. Carbon emissions applies the selected grid intensity; it is not an attribute of a particular GPU model. If a value appears unexpected, trace it backward through runtime, power, price, and carbon inputs.
A concise inference-estimate check can catch common entry mistakes. Confirm every input unit, verify that throughput is per GPU rather than an aggregate figure, and ensure that the token total includes the intended prompt and output work. Relationship checks are also useful: halving per-GPU throughput doubles runtime; doubling electricity price doubles cost; and changing only carbon intensity changes only emissions.
AI inference energy assumptions and exclusions
This AI inference energy model is deliberately compact, so its assumptions should be understood before using the result for a detailed operational inventory:
- Constant throughput: the model assumes tokens per second per GPU remains steady throughout the workload. Actual serving throughput can vary with sequence length, context size, batching, and traffic shape.
- Constant power draw: entered GPU wattage is treated as constant for the run, although utilization, memory activity, and clocks can change actual draw.
- Direct GPU energy only: CPUs, RAM, storage, networking, cooling, and facility power overhead are not added automatically.
- Linear scaling baseline: GPU-count comparisons assume constant per-GPU throughput. Communication overhead or batching changes can make real scaling weaker or stronger.
- Single grid-intensity input: carbon intensity is modeled as one value even though it can vary by time and location.
Those boundaries make the calculator most useful as a transparent AI inference planning baseline. A broader total-cost, facility-energy, or lifecycle analysis can add its own factors to the direct GPU kWh shown here.
Using an AI inference energy estimate responsibly
For AI inference planning with uncertain benchmarks, calculate conservative, expected, and optimistic cases rather than relying on one precise-looking output. Cloud users can compare the direct electricity estimate with billed spend while recognizing that service prices include costs beyond electricity. Operators of their own hardware may apply a separately chosen overhead factor when they need to account for cooling and power distribution. For reporting use, record the source and period for every token, throughput, wattage, price, and carbon input.
Use this calculator as an AI serving decision aid, not as a complete measurement system. Its value is that it makes the drivers visible: token volume, serving rate, GPU count, power draw, electricity price, and grid carbon intensity. Once those assumptions are explicit, runtime, cost, and emissions tradeoffs are easier to evaluate.
AI inference energy estimate questions
Should AI inference use average or peak GPU power?
Use the best available average power draw during the inference workload. Peak wattage can be useful for a conservative case, but it may overstate direct GPU energy if the workload does not sustain that level. The throughput benchmark and power input should describe as nearly as possible the same operating condition.
What if AI inference throughput changes with prompt length or batch size?
Run multiple AI inference scenarios rather than forcing one throughput value to represent unlike traffic. Short prompts, long contexts, small batches, and large batches can behave differently. If the traffic mix is known, use a throughput figure representative of that mix; otherwise, bracket the estimate with slower and faster cases.
Does a lower-emissions grid always lower AI inference electricity cost?
No. Electricity price and grid carbon intensity are independent inputs in this calculator. A cleaner grid can have a higher electricity rate, and a cheaper grid can have a higher emissions intensity. Viewing cost and CO₂e separately helps compare the same inference workload across locations.
Copy status messages will appear here after you use the copy button.
Optional AI inference load-balancing mini-game
This arcade mini-game illustrates the serving tradeoff behind the AI inference calculator. Keep the request queue near the green target zone while switching among Eco, Balanced, and Turbo GPU modes. Eco uses less power but clears requests more slowly; Turbo clears bursts quickly but consumes more power. High scores reward matching throughput to demand instead of using maximum power continuously.