From token throughput to billable revenue

1. Read the denominator before the headline

“Tokens per second” is incomplete without a definition. It may describe output generation, input processing or a combination. The reported system may be one GPU, an eight-GPU server or a much larger cluster. A per-user response speed is different from aggregate throughput across concurrent requests.

An output-token result cannot be divided into input and output volumes by assuming a convenient ratio. Input length is a workload parameter; it must come from the actual requests. Likewise, dividing a cluster score by its GPU count produces an average for that tested configuration, not a promise of linear scaling down to one GPU.

MLPerf specifies models, datasets, quality requirements and scenarios. Its LLM rules include time to first token and time per output token; Server and Offline results answer different operating questions.^[1] Nebius's own MLPerf v6.1 report, for example, distinguishes LLM tokens per second from multimodal query or sample rates and identifies the system configuration.^[2] Those labels are part of the result.

Before comparing two systems, record the exact model and version, precision, accelerator count, input/output lengths, concurrency, caching policy and latency target. A faster result under a looser response-time target may be useful for batch work while missing an interactive product's needs.

2. Count billable categories separately

For one measurement window, let H be the total allocated GPU-hours for the defined service. Count every allocated GPU across that window, including idle time and any prefill/decode pools included in the service boundary. If the allocation changes, sum GPU count × elapsed hours for each interval.

Let Tᵢ be the number of billable tokens in category i, and pᵢ the applicable price in dollars per million tokens. The token charge per allocated GPU-hour is:

Use non-overlapping billing categories. Uncached input, cached input and output are a common starting point. Some providers also charge for cache writes or distinguish retention periods. OpenAI's caching documentation exposes separate cached-token and cache-write counters; its billing example treats those categories separately.^[3] Do not count a cache-write token once as ordinary input and then again at a total write rate.

This equation computes token charges within the chosen boundary. Fixed fees, minimum commitments, credits, taxes and other services need explicit reconciliation to the invoice. It does not establish collected cash or accounting revenue recognition. An API customer's spend is revenue for the contracting provider; it is not automatically revenue for an underlying GPU owner.

3. A small example, with invented numbers

Suppose a fictional service records the following over 8 allocated GPU-hours. Every volume and price below is chosen only to demonstrate the arithmetic. None is a provider quote, benchmark or INTERTIE.ai operating result.

The result is $11.90 ÷ 8 = $1.4875 per allocated GPU-hour, or $1.49 rounded. This simplified example assumes no separately billed cache writes, fixed fees, credits or other charges.

The input-token cache-hit rate is 9 ÷ (3 + 9), or 75%. It is not 9 ÷ 14: output tokens do not belong in this denominator. A request-level hit rate would answer another question, because a short request and a long request would each count once. OpenAI also recommends aggregating cached-token and input-token counts to measure the token hit rate.^[3]

These distinctions matter economically. A higher cache hit rate can reduce work and customer charges for a fixed request stream. Whether it improves a provider's margin depends on the capacity freed, its cost, the prices charged and whether demand fills that capacity. The hit rate alone does not settle the outcome.

4. Do not apply utilization twice

The example already divides measured token charges by all 8 allocated GPU-hours. Idle time within those hours has already affected the result. Multiplying $1.4875 by another “utilization” assumption would discount the same observation again.

If the same work occupied only 4 busy GPU-hours, dividing $11.90 by 4 would produce $2.975 per busy GPU-hour. Both calculations can be valid, but their denominators describe different things. State which one is being used. A device's reported busy percentage is also not necessarily the share of saleable capacity filled at an acceptable latency.

For a scenario model, changing a reference result by target utilization ÷ reference utilization can be a declared simplifying assumption. It requires the same utilization definition and an assumption that throughput, workload mix and price scale proportionally. Batching, queueing, latency limits and cache behavior can break that relationship. Without a measured reference utilization, do not present the extrapolation as a production forecast.

5. Prefill and decode must share the accounting boundary

Prefill processes the prompt; decode generates the response. They can use different pools of hardware. DeepSeek's deployment discussion describes this separation and gives pool-specific operating context.^[4] Adding a prefill rate per prefill GPU to a decode rate per decode GPU does not produce a rate per GPU for the combined service.

The simplest reconciliation uses the actual billable counts and the GPU-hours of every included pool. Where only component rates are available, first weight each by its own pool's share of the total GPU-hours, and confirm that the components describe the same requests and time window. Do not mix one pool's peak test with another pool's daily average.

Reserve capacity, failover and shared workloads need equally clear treatment. Excluding them may be appropriate for an engine test. A service-level business comparison must explain who pays for them and how their costs and hours are allocated.

6. A price match is a specification match

A throughput result for one model version cannot simply be priced with another version's API rate. Match model, provider, region, currency, service tier, billing categories and effective date. If the commercial arrangement sells GPU-hours instead of tokens, use that contract's unit and avoid attributing the customer's token revenue to the infrastructure provider.

Apply commercial adjustments once. A rate that already includes a batch, off-peak or negotiated discount should not receive the same discount again. If the agreement contains minimum spend or service credits, reconcile them explicitly instead of hiding them in a universal multiplier.

For this reason, this article contains no current provider price ranking or estimate of any provider's revenue. Public benchmark figures and price pages are useful evidence, but they do not reveal the matched demand, realized commercial terms and full cost record of a production service.

7. Measure the service before forecasting the business

A useful operating record joins the request and billing logs to a capacity timeline. For each comparable workload, retain the model/version, input and output counts, cache categories, successful and failed requests, response-time distribution, billing treatment and allocated GPU-hours. Failed requests and retries may consume resources even when their charges differ; the invoice must determine the treatment.

Compare measured token charges with the revenue actually attributable to the operator, then account for hardware ownership or rental, power, networking, storage, people and other relevant costs. Revenue per GPU-hour is one diagnostic. It is not profit, margin or a payback period.

At INTERTIE.ai, we see the value of a benchmark in the questions it lets an operator test: which workload fits this system, at what service quality, and with what billable output across the whole capacity window? That is the connection a commercial model needs. A larger tokens/s headline cannot supply it on its own.

Sources and scope

This is a methodology note based on public documentation checked on 21 October 2026. The equations and fictional example are our explanatory construction. The sources support measurement and billing definitions; they do not validate the example as production economics. No private operating records or INTERTIE.ai performance claims are included.