What is Time to First Token (TTFT), and what is a good TTFT for AI?

The clock starts when your request leaves and stops when the first visible token arrives. Everything after that, the rest of the answer streaming in, is a different measurement. A model can start quickly and then write slowly, or the other way round.

We say first visible token on purpose. Reasoning models often emit hidden thinking before they say anything a person can read. That work is real, but it does not make a blank chat window feel any faster, so our measurements only stop the clock once there is text a user could see.

For a short conversational request, this is the rule of thumb we use. It is a practical guide rather than an industry standard, and different products will draw the lines in different places.

Context matters. A 2.5-second TTFT for a one-line chatbot reply feels slow. The same wait for a model working through a hard problem, reading a long document or calling tools before it answers can be perfectly reasonable. Judge the number against what you asked the model to do.

We measure the first-token time of the major AI APIs every day, from several cities, with the same short prompt and the lowest reasoning effort each model offers. This is what that looks like right now.

These are typical times: half of the requests in the window were faster, half were slower. Each provider's own page shows the per-city numbers and how they moved from day to day.

• Prompt length. The model has to read the whole input before it produces anything. A long system prompt or a pasted document pushes the first token back.
• Model and reasoning level. Larger models and higher reasoning settings do more work before the first visible token.
• Provider load and queueing. Your request may wait in line before a GPU picks it up. When a provider is busy, most of the delay is not your network at all.
• Caching. Providers that cache a repeated prefix, such as a long system prompt, can skip much of the input processing on later requests.
• Network distance. It matters, but for hosted AI APIs it is usually a small share of the total.

TTFT tells you when the answer starts. Tokens per second tells you how fast it flows once it has started. The two can point in opposite directions.

A model with a fast TTFT and slow generation starts replying at once, then crawls. That is fine for a one-word answer and painful for a long one. A model with a slow TTFT and fast generation leaves a blank screen for two seconds, then the whole answer lands in a moment. That is fine for a summary and frustrating in a back-and-forth chat.

For conversational products, both matter. Google's inference metrics for GKE track TTFT and time per output token as separate numbers for exactly this reason. Our benchmarks measure TTFT only; we do not publish generation speed.

1. Stream the response. Start the timer when you send the request. Stop it when the first visible output token arrives, not when the HTTP headers do. Some providers send headers early and the first token much later.
2. Do not trust one request. Run enough requests to see the typical result (p50) and the slow tail (p95). The tail is what your unluckiest users get.
3. Keep everything else fixed. Same model, same prompt, same reasoning setting. Change one thing at a time.
4. Measure from where your users are. If they are spread across continents, measure from several test locations. If they are all in one country, one location close to them is enough.

See how we measure AI API latency for the exact prompt, the reasoning settings and how we count a request that never shows visible text.