LLMOps

Our own assistant, measured in public

Most writing about LLM observability is written by people not showing you their numbers. This site runs an AI assistant in production, instruments it under the OpenTelemetry GenAI conventions, and publishes what it measures.

Token counts come from each provider’s own usage reporting, not from an estimate. Where a provider does not report usage, the figure reads not reported rather than zero — absent and none are different measurements. No prompt or completion text is stored, so none can appear here.

Calls: 11; 30 days
Served from cache: 0%; no tokens spent
p50 latency: 1.16s; uncached
p95 latency: 5.52s; n=11

Input tokens: 22,094
Output tokens: 6,182

Failure rate

0% of calls

No failures recorded in this window.

Error classes follow OTel’s error.type convention. A provider_429 is the upstream rate limit; the status is kept and the response body is discarded, because a provider error body can echo the request.

What is answering

Provider	Requested model	Calls
custom	openai/gpt-oss-20b	11

How to instrument yours

The value of the OpenTelemetry GenAI conventions is the vocabulary, and you can adopt the vocabulary without adopting the transport. These are the attribute names this site records:

gen_ai.operation.name	What kind of call it was.
gen_ai.provider.name	Which provider answered. Supersedes the older gen_ai.system.
gen_ai.request.model	The model you asked for.
gen_ai.response.model	The model that actually answered — often more specific.
gen_ai.usage.input_tokens	Prompt tokens, as the provider reported them.
gen_ai.usage.output_tokens	Completion tokens, as the provider reported them.
gen_ai.response.finish_reasons	Why generation stopped. 'length' here means truncation.
error.type	Error class on failure. Never the response body.

Two things worth copying. First, record a cache hit as a call with no token counts rather than not recording it — omitting cache hits overstates both your mean latency and your spend per question. Second, store an unreported token count as null, not zero: the difference between “we did not measure” and “it was free” matters the moment anyone builds a budget on it.

There is deliberately no OpenTelemetry SDK behind this page. This site runs on a single small instance with no collector to export to, so the conventions supply the names while the metrics are persisted directly. Calling that distributed tracing would be false, and the honest gap is written up on our own logging and visibility risk page, alongside the fact that our logs still die with the container.

Figures cover the last 30 days · cached for 5 minutes · no prompt content stored