Systems Logging and Telemetry - Technical
README · ARCHITECTURE · TECHNICAL · source
1. None and 0 are stored as different values
Four providers report reasoning effort in three different ways, and some report nothing. Every optional field has two possible absences: the model did none, or the provider will not say.
Every optional field on Generation is T | None. None means
not reported; 0 means reported as none. had_thinking returns bool | None and
prefers reasoning_tokens when that metadata is present, falling back to block
count and returning None when neither exists.
Storing an absence as zero converts missing information into a confident negative observation, which then averages with real zeros from providers that do report. The result is a cross-provider comparison that is entirely artefact.
The concrete case is in the source: the Anthropic API returns a thinking block with
a signature and an empty thinking string. Presence is observable, content is
not. Deriving presence from chars > 0 reads every thinking response as
not-thinking — correct for other providers, but stuck at floor for this one.
Tri-state values propagate. Every consumer must handle None, and
the CSV needs a convention for it. Nothing enforces that a new field follows the
rule.
It records that a provider does not report something; it cannot recover the value. Reasoning-token metadata is still provider-native: it can help interpret a response, but this implementation does not assume that the values are directly comparable across APIs.
2. Track what responded, not what was requested
The model you asked for and the model that answered are not reliably the same. Aliases resolve to dated snapshots, fallbacks fire, accounts migrate — and none of it announces itself.
trackModel runs after every proxied response, reading the
model id out of the response body or the SSE stream. It writes model:current,
increments per-family model:counts, and appends to model:gossip only when the
model differs from the previous call.
A request log tells you your intent. A drift log tells you what changed underneath you, which is the thing you cannot otherwise find out. Logging only transitions keeps it small enough to live in one KV value.
Three KV reads and writes on a path that already has plenty. The counters are also read-modify-write, so concurrent responses can lose an increment — these are usage indicators, not billing.
model:gossip keeps 20 entries. That is enough to notice drift and
not enough to study it. And a model that changes and changes back between two calls
is invisible.
3. Unparseable responses become "unknown", not errors
Four provider response shapes, two transports. Something will not parse.
Every extraction path returns "unknown" on failure —
extractModelFromSSE wraps the whole parse in try/catch.
Throwing would drop the response from the counters entirely, biasing usage numbers toward providers whose shapes are easy to parse. The numbers would be least reliable about exactly the providers you understand least.
Failures are silent. An unknown bucket can grow without anything
raising an alarm.
Nothing monitors the size of unknown. A parser broken by a
provider format change would show up as a growing bucket that no one is looking at
— which is the same class of failure as an instrument reading floor, and this
subsystem has no defence against it.
4. Keep cleanup inside the data it understands
A cleanup job can only classify objects safely when it understands how their ownership is recorded. The rule I want here is an allowlist: act on the object types the job can cross-reference and leave everything else alone.
That choice can under-clean. An object the job does not understand may remain in storage longer than necessary. For a deletion-only job, that is the safer direction to fail. The implementation review that shaped this rule stays in the private engineering notes.
5. An approximate rate limit, stated as approximate
Some routes spend money per request and need a ceiling. The current limiter uses an expiring counter in shared storage, which keeps the mechanism small and puts the friction close to the spend.
It is deliberately approximate under bursts and around window boundaries. That is useful protection for a small personal system, not a strict quota and not a multi-user authorization layer. The distinction matters more than making the limiter sound stronger than it is.