@@ -0,0 +1,199 @@
---
type : reference
title : LLM Usage Attribution
description : Research catalog of every viable way to answer "who is using the fleet LLMs" — engine-native identity, gateways, auth proxies, tracing platforms, network/host attribution — with the measured fleet baseline (2026-10-13) and a phased adoption plan.
tags : [ llm, observability, auth, gateway, litellm, tracking, identity]
timestamp : 2026-10-14T00:00:00 -07 : 00
---
# LLM Usage Attribution
Deep-research catalog (2026-10-13/14): solutions for tracking WHO uses the
fleet endpoints (`frank-archon` , `frank-dragoon` , `frank-zealot` ). Full web
citations; live-fleet measurements marked * * (measured)**.
## Baseline: what the fleet answers today
- All three endpoints are `auth: none` , plain HTTP, LAN-only
(`models.yml` providers; no vhost on 7770/7771).
- **(measured)** Live `/metrics` label census: `frank-archon` exports
`vllm:*` +`radiance:*` with labels only `{model_name, engine, le,
finished_reason, position, class}` — zero identity labels.
`frank-dragoon` exports `halogen:*` +`llamacpp:*` .
`frank-zealot` answers **501 ** — `serve-frank-zealot.sh` never passes
`--metrics` , so the Prometheus job is structurally never-green
(`LlmEndpointDown` fires). Fix: add `--metrics` to the ARGS array.
- **(measured)** frank-zealot verbosity-4 journal lines carry slot/task/timing
only — no client IP, no user. Alloy ships journal + podman ctr.log to Loki,
so what lands there has no identity either.
- Caddy has no `log` directive anywhere → no per-request access logs on the
OpenWebUI vhost.
- Net: fleet-level token telemetry exists per ENDPOINT; per-CONSUMER
identity exists nowhere.
## The identity layers (pick the layer, get its resolution)
| # | Layer | Identity | Answers |
|---|---|---|---|
| L0 | DHCP/ARP | MAC, hostname, lease | which device was on the LAN |
| L1 | DNS (Pi-hole v6) | device → queries | which device resolved/used a name |
| L2 | L3/L4 (nftables/conntrack, JA3/JA4) | IP, TLS client lib | bytes per client, client software |
| L3 | Transport (API keys, mTLS, SPIFFE, Tailscale) | key/cert/node | which credential connected |
| L4 | App gateway (virtual keys → user/team/end_user) | named consumer | tokens/spend/cost per who |
| L5 | Trace/content (Langfuse, Helicone, halogen records) | user_id, session, prompt text | what they asked, per session |
| L6 | Client host (auditd, eBPF, netdata) | pid/uid/comm per connect | which process on which host |
Keys/headers/`user` fields are CLIENT-DECLARED — they group traffic, they
prove nothing about a human (halogen's own docs warn this verbatim:
"they group requests, and they prove nothing about who sent them").
## Catalog
### A. Engine-native (per-consumer identity inside the inference server)
| Engine | AuthN | Identity in logs | Per-identity metrics | OTel | Notes |
|---|---|---|---|---|---|
| llama.cpp | `--api-key` (multi, comma) / `--api-key-file` * * (measured in the deployed build's `--help` )** | request `user` field drives slot assignment/eviction; `--log-requests` dumps bodies | `--metrics` exports `llamacpp:*` , NO key/user label | none upstream | [server README ](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md ) |
| vLLM | `--api-key` (multi) | request log; no user persistence | `vllm:*` ; NO `api_key` label (cardinality, issue #53926 ) | native (`--otlp-traces-endpoint` ) | [metrics design ](https://github.com/vllm-project/vllm/blob/main/docs/design/metrics.md ) |
| radiance | no public auth docs * * [unverified]** | — | * * (measured)** no identity labels on live /metrics | none found | closed deployment surface; treat authN as absent until probed |
| halogen-flash-server | `HALOGEN_API_KEY` / `HALOGEN_API_KEY_FILE` (multi, comma) | * * `HALOGEN_RECORD_DIR` request records**: `turns/*.jsonl` per request — `client_family` (User-Agent), tokens, timings, finish, tool calls, plus HASHES of `user` / `prompt_cache_key` / `safety_identifier` / `metadata.user_id` (`.salt` -keyed); `units/` content-addressed text; caps `HALOGEN_RECORD_GIB` =2, `HALOGEN_RECORD_DAYS` =90; `HALOGEN_RECORD_TEXT=none` for metadata-only; `stats` /`prune` tools in-container | `halogen:*` +`llamacpp:*` , no per-key label | none | [configuration.md ](https://github.com/peonist-ai/halogen-flash-server/blob/main/docs/configuration.md ) — verbatim-verified 2026-10-13. Best native ledger of the fleet. |
| SGLang | `--api-key` (single) | `--log-requests` | `sglang:*` , no key label | native (`--enable-trace` ) | for any future swap |
| Ollama | single key | request log | `ollama:` no key label | none | not in fleet |
Wire fields usable from clients today (all client-declared): OpenAI
`user` , `safety_identifier` , `prompt_cache_key` , Messages
`metadata.user_id` ; LiteLLM `end_user_id` ; gateway header conventions
(`X-Forwarded-User` , `Helicone-User-Id` , `cf-aig-metadata` ).
### B. LLM gateways (front door with virtual keys — the only layer with
per-consumer TOKEN counts)
| Solution | Identity | Usage lands | License | Footprint |
|---|---|---|---|---|
| **LiteLLM Proxy ** | virtual keys → `user_id` /`team_id` /`end_user` ; budgets, rate limits | Postgres spend ledger; `/metrics` labels `api_key_alias,user,team,end_user,model,client_ip,user_agent` (`litellm_*_tokens_metric` , spend, budget gauges); OTel/Langfuse callbacks | MIT core (SSO/RBAC/audit-log enterprise) | ~150–
| Kong AI Gateway | Consumer + key-auth/JWT/OIDC/mTLS; AI consumer groups | Postgres; `kong_*` per-consumer; file-log/zipkin/otel | OSS core; token-aware AI plugins enterprise | 0.5–
| Apache APISIX | Consumer + key-auth/JWT | `apisix_llm_prompt_tokens` /`completion_tokens` with `consumer` label; http-logger/kafka | Apache-2.0 | 200–
| Bifrost (maximhq) | virtual keys `sk-bf-*` /`x-bf-vk` , budgets | Prometheus/OTel, config store (SQLite/Postgres) | Apache-2.0, Go | ~150–
| Portkey OSS gateway | virtual keys | logs/OTel | MIT | 200–
| Higress AI GW | keys (+OIDC ent) | `higress_*` , ai-logging | Apache-2.0 CE | Istio-based, 0.5 GiB+ |
| Envoy AI GW / Gateway API Inference Extension / LLM Gatekeeper | delegated auth | Envoy metrics/access logs | Apache-2.0 | K8s-native |
| Gloo / KrakenD / Gravitee AI gateways | keys, OIDC | metrics/logs | OSS core, AI features enterprise | heavy |
| Cloudflare AI Gateway | Access JWT `cf.user_id` , `cf-aig-metadata` | Cloudflare-managed analytics/logs | SaaS (exotic for LAN; Tunnel possible) | edge |
([LiteLLM virtual keys ](https://docs.litellm.ai/docs/proxy/virtual_keys ),
[prometheus ](https://docs.litellm.ai/docs/proxy/prometheus ),
[APISIX ai-proxy ](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/ ),
[Bifrost ](https://github.com/maximhq/bifrost ),
[Kong AI ](https://developer.konghq.com/index/ai-gateway/ ))
### C. Auth proxies / identity infrastructure (identity without token counts)
- **Ory Oathkeeper** (MIT, ~50–
client-credentials → header mutation (`X-User: {{ .Subject }}` ) +
decision logs.
- **oauth2-proxy** (Apache-2.0): OIDC (Authentik) incl. client_credentials,
`--pass-user-headers` → `X-Forwarded-User` ; ~100 MiB.
- **Authentik itself**: proxy-provider custom headers (already injects
`X-Authentik-Username` /`-Groups` , spoof-stripped — the /whoami seam proves
the pattern on this host); event log + webhook forwarding.
- **Caddy `tls client_auth` ** (mTLS per client cert) + access-log fields —
Caddy already fronts both hosts.
- **Smallstep step-ca**: short-lived per-device certs, ACME-style.
- **SPIFFE/SPIRE**: per-workload SVIDs (k8s-flavored; heavy for quadlets).
- **Tailscale / NetBird**: per-node identity, ACL + network-flow logs,
headscale for self-hosted control. Best for off-LAN laptops.
- **ApiAuth** (nginx auth_request): the minimal per-consumer apikey sidecar.
### D. LLM observability / tracing (prompt-level per-user audit)
| Platform | Identity | Content | Self-host | Shape |
|---|---|---|---|---|
| **Langfuse ** | `user_id` ,`session_id` , tags | full input/output, masking hook | ✅ ClickHouse, Apache-2.0 | fed via LiteLLM callback / OTLP / SDK |
| **Helicone ** | api-key→user, `Helicone-User-Id` | full, tiered redaction | ✅ compose (ClickHouse+PG+Redis+Jawn), Apache-2.0 | PROXY — can be the front door itself |
| Arize Phoenix | `user.id` ,`session.id` | opt-in `gen_ai.*` | ✅, Apache-2.0 | OTLP collector |
| OpenLIT | `user.id` ,`session.id` | 3-tier capture | ✅, Apache-2.0 | OTLP+ClickHouse |
| SigNoz LLM | OTel attrs | opt-in | ✅, Apache-2.0 | OTLP |
| OpenLLMetry/Traceloop | assoc props | opt-in | ✅ Helm | OTel |
| Lunary / W&B Weave / LangSmith | user/session | full | enterprise-gated | — |
OTel GenAI semconv (Development-stability): `gen_ai.operation.name` ,
`gen_ai.provider.name` , `gen_ai.request.model` , `gen_ai.usage.input_tokens` /
`output_tokens` , `gen_ai.conversation.id` , content attrs opt-in; `user.id`
from the registry. vLLM and SGLang emit OTLP natively; llama.cpp only via
external SDK — **the fleet's engines are OTel-silent at request level ** ,
which is why the gateway callback (LiteLLM→Langfuse) is the practical feed.
### E. Network/host passive attribution (no client cooperation)
- **Pi-hole v6** (already on Dankclaw): query log API per device,
`pihole6_exporter` + query-logger→Loki (GPLv3 exporter).
- **Caddy access logs**: add a `log` directive (JSON) → Alloy→Loki; with the
auth chain it carries `X-Authentik-Username` for browser users.
- **OpenWebUI admin**: per-user daily token stats, events/logs, analytics API
(`/api/v1/analytics/*` ) — per-USER identity exists here already (MIT).
- **auditd** connect-syscall rules (uid/pid/comm per connect to 7770/7771),
ship via audisp→Loki.
- **eBPF**: Tetragon (process+socket events, policies), Coroot, DeepFlow,
bpftrace `tracepoint:syscalls:sys_enter_connect` one-liners. K8s-shaped
for single hosts, but agents.plugin-style tools work.
- **nftables named counters** per consumer IP (bytes; reload resets).
- **netdata apps.plugin**: per-process network bytes, auto-discovered.
- **JA3/JA4 TLS fingerprinting**: client library only, TLS-only; the LAN is
plaintext HTTP today → use the HTTP **User-Agent census ** instead (free
from any access log).
- **DHCP/ARP census** (leases + `ip neigh` ): device roster.
## Phased plan for this fleet
**Phase 0 — repo-only, no new services **
1. frank-zealot: add `--metrics` (fixes the measured 501 + the
structurally-red Prometheus job + the firing `LlmEndpointDown` ).
2. frank-dragoon: set `HALOGEN_API_KEY_FILE` (multi-key per consumer) +
mount `HALOGEN_RECORD_DIR` (start `HALOGEN_RECORD_TEXT=none` ; flip to
`all` if prompt audit is wanted) + Alloy ship of `turns/` .
3. frank-zealot: `--api-key-file` (multi-key, verified in the deployed
build); radiance: probe for `--api-key` support (contract facts are
silent — add to `fleet-probes` ).
4. models.yml: per-consumer bearer keys (`auth: none` → key) — the provider
schema supports apiKey (pi-era conversion note).
5. Caddy: `log` on `ai.franksdank.cloud` (OpenWebUI browser census, JSON →
Loki).
Limits: keys gate access, and only the engines' request records (halogen)
see them per-request; llama.cpp/radiance/vLLM metrics stay identity-blind.
**Phase 1 — LiteLLM Proxy quadlet on Dankclaw = the front door. **
One virtual key per consumer (omp-<workstation> ×
agents, bench). Per-key/user/team Prometheus labels into the existing
Grafana+Alertmanager; Postgres spend ledger; budgets + rate limits per
key. This is the ONLY layer that yields tokens-per-who without engine
cooperation. (Bifrost is the lighter Apache-2.0 Go alternative; Helicone
the proxy-native-observability alternative.)
**Phase 2 — prompt-level audit. ** Langfuse self-hosted fed by the LiteLLM
callback: per-user traces, sessions, cost; masking + retention knobs.
**Phase 3 — ambitious identity infra. ** step-ca + Caddy mTLS per device;
Tailscale (+flow logs) for off-LAN; auditd/eBPF process attribution on the
agent hosts; Pi-hole query logger; nft counters per consumer IP. Browser
users stay owned by OpenWebUI analytics + its per-user API keys; a pipeline
can forward `X-Forwarded-User` so the gateway sees them too.
## Cautions
- Client-declared identity (keys, `user` , headers) groups traffic; it does
not authenticate humans. Authenticated identity = Authentik (browser) /
mTLS (device) / Tailscale (node).
- Plain-HTTP LAN = bearer keys sniffable by any tenant; treat keys as
identity labels on the LAN, not secrets, until TLS/mTLS at the door.
- Prompt-content logging stores a plaintext copy of everything — set
retention caps (halogen GIB/days; Langfuse masking; Loki retention) and
gitignore the record dirs.
- Household-scale cardinality is safe for per-key Prometheus labels;
the engines' refusal of key labels is a multi-tenant concern, not ours.
## Related
- [[Fleet Audit and Posture]] — monitoring gaps census
- [Observability ](../Infrastructure/Observability.md ) — the Loki/Prometheus
pipes every sink above uses
- [LLM index ](../LLM/index.md ) — endpoint runbooks (env trios the Phase-0
edits touch)