Merge pull request 'Alloy ships logs: root host-network service so it can read the journal' (#53 ) from feat/alloy-root-journal into main

docs(alloy): document the root journal-read cutover
Alloy page rewritten to the shipped root design (why root, the two mounts that make it work, deploy-pending status); Observability current state marks the log pipeline fixed-in-code, awaiting deploy.
2026-10-20 20:05:00 -07:00 · 2026-10-20 19:50:15 -07:00 · 2026-10-20 19:50:15 -07:00 · 2026-10-20 19:50:15 -07:00 · 2026-10-20 19:39:38 -07:00 · 2026-10-20 19:28:20 -07:00
30 changed files with 883 additions and 242 deletions
@@ -166,9 +166,11 @@ bare compose container never returned; 2026-09-11: five observability
 services had no units). The observability services now have units under
 `dankclaw/observability/quadlet/` and their compose definitions were removed —
 one owner per container name. The stack is **configured and delivering as of
-2026-09-17**: a Grafana datasource and the `llm-fleet` dashboard are
-provisioned from `shared/grafana/provisioning/`, `shared/prometheus/rules/
-llm-fleet.yml` loads 5 alert rules + 1 recording rule, and firing rules are
+2026-09-17**: Grafana datasources (Prometheus and Loki) and the `llm-fleet`
+dashboard are provisioned from `shared/grafana/provisioning/`, and
+`shared/prometheus/rules/llm-fleet.yml` loads two rule groups — `llm-fleet`
+(llama.cpp/halogen, `llamacpp:*`) and `vllm-fleet` (radiance, `vllm:*`) — alert
+ recording rules, and firing rules are
 delivered: Prometheus → Alertmanager (`dankclaw/observability/quadlet/
 alertmanager.container`, loopback :9093) → host-side Signal relay
 (`scripts/signal-relay.py`, user unit, 192.168.0.143:18093 — the host LAN
@@ -803,6 +805,20 @@ dies, its page is destroyed. Rollback is a commit, not a document.
 `serverAppConfigs-docs/index.md` is the locator for every page that survives;
 retired scripts live in `old/` and `.AGENTTRASH/`.

+## Agent skills
+
+### Issue tracker
+
+Issues live in Gitea (`dank/serverAppConfigs` on Dankclaw), via its REST API with the stored credential. See `serverAppConfigs-docs/Agent/Issue Tracker.md`.
+
+### Triage labels
+
+The five canonical roles keep their default strings (`needs-triage`, `needs-info`, `ready-for-agent`, `ready-for-human`, `wontfix`). See `serverAppConfigs-docs/Agent/Triage Labels.md`.
+
+### Domain docs
+
+Single-context: root `GLOSSARY.md` + `docs/adr/`, created lazily; this vault stays the owner of operational facts. See `serverAppConfigs-docs/Agent/Domain Docs.md`.
+
 ## Why

 Direct edits on the remote drift from version control, and future agents
@@ -1,55 +1,51 @@
-# Quadlet unit: alloy (rootless, boot-persistent).
+# Quadlet unit: alloy (ROOT system service, boot-persistent).
 #
-# Install path on dankclaw: ~/.config/containers/systemd/alloy.container
-# Unit name: alloy.service (container name: alloy)
+# Install path on dankclaw: /etc/containers/systemd/alloy.container
+# Unit name: alloy.service (container name: alloy) — SYSTEM manager, root.
 #
-# Log shipper. This replaces the compose service of the same name in
-# dankclaw/compose.yaml ("Observability" block).
+# WHY ROOT (not rootless, unlike its observability siblings). Alloy reads the
+# host journal. The journal files are root:systemd-journal 0640 (other::---);
+# a rootless container carries no supplementary groups and cannot open them —
+# a rootless read of system.journal returns DENIED (see the Alloy page in the
+# vault). Running as root, one loki.source.journal reads system.journal AND
+# user-1000.journal, so it carries both host logs and the rootless Quadlet
+# containers' logs. Validated 2026-10-20: a root probe read 31,825 journal
+# lines and pushed 26,966 into Loki in 20 s, zero drops.
 #
-# IMAGE. franksdank/alloy is built locally by deploy-remote.sh
-# (build_compose_image) from dankclaw/alloy/Containerfile: upstream
-# grafana/alloy plus a static busybox for the healthcheck. Pull=never keeps a
-# boot from reaching for that local tag at a registry.
+# NETWORK=host. Loki publishes 127.0.0.1:3100 (loopback only) on the host, and
+# no bridge container — rootless or root — can reach the host's loopback, so
+# this unit runs on the host network and pushes to 127.0.0.1:3100. The side
+# effect: the UI must bind loopback too, so Exec sets the listen-addr to
+# 127.0.0.1 (the old rootless unit used 0.0.0.0 only because container loopback
+# was not host loopback; under Network=host they are the same address, so the
+# unauthenticated UI stays loopback-only — nothing off-host reaches it).
 #
-# NETWORK. It joins dankclaw_default so its config's loki.write endpoint
-# resolves the container name loki:3100.
+# THE /etc/machine-id MOUNT IS LOAD-BEARING. sd_journal locates the journal
+# directory by the local machine id; without it the container opens nothing and
+# the journal source reads exactly 0 lines — silently, no error, health green.
+# This mount is the entire reason the pipeline carries data. Never remove it.
 #
-# THE UI PORT EXISTS ON PURPOSE. Alloy's UI listens on the container's own
-# 127.0.0.1 by default and cannot be moved from the config file (alloy 1.x
-# refuses the ui block), so the compose service had no reachable health
-# endpoint at all: the homepage status card for alloy reported "down"
-# continuously and there was nothing for a deploy to wait on. The command
-# below therefore moves the UI to the container's wildcard address and the
-# port is published on host loopback only, which puts alloy in the same class
-# as the other three: reachable from the host, by Caddy over the private
-# network, and by nothing off-host. Alloy has no authentication, which is
-# exactly why it is not published wider.
-#
-# SOURCES. The host journald socket and the host log/container-storage trees
-# are bind-mounted read-only; Alloy tails them and pushes to Loki. The config
-# lives at /opt/alloy/config/config.alloy, installed from
-# shared/alloy/config.alloy on every deploy.
-#
-# ORDERING. Wants/After loki.service: without a reachable Loki, Alloy's write
-# client fails and its log pipeline backs up.
+# IMAGE. Built rootless by build_compose_image alloy, then mirrored into the
+# root image store by the deployer (a root build runs in the build container's
+# own resolver and fails on DNS here). Pull=never: boot must not reach for the
+# local tag at a registry.

 [Unit]
 Description=Alloy log shipper (Dankclaw)
-After=network-online.target loki.service
-Wants=network-online.target loki.service
+After=network-online.target

 [Container]
 ContainerName=alloy
 Image=localhost/franksdank/alloy:latest
 Pull=never
-Network=dankclaw_default
-Exec=run --server.http.listen-addr=0.0.0.0:12345 /etc/alloy/config.alloy
-PublishPort=127.0.0.1:12345:12345
-Volume=/opt/alloy/config:/etc/alloy
+Network=host
+Exec=run --server.http.listen-addr=127.0.0.1:12345 /etc/alloy/config.alloy
+Volume=/opt/alloy/config:/etc/alloy:ro
+Volume=/etc/machine-id:/etc/machine-id:ro
 Volume=/run/systemd/journal/socket:/run/systemd/journal/socket:ro
-Volume=/var/log:/var/log:ro
-Volume=/var/lib/containers:/var/lib/containers:ro
-HealthCmd=/usr/local/bin/busybox wget --spider -q http://localhost:12345/-/healthy
+Volume=/var/log/journal:/var/log/journal:ro
+Volume=/run/log/journal:/run/log/journal:ro
+HealthCmd=/usr/local/bin/busybox wget --spider -q http://127.0.0.1:12345/-/healthy
 HealthInterval=30s
 HealthTimeout=10s
 HealthRetries=3
@@ -60,4 +56,4 @@ Restart=always
 RestartSec=10

 [Install]
-WantedBy=default.target
+WantedBy=multi-user.target
@@ -0,0 +1,26 @@
+# /etc/udev/rules.d/98-halogen-npu-access.rules
+#
+# Why the NPU node is world-writable: `frank-dragoon` runs rootless podman under a
+# unit with `User=dank`, and the container's user namespace maps exactly one uid and
+# one gid (dank). `/dev/accel/accel0` is `root:render 0660`, so inside the container
+# the node reads `nobody:nogroup` and the engine dies with its own diagnosis:
+#
+#   halogen npu: /dev/accel/accel0 is here, but this process cannot open it
+#     (crw-rw---- nobody:nogroup).
+#
+# `--group-add keep-groups` is the vendor's advice and it does NOT work in this
+# shape: measured 2026-10-19 from the host as a `dank` shell that was already in
+# `render`, with and without the flag, the node still read `nobody:nogroup` and the
+# open was refused — an unmapped gid cannot be granted to a process inside a
+# namespace that does not map it. Same conclusion `99-comfyui-gpu-access.rules`
+# reached for `/dev/kfd` from the systemd-user-spawn direction: the mode is the only
+# spawn-context-independent door short of running the stack rootful.
+#
+# Blast radius: any process in this host's single uid domain (every rootless
+# container here shares it) can submit to the XDNA2 NPU. The NPU holds no user data
+# and no privileged context — the same trade already accepted for the GPU compute
+# nodes beside it. Installed by `deploy-remote.sh` when the NPU lane is armed and left
+# in place afterwards — the mode costs nothing while the lane is idle. `serve-frank-dragoon.sh`
+# refuses to arm the lane on a node that is not other-accessible, so a host that lost this
+# file fails loudly at boot instead of crash-looping.
+KERNEL=="accel[0-9]*", SUBSYSTEM=="accel", MODE="0666"
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# Set the authentik login-session TTL on Dankclaw.
+#
+# The login-session lifetime is governed by the `default-authentication-login`
+# stage's `session_duration`, NOT a global setting and NOT SESSION_COOKIE_AGE
+# (that is the pre-login session, sessions.unauthenticated_age). Authentik does
+# not expose SESSION_SAVE_EVERY_REQUEST at all, so the session is an absolute
+# window from login, not a sliding one.
+#
+# The field is a STRING in authentik's own idiom ("days=7", "seconds=30",
+# "hours=12") — not a Python timedelta. Writing str(timedelta(days=7)) stores
+# "7 days, 0:00:00", which authentik's timedelta_from_string parser REJECTS
+# (ValueError), and login then crashes computing the session expiry. Always pass
+# the "key=value" form; this script validates it before writing.
+#
+# Existing sessions keep their old expiry; the new TTL applies from the next
+# login. This is DB state — it is NOT repo-managed and a blueprint re-import or
+# rebuild-from-scratch reverts it. Run this again after such a rebuild.
+#
+# Usage: authentik-set-login-session-duration.sh ["days=7"]   (default days=7)
+set -euo pipefail
+
+DUR="${1:-days=7}"
+STAGE="default-authentication-login"
+HOST="${AUTHENTIK_HOST:-dankclaw}"
+
+# Refuse anything that is not the safe key=value duration idiom.
+if ! printf '%s' "$DUR" | grep -Eq '^[A-Za-z]+=[0-9]+([ ]+[A-Za-z]+=[0-9]+)*$'; then
+    echo "invalid duration '$DUR' — use authentik form, e.g. 'days=7' or 'seconds=30'" >&2
+    exit 2
+fi
+
+ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
+from django.apps import apps
+from authentik.lib.utils.time import timedelta_from_string
+dur = "$DUR"
+timedelta_from_string(dur)  # validate the form before touching the row
+ULS = apps.get_model("authentik_stages_user_login", "UserLoginStage")
+s = ULS.objects.get(name="$STAGE")
+s.session_duration = dur
+s.save(update_fields=["session_duration"])
+back = ULS.objects.get(name="$STAGE").session_duration
+assert back == dur, "persisted value mismatch: %r" % back
+print("session_duration set to", repr(back), "->", timedelta_from_string(back))
+PYEOF
+
+echo "Login-session TTL for '$STAGE' is now $DUR (applies on next login)."
@@ -487,13 +487,15 @@ seed_grafana_env() {
        || { echo "grafana env missing GRAFANA_OAUTH_CLIENT_SECRET; set it in dankclaw/.env" >&2; return 1; }
 }

-# The five observability containers left compose on 2026-09-11 (they were
-# started by hand with podman-compose and never came back after a reboot).
-# They are ordinary rootless Quadlet units now: same images, same mounts, same
-# arguments, same container names, so nothing that addresses them by name —
-# Caddy, Prometheus' own scrape config, Alloy's loki.write endpoint — changed.
-# Only the published ports did: the three HTTP UIs bind 127.0.0.1, because the
-# compose services published on 0.0.0.0 and none of them authenticates.
+# The observability containers left compose on 2026-09-11 (they were started by
+# hand with podman-compose and never came back after a reboot). Five are ordinary
+# rootless Quadlet units: same images, same mounts, same arguments, same
+# container names, so nothing that addresses them by name — Caddy, Prometheus'
+# own scrape config — changed. Only the published ports did: the three HTTP UIs
+# bind 127.0.0.1, because the compose services published on 0.0.0.0 and none of
+# them authenticates. Alloy is the exception: a ROOT system service on
+# Network=host, because reading the host journal needs privileges a rootless
+# container cannot get. It is handled by deploy_alloy, not the loop below.
 deploy_observability() {
    local service="$1"
    # Observability units live under <target>/observability/quadlet: Dankclaw
@@ -512,10 +514,6 @@ deploy_observability() {
            copy_loki_config
            build_compose_image loki
            ;;
-        alloy)
-            copy_alloy_config
-            build_compose_image alloy
-            ;;
        grafana)
            copy_grafana_provisioning
            seed_grafana_env || return 1
@@ -534,11 +532,71 @@ deploy_observability() {
        alertmanager) wait_for_health alertmanager http://127.0.0.1:9093/-/healthy 60 ;;
        grafana) wait_for_health grafana http://127.0.0.1:3002/api/health 90 ;;
        loki) wait_for_health loki http://127.0.0.1:3100/ready 60 ;;
-        alloy) wait_for_health alloy http://127.0.0.1:12345/-/healthy 60 ;;
        node-exporter) wait_for_health node-exporter http://127.0.0.1:9100/metrics 60 ;;
    esac
 }

+# Mirror an image the deployer built rootless into the ROOT image store. A root
+# quadlet reads root storage; a root build (build_compose_image is rootless) is
+# not possible on this host — the build container's resolver fails DNS — so the
+# only way to get a root-readable copy is save-and-load. Compares image ids so a
+# rebuilt alloy actually re-lands in root storage instead of leaving a stale tag.
+mirror_image_to_root() {
+    local ref="$1" rid
+    rid="$(podman image inspect "$ref" --format '{{.Id}}' 2>/dev/null)" || {
+        echo "ERROR: rootless image $ref is not built; cannot mirror to root" >&2
+        return 1
+    }
+    if [[ "$(run_privileged podman image inspect "$ref" --format '{{.Id}}' 2>/dev/null)" == "$rid" ]]; then
+        return 0
+    fi
+    local tar; tar="$(mktemp /var/tmp/alloy-mirror.XXXXXX.tar)"
+    podman save -o "$tar" "$ref"
+    run_privileged podman load -i "$tar"
+    rm -f "$tar"
+}
+
+# Install a quadlet into the SYSTEM generator dir (root manager) instead of the
+# user dir. Boot persistence is the unit's own [Install] section (multi-user
+# target); the v1 system generator creates the .wants on daemon-reload, so —
+# like the user units — no `systemctl enable` (generated units cannot be enabled
+# here).
+install_system_quadlet() {
+    local source_dir="$1" unit="$2"
+    run_privileged mkdir -p /etc/containers/systemd
+    run_privileged cp "$source_dir/$unit.container" /etc/containers/systemd/
+    run_privileged systemctl daemon-reload
+}
+
+# Alloy as a ROOT system service on Network=host. See the unit file: a rootless
+# container cannot read the 0640 journal files, and it cannot reach Loki's
+# 127.0.0.1:3100 publish from a bridge netns. Root + host network fixes both;
+# the price is that this one container runs as root and on the host network.
+deploy_alloy() {
+    copy_alloy_config
+    build_compose_image alloy                      # rootless build (resolver works)
+    mirror_image_to_root localhost/franksdank/alloy:latest
+
+    # Retire the rootless unit BEFORE the system one starts: both managers
+    # generate alloy.service, and two containers named "alloy" both binding
+    # 127.0.0.1:12345 cannot coexist. Move the user quadlet out of the generator
+    # dir (it regenerates from there) and stop the user unit.
+    if [[ -f "$quadlet_unit_dir/alloy.container" ]] ||
+       systemctl --user is-active --quiet alloy.service 2>/dev/null; then
+        systemctl --user stop alloy.service 2>/dev/null || true
+        if [[ -f "$quadlet_unit_dir/alloy.container" ]]; then
+            local trash="$HOME/.AGENTTRASH/$(date +%Y%m%d)-alloy-root-cutover"
+            mkdir -p "$trash"
+            mv -- "$quadlet_unit_dir/alloy.container" "$trash/alloy.container"
+            echo "retired rootless alloy.container -> $trash"
+        fi
+        systemctl --user daemon-reload
+    fi
+
+    install_system_quadlet dankclaw/observability/quadlet alloy
+    run_privileged systemctl restart alloy.service
+    wait_for_health alloy http://127.0.0.1:12345/-/healthy 60
+}
 build_compose_image() {
    local service="$1" image cfhash oldhash containerfile
    # Every file the image COPIES IN, beside the Containerfile. The skip below
@@ -1329,6 +1387,9 @@ deploy_anubis() {
 # port gate is the same as every cutover since 2026-09-16: nothing
 # starts until 7770 is released.
 LLM_DRAGOON_IMAGE="${LLM_DRAGOON_IMAGE:-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a}"
+# The payload dir is its own twin because the NPU lane's containment gate below
+# reads it; every artifact path under it stays a full literal as usual.
+LLM_DRAGOON_MODEL_DIR="${LLM_DRAGOON_MODEL_DIR:-/home/dank/models/halogen-v2-abliterated}"
 LLM_DRAGOON_CKPT="${LLM_DRAGOON_CKPT:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-v2-abliterated.hgn}"
 LLM_DRAGOON_CKPT_BYTES="${LLM_DRAGOON_CKPT_BYTES:-66687678432}"
 LLM_DRAGOON_NGRAM="${LLM_DRAGOON_NGRAM:-/home/dank/models/halogen-v2-abliterated/qwen38-flash-next-ngram.hgn}"
@@ -1441,16 +1502,28 @@ deploy_llm_dragoon() {
        # payload dir. A staging dir outside it needs a nested bind whose
        # mountpoint runc cannot create through a read-only parent, so the
        # parity twin of the serve gate refuses the shape before the restart.
-        npu_model_dir="$(sed -nE 's/^FRANK_DRAGOON_MODEL_DIR=(.*)$/\1/p' \
-            "$CHECKOUT/scripts/frank-dragoon.env" | tail -1)"
-        if [[ -n "$npu_model_dir" && "$LLM_DRAGOON_NPU_DIR/" != "$npu_model_dir"/* ]]; then
-            echo "ERROR: NPU staging dir $LLM_DRAGOON_NPU_DIR is outside the payload dir $npu_model_dir — the lane's files reach the container only through the read-only /models mount" >&2
+        if [[ "$LLM_DRAGOON_NPU_DIR/" != "$LLM_DRAGOON_MODEL_DIR"/* ]]; then
+            echo "ERROR: NPU staging dir $LLM_DRAGOON_NPU_DIR is outside the payload dir $LLM_DRAGOON_MODEL_DIR — the lane's files reach the container only through the read-only /models mount" >&2
            return 1
        fi
        if [[ ! -e "$LLM_DRAGOON_DEV_ACCEL" ]]; then
            echo "ERROR: NPU node $LLM_DRAGOON_DEV_ACCEL absent — amdxdna is not bound (is iommu=pt still on the boot line?)" >&2
            return 1
        fi
+        # Access, not existence: a rootless container cannot open a root:render
+        # 0660 node (its namespace maps one uid/gid, and keep-groups cannot grant
+        # an unmapped gid — measured 2026-10-19). Install the mode rule, re-trigger
+        # the node, then PROVE the node is other-accessible before the restart.
+        run_privileged install -m 0644 dankclaw/udev/98-halogen-npu-access.rules \
+            /etc/udev/rules.d/98-halogen-npu-access.rules
+        run_privileged udevadm control --reload-rules
+        run_privileged udevadm trigger --sysname-match="$(basename "$LLM_DRAGOON_DEV_ACCEL")"
+        # (`-o FILE` is not "other-writable" — it means "exists", and `-o` is also
+        # the OR operator inside test), so the bit is read straight off the mode.
+        if [[ $(( 8#$(stat -c '%a' "$LLM_DRAGOON_DEV_ACCEL") & 8#2 )) -eq 0 ]]; then
+            echo "ERROR: $LLM_DRAGOON_DEV_ACCEL is still $(stat -c '%a %U:%G' "$LLM_DRAGOON_DEV_ACCEL") after installing the access rule — the lane cannot open it rootless" >&2
+            return 1
+        fi
        for xl in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
            if [[ ! -e "$LLM_DRAGOON_XRT_DIR/$xl" ]]; then
                echo "ERROR: XRT NPU library $LLM_DRAGOON_XRT_DIR/$xl absent (Ubuntu libxrt2 + libxrt-npu2)" >&2
@@ -1926,7 +1999,8 @@ deploy_service() {
        copyparty) deploy_copyparty ;;
        plex) deploy_plex ;;
        tautulli) deploy_tautulli ;;
-        prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
+        alloy) deploy_alloy ;;
+        prometheus|grafana|loki|node-exporter|alertmanager) deploy_observability "$service" ;;
        comfyui) deploy_comfyui ;;
        *) deploy_compose_service "$service" ;;
    esac
@@ -7,7 +7,7 @@
 # whichever TTS engine wins the engine decision (dankclaw/voiceservice/app.py
 # is a single stdlib-only file in a locally built image). Consumers — omp's
 # TTS device, OpenWebUI, open-notebook — address a VOICE BY NAME
-# (voice=narrator1, or the harness' voice_id=narrator1) and never see a path:
+# (voice=wilwheaton, or the harness' voice_id=wilwheaton) and never see a path:
 # a name is real exactly when /voices/<name>/reference.wav exists, and
 # anything else is a 400 that lists the names that are.
 #
@@ -65,7 +65,7 @@ TTS_UPSTREAM_MODEL=voxcpm2
 # name still 400s with the list that exists — this covers the absent case only —
 # and the journal marks such a line `(default)` so an unrequested voice never
 # reads as a requested one. Any pack directory in /home/dank/voices is legal.
-DEFAULT_VOICE=narrator1
+DEFAULT_VOICE=wilwheaton

 # CLI-backend knobs, inert while TTS_BACKEND=omni-http. Empty means "not
 # configured": the endpoint answers a 500 that says so instead of running
@@ -165,6 +165,18 @@ if [ -n "$NPU_MODELS" ]; then
        echo "frank-dragoon: $DEV_ACCEL missing — the NPU lane needs the amdxdna node (is iommu=pt still on the boot line?)" >&2
        exit 1
    fi
+    # Existence is not access. This endpoint runs rootless, so the container's
+    # namespace maps exactly one uid/gid (dank) and a root:render 0660 node reads
+    # `nobody:nogroup` inside it — the engine then dies naming the node itself
+    # (measured 2026-10-19). --group-add keep-groups cannot help: an unmapped gid
+    # cannot be granted into a namespace that does not map it, verified with and
+    # without the flag from a shell already in `render`. The door is the udev mode.
+    # NB: `[ -o FILE ]` is NOT "other-writable" — it means "exists", and `-o` is
+    # also the OR operator inside test — so the permission bit is read directly.
+    if [ $(( 8#$(stat -c '%a' "$DEV_ACCEL") & 8#2 )) -eq 0 ]; then
+        echo "frank-dragoon: $DEV_ACCEL is $(stat -c '%a %U:%G' "$DEV_ACCEL") — a rootless container cannot open it; install dankclaw/udev/98-halogen-npu-access.rules and re-trigger udev (sudo udevadm control --reload-rules && sudo udevadm trigger --sysname-match=accel0)" >&2
+        exit 1
+    fi
    for M in ${NPU_MODELS//,/ }; do
        if [ ! -d "$NPU_DIR/$M" ]; then
            echo "frank-dragoon: NPU model $M not staged in $NPU_DIR;" >&2
@@ -0,0 +1,55 @@
+---
+type: reference
+title: Domain Docs
+description: "Consumer rules for GLOSSARY.md and docs/adr/, and where this vault out-ranks them on operational facts."
+tags: [agent, glossary, adr, documentation]
+timestamp: 2026-10-20T00:00:00Z
+---
+
+# Domain Docs
+
+How the engineering skills (`/grill-with-docs`, `/domain-modeling`,
+`/improve-codebase-architecture`, `/to-spec`) should consume this repo's design
+vocabulary when exploring the codebase.
+
+## Before exploring, read these
+
+- **`GLOSSARY.md`** at the repo root
+- **`docs/adr/`**: read ADRs that touch the area you're about to work in
+
+If either doesn't exist, **proceed silently**. Don't flag their absence; don't
+suggest creating them upfront. `/domain-modeling` (reached via
+`/grill-with-docs` and `/improve-codebase-architecture`) creates them lazily
+when terms or decisions actually get resolved. As of 2026-10-20 neither exists;
+this page is the pointer, not a promise.
+
+## Ownership against this vault
+
+This repo already carries `serverAppConfigs-docs/` — the operational knowledge
+base and, per repo law, the owner of operational facts (endpoint shapes, Eras,
+Laws, runbooks). `GLOSSARY.md` / `docs/adr/` hold **design vocabulary and
+decisions**. Where a term is defined in both, the vault page that owns the fact
+wins — one fact, one owner. A glossary entry restating an endpoint number is
+the bug, not the second source.
+
+## Use the glossary's vocabulary
+
+When output names a domain concept (an issue title, a refactor proposal, a
+hypothesis, a test name), use the term as defined in `GLOSSARY.md`. Don't drift
+to synonyms the glossary explicitly avoids.
+
+If the concept you need isn't in the glossary yet, that's a signal: either
+you're inventing language the project doesn't use (reconsider) or there's a
+real gap (note it for `/domain-modeling`).
+
+## Flag ADR conflicts
+
+If output contradicts an existing ADR, surface it explicitly rather than
+silently overriding:
+
+> _Contradicts ADR-0007 (…), but worth reopening because…_
+
+## Related
+
+- [[Issue Tracker]] — where specs and tickets built on this vocabulary are published.
+- [[Agent Docs and Edits]] — the edit-safety rules that apply while touching any doc in this checkout.
@@ -0,0 +1,85 @@
+---
+type: reference
+title: Issue Tracker
+description: "How agent skills read and write Gitea issues here: credential-safe curl, label IDs, and wayfinding without sub-issues."
+tags: [agent, gitea, issues, triage, api]
+timestamp: 2026-10-20T00:00:00Z
+---
+
+# Issue Tracker
+
+Issues and specs for this repo live as Gitea issues in `dank/serverAppConfigs`
+on `http://192.168.0.143:3000` (Dankclaw). There is no `gh`/`glab` CLI here —
+the door is the GitHub-compatible REST API driven with `curl`. The
+mattpocock-lineage skills (`/triage`, `/to-tickets`, `/to-spec`, `/wayfinder`)
+read this page as their tracker contract. The git workflow around issues and
+PRs (branches, PR-only merges, attribution footer) is owned by
+[[Deployment Flow]]; this page owns only the API shape.
+
+## Auth
+
+The token lives in the git credential file `~/.config/secrets/gitea`: one
+line in git-credential URL form, `http://user:TOKEN@192.168.0.143`. Extract it
+without echoing:
+
+```bash
+tok=$(sed -nE 's#^http://[^:/@]+:([^@]+)@192\.168\.0\.143.*#\1#p' ~/.config/secrets/gitea | head -1)
+```
+
+**A token on a command line is world-readable in `/proc/<pid>/cmdline`** (the
+same leak class AGENTS.md records for HF tokens). Never pass it with `-H` —
+put it in a mode-600 curl config and pass `-K`:
+
+```bash
+cfg=$(mktemp); chmod 600 "$cfg"
+printf 'header = "Authorization: token %s"\n' "$tok" > "$cfg"
+# ... every curl below uses -K "$cfg"; rm -f "$cfg" when done
+```
+
+**GETs also accept HTTP Basic; writes do not** — a POST/PATCH with Basic auth
+is refused with `422 {"message":"token is required"}` (measured 2026-10-20).
+Use the `Authorization: token` header for everything.
+
+## Conventions
+
+`$API=http://192.168.0.143:3000/api/v1/repos/dank/serverAppConfigs`
+
+- **Create an issue**: `curl -s -K "$cfg" -X POST -H 'Content-Type: application/json' -d "$(jq -n --arg t "$TITLE" --rawfile b "$BODY" '{title:$t, body:$b}')" $API/issues`
+- **Read an issue**: `curl -s -K "$cfg" $API/issues/<number>` (fields: `number`, `title`, `body`, `labels[].name`, `state`)
+- **List issues**: `curl -s -K "$cfg" "$API/issues?state=open&limit=50"` — add `&labels=<id>` to filter
+- **Comment**: `curl -s -K "$cfg" -X POST -d "$(jq -n --rawfile b "$BODY" '{body:$b}')" $API/issues/<number>/comments`
+- **Apply labels**: `POST $API/issues/<number>/labels` with `{"labels":[<label-ID numbers>]}` — Gitea wants numeric label **IDs**, not names; resolve them from `GET $API/labels` (see [[Triage Labels]] for which labels exist)
+- **Remove a label**: `DELETE $API/issues/<number>/labels/<label-id>`
+- **Close**: `PATCH $API/issues/<number>` with `{"state":"closed"}`
+- **Blocking relationships**: this instance has no native sub-issues or dependencies — put `Part of #<parent>` / `Blocked by: #<n>, #<n>` lines at the top of the child body. A ticket is unblocked when every listed blocker is closed.
+- **Attribution footer (repo law)**: every issue body you open ends with `cwd: <working directory> · session: <omp session id>` — full rule in [[Deployment Flow]].
+
+**PRs as a request surface: no.** Issues and PRs have separate number spaces on
+Gitea; a bare `#42` is an issue, a PR is spelled "PR #42".
+
+## When a skill says "publish to the issue tracker"
+
+Create a Gitea issue as above.
+
+## When a skill says "fetch the relevant ticket"
+
+`GET $API/issues/<number>` and read `title` + `body` + `labels[].name` +
+comments.
+
+## Wayfinding operations
+
+Used by `/wayfinder`. The **map** is a single issue labelled `wayfinder:map`,
+holding the Notes / Decisions-so-far / Fog body.
+
+- **Map**: create an issue, apply the `wayfinder:map` label (create it via `POST $API/labels` `{"name":"wayfinder:map","color":"#6f42c1"}` if missing).
+- **Child ticket**: an issue carrying `Part of #<map>` at the top of its body, plus a `wayfinder:<type>` label (`research`/`prototype`/`grilling`/`task`).
+- **Blocking**: the `Blocked by:` body line (see Conventions).
+- **Frontier**: list open issues (`?state=open`), drop any with an open `Blocked by` target or an assignee; lowest `number` wins.
+- **Claim**: `POST $API/issues/<n>/assignees` with `{"assignees":["dank"]}` — actually assign it or it stays in the frontier; the session's first write.
+- **Resolve**: post the answer as a comment, `PATCH` to `closed`, then update the map issue's body (GET → edit Decisions-so-far → PATCH) with a context pointer.
+
+## Related
+
+- [[Triage Labels]] — the role strings these operations apply.
+- [[Domain Docs]] — what a skill must read before touching the codebase.
+- [[SSH and Remote Access]] — reaching the host when an issue needs live evidence.
@@ -0,0 +1,39 @@
+---
+type: reference
+title: Triage Labels
+description: "The five canonical triage roles mapped to this repo's Gitea label strings, plus the numeric-ID quirk."
+tags: [agent, gitea, triage, labels]
+timestamp: 2026-10-20T00:00:00Z
+---
+
+# Triage Labels
+
+The mattpocock-lineage skills speak in terms of five canonical triage roles.
+This page maps those roles to the actual label strings used in this repo's
+tracker ([[Issue Tracker]] owns the API shape).
+
+| Label in mattpocock/skills | Label in our tracker | Meaning                                  |
+| -------------------------- | -------------------- | ---------------------------------------- |
+| `needs-triage`             | `needs-triage`       | Maintainer needs to evaluate this issue  |
+| `needs-info`               | `needs-info`         | Waiting on reporter for more information |
+| `ready-for-agent`          | `ready-for-agent`    | Fully specified, ready for an AFK agent  |
+| `ready-for-human`          | `ready-for-human`    | Requires human implementation            |
+| `wontfix`                  | `wontfix`            | Will not be actioned                     |
+
+When a skill mentions a role (e.g. "apply the AFK-ready triage label"), use the
+corresponding label string from this table. Edit the "Label in our tracker"
+column to match whatever vocabulary you actually use.
+
+## Instance quirks (Dankclaw Gitea)
+
+- Labels are applied by **numeric ID**, never by name: `POST /issues/<n>/labels`
+  with `{"labels":[<id>]}`; resolve names via `GET /labels`.
+- The repo also carries the pre-existing `human-only` label from the manual era
+  (e.g. the podcast-destination issues). `ready-for-human` maps to itself; the
+  `(HUMAN ONLY)` title prefixes on existing issues stay as they are — the label
+  is what the skills read.
+- An issue with **no** triage label is simply untriaged; absence is not a state.
+
+## Related
+
+- [[Issue Tracker]] — the auth-safe curl shape that applies these labels.
@@ -6,6 +6,9 @@ docs get edited safely, and how the agent's home is deployed.
 - [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
 - [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
 - [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
+- [[Issue Tracker]] — How agent skills read and write Gitea issues here: credential-safe curl, label IDs, and wayfinding without sub-issues.
+- [[Triage Labels]] — The five canonical triage roles mapped to this repo's Gitea label strings, plus the numeric-ID quirk.
+- [[Domain Docs]] — Consumer rules for GLOSSARY.md and docs/adr/, and where this vault out-ranks them on operational facts.

 ## Related

@@ -8,7 +8,7 @@ tags:
  - netavark
  - rootless
  - dankclaw
-timestamp: 2026-09-02
+timestamp: 2026-10-19
 ---

 # Container Network Namespaces
@@ -106,8 +106,8 @@ private `systemd-open-notebook`), each address aimed at a live listener:
 | `host.containers.internal:7771` (`169.254.1.2`) | **200** | the gateway alias is the one address that carries a rootless container to a host-netns listener. Why `hermes-dashboard` timed out on this alias in 2026-09-02 while `open-notebook` answers today is **not established**; the difference to check first is the second network and where the default route lands |
 | `192.168.0.235:7770` (`frank-archon`) | **200** | a different machine: plain routing, no DNAT in the path |

-The consequence that changes designs: **a rootless container's menu of local model
-endpoints is `frank-tts` (container DNS, same network) and `frank-archon` (LAN
+The consequence that changed designs: **a rootless container's menu of local model
+endpoints was only `frank-tts` (container DNS, same network) and `frank-archon` (LAN
 address on another host).** A `pasta` endpoint has no address an app container can
 use: the LAN address is refused, and the one alias that answers is link-local,
 which some applications refuse to store at all ([[Open Notebook]] meets exactly
@@ -127,6 +127,15 @@ API port becomes name-resolvable, which is the same trust the published 7770
 already carries on the LAN. `frank-zealot` (7771) stays a host-netns binary and
 remains unreachable except through the link-local alias.

+The move is verified from the consumer side, so the table above reads as the
+**pre-move** census and not as today's menu: inside `open-notebook`,
+`getent hosts frank-dragoon-halogen` → `10.89.1.39`, `GET :8731/v1/models`
+lists both served models, and the engine's log carries the app's
+`POST /v1/embeddings 200` from that address (2026-10-19) — see
+[[Open Notebook]] for what runs over it. What the table still owns is the
+pasta rule itself: nothing about pasta's forwarding changed, the endpoint was
+moved out of it.
+
 ## Operational rules

 - Health checks for a bridge-network service must target a **published** port (or
@@ -21,7 +21,7 @@ never learn a path.
 **Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`
 both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
 the same change. Probes on the deployed endpoint: `/health` answers
-`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`
+`{"status":"ok","backend":"omni-http","voices":["dr.kern","garrickhagon","jeffhays","vaati","wilwheaton"]}`
 and two different voices rendered the same sentence into **different** bytes
 (24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
 pass-through of one default. The operator harness routes its `speech` role here;
@@ -42,8 +42,8 @@ render needs the app installed, not this endpoint.

 ## Naming a voice

-A voice **is** a directory name. `voice=narrator1` resolves to
-`/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
+A voice **is** a directory name. `voice=wilwheaton` resolves to
+`/voices/wilwheaton/reference.wav`, and that file — not a manifest, not a config
 entry — is what makes the name real. omp's TTS device may send `voice_id`; the
 shim accepts either. A call that sends **neither** — the harness's normal shape,
 because no harness setting carries a voice for this transport — speaks
@@ -59,10 +59,11 @@ loudly instead of letting it half-work.

 Names are user-visible: an unknown voice comes back as
 `400 unknown voice 'x'; available voices: …`, and that list is the honest
-census. The podcast speakers are `narrator1`, `narrator2`, `narrator3` —
-[[Open Notebook]] is configured with exactly those three, so renaming them
-breaks episode rendering, not just speech. A personal voice gets the person's
-name, lowercase.
+census. As of 2026-10-20 the live packs are `dr.kern`, `garrickhagon`,
+`jeffhays`, `vaati`, `wilwheaton`. [[Open Notebook]]'s `dank-duo` profile speaks
+`wilwheaton` (Frank) and `vaati` (Comrade), and `DEFAULT_VOICE` is `wilwheaton`.
+Renaming a pack that a speaker profile names breaks episode rendering, not just
+speech. A personal voice gets the person's name, lowercase.

 ## Voice packs

@@ -221,5 +222,5 @@ harness bundle and measured the day of the flip:
 | Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
 | First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
 | The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
-| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
+| A consumer rejects `wilwheaton` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
 ---
@@ -45,6 +45,8 @@ open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook))

 The app's own login stays on behind the chain. It is not redundant: publishing the API on loopback does **not** hide it from this host's other containers, which can dial `open-notebook:5055` by name on `dankclaw_default`. The chain decides *who from outside* gets in; `OPEN_NOTEBOOK_PASSWORD` decides what anything that already reached the port may do.

+**Single tenant — there are no user accounts, so this is an admin-side tool.** Upstream ships one global password and nothing else: `PasswordAuthMiddleware` compares `Bearer <OPEN_NOTEBOOK_PASSWORD>`, and the SurrealDB schema has no user/account/owner table (verified in the running image, 2026-10-20). One shared password therefore means **one shared workspace** — the guard chain can gate the door but cannot partition the data, and Authentik identity is never consumed. Multi-user is *approved but unbuilt* upstream (issue **#712**, "design phase, targeted post-v2"); OIDC/SSO was declined for now (**#607** closed). Real per-person isolation means **one deployment per person**, not a flag to flip. For that reason the homepage lists open-notebook under **admin**, not as a user-facing app.
+
 ## What the image actually runs

 One container, three processes under supervisord (`supervisord.conf` upstream): `uvicorn api.main:app --port 5055` (`:12`), the job worker (`:25`), and the Next.js frontend (`:39`). Two consequences the config depends on:
@@ -64,9 +66,11 @@ Credentials live in the database, encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`,
 |--------|-------|
 | Credential `frank-tts` | modality `text_to_speech`, base URL `http://frank-tts:7772/v1`, no API key (the shim does not authenticate) |
 | Credential `frank-archon` | modality `language`, base URL `http://192.168.0.235:7770/v1` |
+| Credential `frank-dragoon NPU` | modality `embedding`, base URL `http://frank-dragoon-halogen:8731/v1`, dummy API key (the lane does not authenticate) — a container name on `dankclaw_default`, which is the address shape that clears the URL guard below |
 | Model row `voxcpm2` | type `text_to_speech` on the frank-tts credential. The name is decorative: frank-tts ignores the request's `model` field and synthesizes whatever `voice` names ([[Frank TTS]]) |
-| Model row `frank-archon` | type `language`; `PUT /api/models/defaults` points chat, transformation and large-context at it. **The embedding default stays null — nothing in the fleet serves `/v1/embeddings`** |
-| Speaker profile `dank-duo` | two speakers, Frank → `voice_id: narrator1`, Comrade → `narrator2`, `voice_model` = the TTS row |
+| Model row `frank-archon` | type `language`; `PUT /api/models/defaults` points chat, transformation and large-context at it |
+| Model row `qwen3-embedding-0.6b` | type `embedding` on the NPU-lane credential; `default_embedding_model` points at it, so every note save embeds through the XDNA2 lane ([[frank-dragoon]]) |
+| Speaker profile `dank-duo` | two speakers, Frank → `voice_id: wilwheaton`, Comrade → `vaati`, `voice_model` = the TTS row |
 | Episode profile `dank-daily` | `num_segments: 4`, `speaker_config` = `dank-duo`, outline and transcript LLM = the archon row, briefing that ends "Never invent numbers." |

 Order that works over REST: credentials → model rows (`POST /api/models` with `credential: <credential id>`) → `PUT /api/models/defaults` → speaker profile (`voice_model: <model id>`) → episode profile (`speaker_config` / `outline_llm` / `transcript_llm`). `POST /api/models/{id}/test` proves each row: the language row answered with a real completion from frank-archon; the TTS row reached frank-tts and came back `unknown voice 'alloy'; available voices: narrator1, narrator2, narrator3` — the app's own test path hard-codes an OpenAI voice, so that message *is* the proof the leg works.
@@ -78,11 +82,16 @@ Order that works over REST: credentials → model rows (`POST /api/models` with
 - `192.168.0.143:7771` (frank-zealot, a host-netns listener) and `192.168.0.143:7770` (frank-dragoon, a **root**-published port) both refuse the connection; `host.containers.internal:7771` answers. Full census: [[Container Network Namespaces]].
 - The app refuses to **store** that one working address: its URL guard rejects any base URL that resolves inside link-local `169.254.0.0/16` (`open_notebook/utils/url_validation.py`, "Link-local addresses are not allowed (metadata endpoints)"), unconditionally — no env var touches it, and `AddHost=` cannot dodge it because the check runs on the resolved address.

-`http://192.168.0.235:7770/v1` — another machine, plain RFC1918 — is the only fleet LLM address that passes both tests. Cost: frank-archon runs `--max-num-seqs 2` and the operator session itself lives there, so a render is a slow-down for whoever else is talking to that endpoint, not a stall; `ESPERANTO_LLM_TIMEOUT=180` covers an outline or transcript call comfortably.
+`http://192.168.0.235:7770/v1` — another machine, plain RFC1918 — was, when this was first written, the only fleet LLM address that passed both tests; a container name on `dankclaw_default` passes them as well (the embedding credential below is stored that way), but frank-archon stays the scriptwriter for the reason in the next paragraph. Cost: frank-archon runs `--max-num-seqs 2` and the operator session itself lives there, so a render is a slow-down for whoever else is talking to that endpoint, not a stall; `ESPERANTO_LLM_TIMEOUT=180` covers an outline or transcript call comfortably.

-**frank-dragoon can write transcripts, but only greedily — and open-notebook cannot ask for greedy.** Measured 2026-10-19 against the live endpoint: `response_format` as `json_schema` or `json_object` returns schema-valid JSON at `temperature: 0` and a **400** at any sampled temperature — halogen decodes greedy under a schema, and because its sampler default is server-side, omitting `temperature` is sampled, not greedy. That 400 is the trap: esperanto's `is_json_schema_unsupported_error` matches the words *not implemented* in it, so `_SchemaFallbackModel` in `podcast_creator/nodes.py` reads it as "this endpoint has no schema support", downgrades to `json_object`, gets the same 400, and **fails the run** — esperanto's own rescue (drop `response_format` entirely) fires only on the different message `'response_format.type' must be 'json_schema'`. The full law is on [[frank-dragoon]]. And the fix is not reachable from here: a model record is `{credential, name, provider, type}` and a model response adds only timestamps (`GET /openapi.json`, `ModelCreate`/`ModelResponse`) — **there is no per-row config field**, no way to attach `temperature: 0` to one row from the UI or the API. Making `frank-dragoon` the writer would therefore need something in the path that rewrites request bodies, which is a heavier tenant than the contention it removes. **frank-archon stays the scriptwriter.** Reachability would be a separate problem anyway: `frank-dragoon` is not on `dankclaw_default` (the serve script passes no `--network`), so nothing on this network has a name for it, and the one address that does answer — `host.containers.internal:7770` — is the link-local the guard refuses to store ([[Container Network Namespaces]]).
+**frank-dragoon can write transcripts, but only greedily — and open-notebook cannot ask for greedy.** Measured 2026-10-19 against the live endpoint: `response_format` as `json_schema` or `json_object` returns schema-valid JSON at `temperature: 0` and a **400** at any sampled temperature — halogen decodes greedy under a schema, and because its sampler default is server-side, omitting `temperature` is sampled, not greedy. That 400 is the trap: esperanto's `is_json_schema_unsupported_error` matches the words *not implemented* in it, so `_SchemaFallbackModel` in `podcast_creator/nodes.py` reads it as "this endpoint has no schema support", downgrades to `json_object`, gets the same 400, and **fails the run** — esperanto's own rescue (drop `response_format` entirely) fires only on the different message `'response_format.type' must be 'json_schema'`. The full law is on [[frank-dragoon]]. And the fix is not reachable from here: a model record is `{credential, name, provider, type}` and a model response adds only timestamps (`GET /openapi.json`, `ModelCreate`/`ModelResponse`) — **there is no per-row config field**, no way to attach `temperature: 0` to one row from the UI or the API. Making `frank-dragoon` the writer would therefore need something in the path that rewrites request bodies, which is a heavier tenant than the contention it removes. **frank-archon stays the scriptwriter.** Reachability stopped being the objection on 2026-10-19 — the endpoint joined `dankclaw_default`, which is exactly why the embedding lane below is dialable by name — but no network change reaches the no-per-row-config wall.

-**No embedding model is deployed, and saving a note is where that shows up.** Every note save submits an `embed_note` command that fails in the worker with `No embedding model configured. Please configure one in the Models section.` The note itself is stored and readable; semantic search, source ingestion and notebook-context questions stay dead until something serves `/v1/embeddings`. Podcast generation from `content` does not need embeddings, which is why renders work today. The cheapest door is **halogen's NPU lane, not a new endpoint**: `qwen3-embedding-0.6b` is already in the catalog of the image `frank-dragoon` runs and answers `/v1/embeddings` on the same port (1024 unit-length floats; the instruct prefix goes in the text, and `check_embedding_ctx_length=False` for LangChain). What enabling it costs — the accel device, the host XRT mount, and the fabric-clock hold that GPU+NPU co-run requires — is on [[frank-dragoon]]; after that, the embedding row still has to clear the reachability rule above.
+**Embeddings run on the XDNA2 lane — provisioned 2026-10-19.** `qwen3-embedding-0.6b` answers `/v1/embeddings` at `http://frank-dragoon-halogen:8731/v1`, the flash container's own bridge address; a container name on `dankclaw_default` resolves to `10.89.1.39`, plain RFC1918, so it clears the URL guard that ruled out `host.containers.internal` above. The lane's shape (one program, the token ceiling, what it costs the Flash model) is owned by [[frank-dragoon]]; the two shapes that bite on the app side are open-notebook's own:
+
+- **`PUT /api/models/defaults` stores the record ID, not the name.** `"default_embedding_model": "qwen3-embedding-0.6b"` is accepted and echoed back, and then every embed dies in the worker with `Model with ID qwen3-embedding-0.6b not found`. The value has to be the row's `model:…` id — the other defaults in that same object are ids, so mirror them.
+- **`POST /api/embed` answers 500 while succeeding.** Upstream's `EmbedResponse.command_id` declares `str` and the worker hands it a `RecordID`, so FastAPI raises on serialization *after* the vector is written. The truth is the worker line `Successfully embedded note … in 0.19s`, never the HTTP code. Saving a note (`POST /api/notes`) submits `embed_note` by itself, so that route is only needed to backfill old rows.
+
+Proof from the app's own paths, 2026-10-19: `POST /api/models/{id}/test` → `Embedding dimensions: 1024`; two notes with nothing in common, then `POST /api/search` with `type: "vector"` (the enum is `text` or `vector`; the score field is `similarity`) — "a yellow waterbird beside a glowing beacon tower" put the grebe note at **0.659** and the boiler note at 0.292, and "quarterly depreciation schedule for industrial boilers" inverted it to **0.906** / 0.318. Embed cost 0.19–0.43 s per note. Semantic search, source ingestion and notebook-context questions are live; podcast generation never needed any of it.

 **Secrets never go in a command line.** The API's bearer token *is* `OPEN_NOTEBOOK_PASSWORD`, and a `-H "Authorization: Bearer …"` argument is world-readable in `/proc/<pid>/cmdline`; put it in a mode-600 curl config and pass `-K`:

@@ -99,13 +108,13 @@ A **speaker profile** carries one to four speakers (`docs/3-USER-GUIDE/creating-

 | Speaker | Voice ID to type |
 |---------|------------------|
-| Speaker 1 | `narrator1` |
-| Speaker 2 | `narrator2` |
-| Speaker 3 | `narrator3` |
+| Speaker 1 | `wilwheaton` |
+| Speaker 2 | `vaati` |
+| Speaker 3 | `dr.kern` |

 Those are the pack names frank-tts resolves from `/home/dank/voices` (`scripts/frank-tts.env` header: a voice is real exactly when `/voices/<name>/reference.wav` exists). Two traps, both from upstream's shipped defaults:

- The three seeded speaker profiles carry **OpenAI voice names** (`nova`, `alloy`, …) and **no voice model at all** — they show a "Setup required" badge, and generation fails with a voice error until they are edited (`creating-podcasts.md:13`, `:33`; `local-tts.md:89`). Add a `narrator1`/`narrator2` pair (or a third/fourth) rather than editing the seeded ones in place.
+- The three seeded speaker profiles carry **OpenAI voice names** (`nova`, `alloy`, …) and **no voice model at all** — they show a "Setup required" badge, and generation fails with a voice error until they are edited (`creating-podcasts.md:13`, `:33`; `local-tts.md:89`). Add a `wilwheaton`/`vaati` pair (or a third/fourth) rather than editing the seeded ones in place.
 - Two speakers with the same voice are hard to follow (`creating-podcasts.md:89`). Distinct packs per speaker is the point of the 1–4 shape.

 ## Two buttons that fail against frank-tts, and why that is cosmetic
@@ -164,6 +173,7 @@ podman pull docker.io/lfnovo/open_notebook:v1-latest@sha256:54b2025a4a4853c188f7
 - Local measurements (2026-10-18, workstation, pinned digests): `surreal start --help` (credential and bind options carry `[env: SURREAL_*]`); running the pinned SurrealDB digest with env-only credentials (credential init in the log, correct login authenticates, wrong password refused, `/surreal is-ready` → `OK` exit 0, named volume owned by the mapped uid); the v1-compatible Quadlet generator dry-run of all three unit files (two `--env-file` in order, two `--network` in order, named volume + host bind, both health probes, `--pull never`)
 - Host evidence (Dankclaw, 2026-10-18, read-only): `ss -ltn` (`127.0.0.1:5055` held by seerr, `0.0.0.0:7771` a host listener, nothing on 7772/8502/5056/8000), `podman ps -a` (no open-notebook container, `dankclaw_default` present), subnet registry from `dankclaw/open-webui/quadlet/open-webui.network`
 - Deploy + provisioning (Dankclaw, 2026-10-19): first-boot log — migrations 1→25, `Credentials were provided, and no root users were found. The root user 'open_notebook' will be created`, `api`/`worker`/`frontend` all RUNNING under supervisord; `GET :8502/config` → the `API_URL` value; `GET /api/providers` (the `openai_compatible` modality list); `GET /openapi.json` schemas for `CreateCredentialRequest` / `ModelCreate` / `SpeakerProfileCreate` / `PodcastGenerateRequest`; the POSTs listed above; the note-save worker error `No embedding model configured.`
+- Embedding lane (Dankclaw, 2026-10-19): `POST /api/credentials` → `credential:7fdqz09tt0c4xp68x7cm`, `POST /api/models` → `model:p3ki598dydfinu93tbny`, `POST /api/models/{id}/test` → `Embedding dimensions: 1024`; worker lines `Successfully embedded note … in 0.43s` / `0.19s`; the name-not-id failure `Model with ID qwen3-embedding-0.6b not found`; the `/api/embed` 500 body (`EmbedResponse.command_id`, `input_value=RecordID(table_name=command…)`); `POST /api/search` `similarity` 0.6590/0.2921 and 0.9059/0.3176 for the two-note pair; `podman logs frank-dragoon-halogen` showing `POST /v1/embeddings 200` from `10.89.1.21`. Both probe notes deleted afterwards.
 - First render (2026-10-19): job `command:9rdojq5ne7dj9caicq20`, `execution_time: 301.21 s`, `episode:7ft03thj98sn0ek9xf29`; `episodes/d5fed29b-33e9-40ac-b46a-e0537c56c256/clips/0001.mp3 … 0023.mp3`; `…/audio/d5fed29b-….mp3` = 11,957,420 bytes, mp3 24 kHz mono, `ffprobe` duration 747.31 s; `GET /api/podcasts/episodes/{id}/audio` → 200 `audio/mpeg` with the bearer, 401 without.
 - Upstream code read inside the pinned image (2026-10-19): `frontend/next.config.ts:26-43` (the `/api/*` rewrite and the one-port claim), `frontend/src/app/config/route.ts:122-173` (`/config` priority order; autodetect appends `:5055`), `frontend/src/app/api/_sse-proxy.ts` (SSE proxied with `no-transform`), `open_notebook/utils/url_validation.py` (link-local refusal, `is_blocked_ip`, no environment override anywhere in the file).
 - Guard chain (2026-10-19): provider `notebook-proxy` created through `podman exec -i authentik-server ak shell` with the flows copied from `embedded-outpost` (pk 1), Application `Open Notebook`/`notebook-guard`, embedded-outpost provider set 9 → 10; afterwards the outpost logged zero `Failed to fetch outpost from API` and `seerr`/`grafana` still answered 200. Before the route existed, `notebook.franksdank.cloud` resolved to nothing (`000`).
@@ -290,9 +290,12 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.

 ## Text-to-speech (frank-tts)

-TTS is the fleet's one OpenAI-compatible speech endpoint —
-`http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
-in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
+TTS is the fleet's one OpenAI-compatible speech endpoint. The config DB
+`audio.tts.*` is set to engine `openai`, `audio.tts.openai.api_base_url`
+`http://192.168.0.143:7772/v1`, `audio.tts.voice` `wilwheaton` (re-keyed
+2026-10-20 from the retired `narrator1`; 7772 is a published port so
+`host.containers.internal` also reaches it, but the LAN IP is the canonical
+on-host form). That
 port is the rootless quadlet `frank-tts` (this repo:
 `dankclaw/voiceservice/quadlet/frank-tts.container`
 `scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
@@ -306,8 +309,8 @@ ships an in-page Kokoro implementation (chunk carrying a frozen voice table
 `onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
 setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
 `browser-kokoro` is chosen the page synthesizes locally and rejects any
-voice outside its fixed pack — so `narrator1` fails with
-`Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
+voice outside its fixed pack — so `wilwheaton` fails with
+`Voice: "wilwheaton" not found. Should be one of: af_heart, …` **without a
 single request reaching the server**. Personal audio settings are
 browser: no `user.settings` row and no container env carries an audio
 override, yet the toast still appears ⇒ the choice lives in that browser's
@@ -3,70 +3,111 @@ type: reference
 description: "Alloy is Grafana's log and telemetry shipper."
 tags:
  - infrastructure
-timestamp: 2026-09-11
+timestamp: 2026-10-20
 ---

 # Alloy

-Alloy is Grafana's log and telemetry shipper. It replaces Promtail (end of life as of 2026-03-02, https://grafana.com/docs/loki/latest/send-data/promtail/). It is wired to forward container and journal logs to Loki, and every unit in the path is healthy — but as of 2026-10-18 it is delivering **zero** log lines. See "Delivery state" below.
+Alloy is Grafana's log and telemetry shipper. It replaces Promtail (end of life
+as of 2026-03-02, https://grafana.com/docs/loki/latest/send-data/promtail/). It
+forwards the host journal — including the rootless Quadlet containers' logs —
+to Loki. Unlike its observability siblings it runs as a **root system service on
+the host network**, because a rootless container cannot read the journal (see
+*Why root* below).

-**Host:** Dankclaw (one instance per host) — **Port:** 12345 (loopback) — **Docs:** https://grafana.com/docs/alloy/latest
+**Host:** Dankclaw (one instance per host) — **UI/health:** 127.0.0.1:12345
+(loopback) — **Docs:** https://grafana.com/docs/alloy/latest

 ## Facts

 | Item | Value |
 |------|-------|
-| Image | `franksdank/alloy:latest` (local build: upstream + a static busybox for the healthcheck) |
-| Published | `127.0.0.1:12345` → 12345 (its own UI/health port) |
-| Unit | `dankclaw/observability/quadlet/alloy.container` → `alloy.service` |
-| Config | `shared/alloy/config.alloy`, installed to `/opt/alloy/config/config.alloy` on every deploy (`scripts/deploy-remote.sh:281-284`) |
-| Inputs | host journald socket, podman container logs, `/var/log` |
-| Output | `http://loki:3100/loki/api/v1/push` (`shared/alloy/config.alloy:13-17`) |
-| Successor to | Promtail (EOL 2026-03-02, https://grafana.com/docs/loki/latest/send-data/promtail/) |
-| Boot persistence | `Restart=always`/`RestartSec=10` + `WantedBy=default.target` (`dankclaw/observability/quadlet/alloy.container:58-63`); installed at `~/.config/containers/systemd/alloy.container`, reporting `generated` + `active` (`systemctl --user is-enabled|is-active alloy` 2026-10-18) |
-| Delivery | **Configured and healthy, currently delivering zero data** — see "Delivery state" |
+| Image | `localhost/franksdank/alloy:latest` — built rootless, then mirrored into the root image store by the deployer (a root build fails on DNS in this host's build container) |
+| Unit | `dankclaw/observability/quadlet/alloy.container` → system `alloy.service`, installed at `/etc/containers/systemd/` (root manager) |
+| Config | `shared/alloy/config.alloy` → `/opt/alloy/config/config.alloy` on every deploy |
+| Network | `host` — required so it can reach Loki's `127.0.0.1:3100` publish |
+| UI | `--server.http.listen-addr=127.0.0.1:12345` (loopback; nothing off-host reaches it) |
+| Inputs | host journal: `system.journal` + `user-*.journal` (root reads both) |
+| Output | `http://127.0.0.1:3100/loki/api/v1/push` |
+| Boot persistence | `Restart=always`/`RestartSec=10` + `WantedBy=multi-user.target`; boot persistence is the `[Install]` section (generated system units cannot be `enable`d here) |
+| Successor to | Promtail (EOL 2026-03-02) |
+| Deploy status | root cutover built and tested in-repo (2026-10-20); the **live** instance is still the old rootless unit until `./scripts/deploy.sh dankclaw --service alloy` runs |

-## Why Alloy and not Promtail
+## Why root (the load-bearing decision)

-Promtail reached end-of-life on 2026-03-02 and Grafana Labs directs new installs to Alloy. This stack uses Alloy from day one: no Promtail config or unit has ever existed in this repository (`git log -S promtail` over the whole history returns no commit that added one; the compose-era service of the same name was already Alloy — `dankclaw/observability/quadlet/alloy.container:6-7`).
+A rootless Alloy delivered **zero** lines, silently and with a green health
+check. Two independent causes, both proven on the host 2026-10-20:
+
+1. **The journal is not readable by a rootless container.** The files are
+   `root:systemd-journal 0640` (`group:adm`, `other::---`). Even bind-mounted, a
+   rootless container has no supplementary groups, so opening them returns
+   `DENIED`. `loki.source.journal` then sees an empty view and the counter just
+   sits at 0 — no error logged.
+2. **Container logs live in the *user* journal, not `ctr.log`.** The rootless
+   Quadlet containers log via the journald driver into `user-1000.journal`; the
+   old file globs (`/var/lib/containers/.../ctr.log`, `/var/log/containers/*.log`)
+   point at a root-storage layout that does not exist here.
+
+**Root fixes both at once.** As root the container opens every journal file
+under the machine-id directory — `system.journal` *and* `user-1000.journal` —
+so a single `loki.source.journal` carries host services and the rootless
+containers' logs together. Validated with a throwaway root probe before writing
+the unit: **31,825 journal lines read, 26,966 pushed to Loki in 20 s, zero
+drops**, and Loki's label set went from empty to populated.
+
+### The two mounts that make it work
+
+- **`/etc/machine-id` (ro) — load-bearing.** `sd_journal` locates the journal
+  directory by the *local* machine id. Without it the container opens nothing and
+  the journal source reads exactly 0 lines — silently, health green. This single
+  mount is the whole difference between a working and an empty pipeline; do not
+  remove it. (A root probe without it reads 0; with it, 31,825.)
+- **`Network=host`.** Loki publishes `127.0.0.1:3100` (loopback only). No bridge
+  container — rootless or root — can reach the host's loopback, so Alloy must
+  share the host network to push to `127.0.0.1:3100`. Consequence: the UI binds
+  `127.0.0.1` directly (no `PublishPort`), so it stays loopback-only as before.

 ## Configuration

-`shared/alloy/config.alloy` is the source of truth:
-
- Reads journald logs from the host's `/run/systemd/journal/socket`, labelled `job="systemd-journal"` (`shared/alloy/config.alloy:20-26`).
- Reads container logs from two globs: `/var/lib/containers/storage/overlay-containers/*/userdata/ctr.log` (`job="podman"`) and `/var/log/containers/*.log` (`job="containers"`) (`config.alloy:29-35`).
- Forwards everything to `http://loki:3100/loki/api/v1/push` by container name on `dankclaw_default` (`config.alloy:13-17`).
+`shared/alloy/config.alloy` is the source of truth — one `loki.source.journal`
+(labelled `job="systemd-journal"`) forwarding to the loopback `loki.write`
+endpoint. There is deliberately **no** container-file source: container stdout is
+in the journal.

 ## Volumes

 | Container path | Host path |
 |----------------|-----------|
-| `/etc/alloy` | `/opt/alloy/config` |
-| `/run/systemd/journal/socket` | host journald socket (read-only) |
-| `/var/log` | host logs (read-only) |
-| `/var/lib/containers` | podman storage (read-only) |
+| `/etc/alloy` | `/opt/alloy/config` (ro) |
+| `/etc/machine-id` | host machine id (ro) — **required** |
+| `/run/systemd/journal/socket` | host journald socket (ro) |
+| `/var/log/journal` | persistent journal files (ro) |
+| `/run/log/journal` | runtime journal files (ro) |

-## The UI port is published on purpose
+## The UI stays on loopback

-Alloy's UI listens on the container's own `127.0.0.1:12345` by default, and Alloy 1.x refuses to configure the `ui` service from a config block ("service ui does not support being configured", `shared/alloy/config.alloy:8-10`). Under compose that meant **no reachable health endpoint at all**: the homepage status card for alloy read `down` continuously and a deploy had nothing to wait on. The unit therefore passes `--server.http.listen-addr=0.0.0.0:12345` (`dankclaw/observability/quadlet/alloy.container:46`) and publishes `127.0.0.1:12345` (`alloy.container:47`, confirmed by `ss -ltnp` 2026-10-18) — same posture as the other three: reachable from the host and from Caddy's network, and from nothing off-host. Alloy has no authentication, which is exactly why it is not published wider. The homepage card probes that same loopback port (`shared/homepage/services.json:233`).
-
-## Delivery state (2026-10-18): healthy, shipping nothing
-
-The unit is `active`, the container reports `healthy`, the log says `Alloy is ready`, and the pipeline carries no data at all. Measured on 2026-10-18:
-
- `curl -s http://127.0.0.1:12345/metrics` → `loki_source_journal_target_lines_total{component_id="loki.source.journal.journal"} 0` and `loki_source_file_files_active_total{component_id="loki.source.file.containers"} 0`. The journal source logs no error; the counter simply never moves. The journal files it would have to read through the `/var/log` bind are `root:systemd-journal 0640` with an ACL (`ls -l /var/log/journal/64a287d7…/` 2026-10-18).
- Both file globs were refused at start (`journalctl --user -u alloy`, 2026-10-18T08:25:27Z), verbatim: `failed to create source, skipping … component_id=loki.source.file.containers error="failed to tail file, stat failed: stat /var/lib/containers/storage/overlay-containers/*/userdata/ctr.log: permission denied"` and the same message for `/var/log/containers/*.log: no such file or directory`.
- Downstream that means Loki holds zero streams — `GET /loki/api/v1/labels` returns `{"status":"success"}` with no `data` — see [[Loki]].
-
-No fix is claimed here: the unit, the config and the push endpoint are all correct as written, and the gap is source-side access (and a Kubernetes-style glob that does not exist on this host). Anyone closing it should re-check the two counters above, not the health status — health is exactly what hides this failure mode.
+Alloy 1.x refuses to configure the `ui` service from a config block, so the UI
+address is set on the command line instead. Under the old rootless unit that
+meant `0.0.0.0` inside the container plus a published `127.0.0.1:12345`; under
+`Network=host` the listen-addr is `127.0.0.1:12345` directly — the host's
+loopback, reachable from the host and from nothing off-host. Alloy has no
+authentication, which is exactly why it is never published wider. The homepage
+card and the deploy health gate both dial `http://127.0.0.1:12345/-/healthy`.

 ## Gotchas

- The journald socket mount is required for journal scraping.
- Container log paths differ between docker (`/var/lib/docker/containers`) and podman (`/var/lib/containers/storage/...`); point the config at the right one per host.
- One Alloy per host; do not run a single instance that tries to reach both hosts' logs.
- The unit `Wants=`/`After=` `loki.service`: without a reachable Loki the write client fails and the pipeline backs up.
+- **No `/etc/machine-id` mount ⇒ silent zero.** The health check still passes.
+  This is the failure mode that hid the broken pipeline; check
+  `loki_source_journal_target_lines_total` and the Loki label query, never
+  `/healthy`.
+- **`After=loki.service` is gone.** Alloy is on the system manager; Loki is a
+  rootless user unit — a system unit cannot order against a user unit. Alloy
+  retries its push until Loki answers, so ordering is unnecessary.
+- **The image must be in the ROOT store.** The deployer builds rootless and
+  mirrors (`mirror_image_to_root`); a `Pull=never` root quadlet with only a
+  rootless tag fails at create.
+- One Alloy per host; do not run a single instance that tries to reach both
+  hosts' logs.

 ## See Also

@@ -267,6 +267,12 @@ The silent re-auth path was proven end-to-end: after deleting
 `qa-e2e-admin`'s rows, the e2e admin spec ran against the stale storage
 state and passed with no login prompt (2 tests, 2.9 s).

+## Login-session lifetime (how often you re-authenticate)
+
+How long a login stays valid is the **`default-authentication-login` stage's `session_duration`** (`UserLoginStage`) — not a global setting, and not `SESSION_COOKIE_AGE` (that is the *pre-login* session). Raised to **`days=7`** on 2026-10-20 (was stage value `0` = the 24 h framework default). It is an **absolute** window, never sliding: authentik implements `SESSION_SAVE_EVERY_REQUEST` nowhere, so activity does not extend it and no supported knob makes it one.
+
+**`session_duration` is a STRING** in authentik's idiom (`"days=7"`, `"seconds=30"`, `"hours=12"`). A Python `timedelta` stores `"7 days, 0:00:00"`, which `authentik.lib.utils.time.timedelta_from_string` rejects with `ValueError` and login then crashes computing the expiry — so set it with **`scripts/authentik-set-login-session-duration.sh`** (validates the form first). Any non-zero value applies to *every* login (the "remember me" box only adds `remember_me_offset` on top) and a positive `set_expiry` also overrides `SESSION_EXPIRE_AT_BROWSER_CLOSE`, so closing the browser no longer logs you out; **existing sessions keep their old expiry** — delete the user's `authentik_core_session`/`…proxysession` rows and re-login to force it now. This is **DB state, not repo-managed**: a blueprint re-import or rebuild reverts the stage to `0`, so re-run the script after one. The other two login stages (`default-source-*`) drive external-source/enrollment logins and stay at the framework default.
+
 ## Guard-chain flip recipe (adding a guarded vhost)

 Move a public vhost from `reverse_proxy <app>:<port>` onto the chain:
@@ -3,7 +3,7 @@ type: reference
 title: "Observability on Dankclaw"
 description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
 tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
-timestamp: 2026-09-11
+timestamp: 2026-10-20
 ---

 # Observability on Dankclaw
@@ -83,27 +83,37 @@ health endpoint (`/prometheus`: `/-/healthy`, `/alertmanager`: `/-/healthy`
 [[Observability Provisioning]] for what is provisioned as code and how to
 verify it through the APIs.

-## Current state (2026-10-18)
+## Current state (2026-10-20)

 * **Metrics: working.** `curl -s http://127.0.0.1:9090/api/v1/targets` reports
-  eight up targets; the four downs are structural (the `comfyui` and `services`
-  HTML probes, and Gitea's disabled metrics endpoint), not outages. Rule set:
-  4 alert rules + 2 recording rules (`/api/v1/rules`, matching
-  `shared/prometheus/rules/llm-fleet.yml`).
+  eight up targets — the three LLM endpoints (frank-archon on `vllm:*`,
+  frank-dragoon and frank-zealot on `llamacpp:*`), grafana, loki, prometheus
+  and both node exporters. The four downs are structural (the `comfyui` and
+  `services` HTML probes, and Gitea's disabled metrics endpoint), not outages.
+  Two rule groups in `shared/prometheus/rules/llm-fleet.yml`: `llm-fleet`
+  (2 recording + 4 alert, `llamacpp:*`) and `vllm-fleet` (1 recording + 4
+  alert, `vllm:*` — archon queueing, prefix collapse, preemption, p95
+  time-to-first-token), each keyed only to the namespace its runtime exports.
 * **Delivery: working.** Alertmanager's single `signal` receiver webhooks the
  relay, and `journalctl --user -u signal-relay` records successful
  `sent to +12062476336:` lines for every rule in the set.
-* **Dashboards: working.** One provisioned Prometheus datasource (uid
-  `prometheus`) and the `llm-fleet` dashboard, both installed from
-  `shared/grafana/provisioning/`.
-* **Logs: configured, empty.** Alloy ships zero lines (journal counter 0, both
-  file globs refused) and Loki therefore holds zero streams — `/opt/loki/data`
-  is a few hundred KB of WAL/compactor bookkeeping with **no chunk files**
-  (8 files total by `find`, 2026-10-18) and `GET /loki/api/v1/labels` returns
-  no label names.
-  See [[Alloy]] → Delivery state and [[Loki]] → Contents.
-* **Not configured:** no Loki datasource in Grafana, no blackbox exporter (the
-  structural downs have no real uptime signal), no Alertmanager vhost.
+* **Dashboards: working.** Provisioned datasources — Prometheus (uid
+  `prometheus`) and Loki (uid `loki`) — and the `llm-fleet` dashboard, all
+  installed from `shared/grafana/provisioning/`. The Loki datasource is how
+  you read logs: Loki has no web UI of its own, so Grafana Explore is the
+  human surface (the guarded `loki.franksdank.cloud` vhost serves only the API).
+* **Logs: fixed in code, awaiting deploy.** The pipeline shipped nothing because
+  a rootless Alloy cannot read the `0640` journal files, and the container-log
+  globs point at a root-storage layout that does not exist here. The fix is built
+  and tested in-repo (2026-10-20): Alloy is now a **root system service on
+  `Network=host`**, reading `system.journal` + `user-*.journal` and pushing to
+  Loki's `127.0.0.1:3100` loopback publish — a throwaway root probe read 31,825
+  lines and pushed 26,966 with zero drops. The **live** instance is still the old
+  rootless unit until `deploy.sh dankclaw --service alloy` runs. Detail in
+  [[Alloy]] → Why root; Loki's store state in [[Loki]] → Contents.
+* **Not configured:** no blackbox exporter (the structural downs have no real
+  uptime signal) and no Alertmanager vhost. The log pipeline's fix is in code but
+  not yet deployed (see *Logs* above).

 ## Citations

@@ -4,7 +4,7 @@ description: "Pi-hole is the LAN DNS server for the homelab."
 tags:
  - infrastructure
  - pi-hole
-timestamp: 2026-08-28
+timestamp: 2026-10-19
 ---

 # Pi-hole
@@ -46,6 +46,8 @@ The file format is hosts-style (`<IP> <name> [<name> ...]`), one A record per Ca

 **Pitfall (2026-08-24):** the first quadlet mounted the file into `/etc/dnsmasq.d/`. FTL never read it; names only resolved from the query cache. A fresh name returned NXDOMAIN. Mount into `/etc/pihole/hosts/`, not `/etc/dnsmasq.d/`.

+**Pitfall (2026-10-19): FTL can come up with the record file mounted and still serve none of it.** A plain `systemctl --user restart pihole` (16:25, inside the open-notebook deploy window) left every `franksdank.cloud` name NXDOMAIN for LAN clients while **external names kept answering** — from FTL's cache, which is exactly what makes the failure read like a client or an upstream problem. **The trigger is a container restart, not a boot**, and that matters for where the check belongs: `uptime -s` reads 2026-10-18 01:24 (two days of uptime) while `journalctl --user -u pihole.service` shows `Stopping pihole.service` at 16:25:23 and the container's `StartedAt` at 16:25:27 — so verify after **any** pihole restart, deploy-triggered or manual, not only after a reboot. The instrument is on the box: `getent hosts grafana.franksdank.cloud` (or `dig +short @192.168.0.143 <name>`) answering empty means FTL is holding no local records at all; here the file was present and readable inside the container (`-rw-rw-r-- … /etc/pihole/hosts/addn-hosts`, 44 records) and `dns.hosts` in `pihole.toml` was intact. `podman exec pihole pihole restartdns reload` had every name answering within seconds. The mechanism is **not established** — the same mounts and the same config read fine before and after the reload — so treat it as a load failure of the record sources at FTL start, not as a lost or unmounted file. What it cost: the whole Playwright gate, 18 specs across every vhost failing `ERR_NAME_NOT_RESOLVED` while the fleet itself was healthy ([[E2E Suite]]).
+
 ## Upstream

 The slirp netns has no useful `resolv.conf`. The env file pins the upstreams: `DNS1=1.1.1.1`, `DNS2=1.0.0.1` (mirrors the retired `dnsmasq.conf` `server=` lines). Every query that is not a local record goes to those resolvers. The zone apex and unknown subdomains fall through to Porkbun.
@@ -74,6 +76,8 @@ Both prerequisites survive the dnsmasq → Pi-hole swap. Do not remove them.
 | Upstream queries fail (local names still answer) | `DNS1`/`DNS2` in the env file are missing or wrong |
 | Port 53 `bind: permission denied` crash loop after a reboot | sysctl conflict: the value must be 53 at boot, and remember `/etc/sysctl.conf` is applied **after** all of `sysctl.d/`. Redeploy pihole — `deploy_pihole` reinstalls the drop-in, neutralizes conflicting drop-ins, and comments out a stale `/etc/sysctl.conf` line |
 | Single-label name (for example `dankclaw`) answers `127.0.1.1` | The rootless container `/etc/hosts` carries the host entry (`127.0.1.1 <host>`), and FTL serves it. Use the FQDN, or add an `/etc/hosts` entry on the client |
+| Every `franksdank.cloud` name is NXDOMAIN for LAN clients while external names still answer | FTL holds no local records: `getent hosts <name>` on Dankclaw comes back empty → `podman exec pihole pihole restartdns reload` (the 2026-10-19 pitfall under **Records**) |
+| One client gets nothing while every other client is fine | FTL rate-limiting that client: `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`. A `10.0.2.x` client address is a pasta container's DNS forwarder, not a machine |

 ## See Also

@@ -215,9 +215,10 @@ is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
 `GroupAdd=keep-groups`. **No NPU endpoint is deployed** —  stays the
 history of the retired endpoint; what changed is only its precondition.

-**Deliberately deferred:** the `pp_dpm_fclk` pin (level 7, 2000 MHz) from the issue's
-co-run step. Pinning the fabric clock changes iGPU timing for every workload, and
-nothing co-runs on the NPU yet; it goes in when an NPU endpoint actually exists.
+**The `pp_dpm_fclk` pin (level 7, 2000 MHz) went in on 2026-10-19**, with the first thing that needed it: the
+vendor's root host unit, installed and enabled by `frank-dragoon`'s deployer while its NPU lane is named. Pinning
+the fabric clock changes iGPU timing for every workload, which is exactly why it waited for a co-run to exist; the
+law, the hold and the serve-side re-check live in [[frank-dragoon]].
 PerfOpt still needs Linux 7.4 — `7.0.0-38-generic` has zero `perfopt` symbols and the
 26.04 archive caps at `7.0.0-38.38`.

@@ -10,7 +10,7 @@ tags:
  - gfx1151
  - halogen
  - endpoint
-timestamp: 2026-10-18
+timestamp: 2026-10-19
 ---

 # frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
@@ -219,7 +219,7 @@ created it — earlier topologies in .
 - Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a
  leftover Quadlet beside the root unit guarantees the race.

-## The NPU lane (present in the image, not enabled)
+## The NPU lane (XDNA2, armed by one env var)

 XDNA2 executes only vendor-precompiled AIE partition programs — that is the wall the TTS attempt died at
 ([[Frank TTS]]) — and flash-server ships those programs. `HALOGEN_NPU_MODELS=<names>` starts a second engine **in the
@@ -235,11 +235,19 @@ Embedding contract on that lane: `POST /v1/embeddings`, `model: "qwen3-embedding
 (`Instruct: …\nQuery:…`), `base64` accepted, and LangChain's `OpenAIEmbeddings` needs
 `check_embedding_ctx_length=False`.

-Four prerequisites the trio does not satisfy today, from the vendor's own host list — and this box is one of the two
-configurations they test (Ubuntu 26.04's XRT 2.21 `libxrt2`+`libxrt-npu2`, the kernel's `amdxdna`, `iommu=pt`):
+Four host prerequisites, each refused **by name** in the serve script and mirrored in `deploy_llm_dragoon` — a
+precondition that surfaces as an engine crash-loop is a gate that was never written. All four are satisfied and the
+lane is **armed and serving** since 2026-10-19 (`HALOGEN_NPU_MODELS=qwen3-embedding-0.6b`):

-1. `--device /dev/accel/accel0` plus `--group-add keep-groups` (the pattern ComfyUI uses for `/dev/kfd`; `dank` is
-   already in `render`).
+1. `--device /dev/accel/accel0` **and** a node the container's namespace can open. `/dev/accel/accel0` is
+   `root:render 0660` and a rootless namespace maps exactly one uid/gid, so the engine's own first words are
+   *"this process cannot open it (crw-rw---- nobody:nogroup)"*. `--group-add keep-groups` — the vendor's advice and
+   the ComfyUI pattern for `/dev/kfd` — is **dead in this shape**: measured 2026-10-19 from a `dank` shell already
+   in `render`, with and without the flag the node stayed `nobody:nogroup` and the open was refused; an unmapped
+   gid cannot be granted across a namespace that does not map it. The door is the udev mode
+   (`dankclaw/udev/98-halogen-npu-access.rules`, installed and re-triggered by the deployer, `0666` like the GPU
+   compute nodes beside it); the serve script refuses any node that is not other-accessible, so the fault cannot
+   return as a crash-loop.
 2. The host's XRT reachable inside: there is **no `/opt/xilinx/xrt`** here — distro packages put the three files the
   container needs (`libxrt_coreutil.so.2`, `libxrt_core.so.2`, `libxrt_driver_xdna.so.2`) in
   `/usr/lib/x86_64-linux-gnu`, where all three exist.
@@ -248,17 +256,27 @@ configurations they test (Ubuntu 26.04's XRT 2.21 `libxrt2`+`libxrt-npu2`, the k
   engine holds the clock itself only when it runs as root with `/sys` mounted; `frank-dragoon` is a **rootless**
   `podman run` under `User=dank`, so the path is the vendor's root host unit (`deploy/host/halogen-fabric-clock`) —
   which is exactly the `pp_dpm_fclk` level-7 pin deferred in [[dankclaw-iommu-translation-off]] with the words "it
-   goes in when an NPU endpoint actually exists". Held: about a watt. Unheld: the hang. Measured 2026-10-19 nothing
-   is held (fabric DPM lists 400–2000 MHz with no forced level, graphics clock sitting at 857 of 2900 MHz).
-4. `HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next` on a boot with a **writable** `/models/npu` — the payload
-   mount is `:ro` by law, so the programs land via a throwaway writable run or a dedicated read-write bind for that
-   one subdirectory; the vendor notes a read-only volume works but re-runs the conversion at every start.
+   goes in when an NPU endpoint actually exists". Held: about a watt. Unheld: the hang. The deployer installs and
+   enables that unit whenever the lane is named, and `halogen-fabric-clock status` reports `held (manual)` here as of
+   2026-10-19; the serve script re-reads the same two sysfs files at every start and refuses by name if the top level
+   is not the starred one, because a hold done by hand is a countdown to the next reboot.
+4. The programs are staged under `$FRANK_DRAGOON_NPU_DIR`, which **must sit inside the payload dir**
+   (`/home/dank/models/halogen-v2-abliterated/npu`): the payload's read-only mount is the container's only handle on
+   `/models`. A nested bind at `/models/npu` cannot work — the image ships no `/models`, so runc has to `mkdir` the
+   mountpoint through a read-only parent and dies at container init (measured 2026-10-19: 52 crash-loop restarts with
+   chat down before the mount shape was fixed). `HALOGEN_NPU_DIR` is no escape; the shipped entrypoint rejects it by
+   name ("our NPU models live under `/models/npu/<id>/`"). Nothing downloads at boot — `HALOGEN_DOWNLOAD` stays
+   unset, so the container opens no outbound connection; `scripts/download-frank-dragoon-npu.sh` pins every file by
+   size and sha256 against the catalog and the payload mount carries the tree in.

-Price: the programs and their weights sit in host RAM beside the 62.9 GiB of locked weights ("a few GB in all" for
-the default set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same
-memory bus, same power budget. Under this page's RAM law the question is pool versus n-gram table, so read the boot
-ledger after the first NPU boot instead of modelling it. If the lane is opened, prefer **0.18.8 or newer**: that
-release budgets NPU memory before asking for it, fixing kernel errors from several NPU models under a concurrent mix
+Armed, the boot ledger reads (`/home/dank/logs/frank_dragoon.log`, 2026-10-19): the GPU fabric clock is held at its top level (2000 MHz) **before** the NPU starts; `/dev/accel/accel0` is `crw-rw-rw- nobody:nogroup` under driver `amdxdna` with the distro XRT and its NPU plugin; the program arrives as `15 files checked against this image's record`; then `npu_api: qwen3-embedding-0.6b (embeddings) on the NPU engine at 127.0.0.1:8740 (buckets u0…u9; inputs up to 4096 tokens)` and `up to 64 passes queued … up to 7200 s`. Callers never see `:8740` — the lane answers on the same `:8731` as the Flash model and a request picks the engine by its `model` field, so `/v1/models` lists `frank-dragoon` **and** `qwen3-embedding-0.6b`. The fleet's consumer is open-notebook's embedding default ([[Open Notebook]]); that page owns the app-side shapes (the URL guard, defaults-by-id, the `/api/embed` 500 that still writes the vector).
+
+Price: the programs and their weights sit in host RAM beside the locked weights ("a few GB in all" for the default
+set; ~1.1 GiB for `embeddinggemma-2`), and **the Flash model decodes slower while the NPU works** — same memory bus,
+same power budget. Measured 2026-10-19 with one program armed and the endpoint idle, the whole container holds
+**1.55 GiB** of host RSS (`podman stats`), so this lane is not what the RAM law above is negotiating against; a
+multi-model set is where that question returns. If the lane is widened, prefer **0.18.8 or newer**: that release
+budgets NPU memory before asking for it, fixing kernel errors from several NPU models under a concurrent mix
 **on the stock kernel driver** — the driver this box runs.

 ## Files
@@ -130,7 +130,10 @@ is a locator: one row, one clause, no numbers.
 |---|---|
 | [[Agent Docs and Edits]] | Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each. |
 | [[Command Center Dotfiles]] | How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas. |
+| [[Domain Docs]] | Consumer rules for GLOSSARY.md and docs/adr/, and where this vault out-ranks them on operational facts. |
+| [[Issue Tracker]] | How agent skills read and write Gitea issues here: credential-safe curl, label IDs, and wayfinding without sub-issues. |
 | [[OMP Harness]] | The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits. |
+| [[Triage Labels]] | The five canonical triage roles mapped to this repo's Gitea label strings, plus the numeric-ID quirk. |

 ## Research and methodology

@@ -1,22 +1,34 @@
 // Alloy configuration — ships host and container logs to Loki.
 // One instance per host. Deployed to /opt/alloy/config/config.alloy.
+//
+// Runs as a ROOT system service on Network=host (see
+// dankclaw/observability/quadlet/alloy.container), so it reaches Loki on the
+// host's loopback and reads every journal — host services and rootless-quadlet
+// containers alike.

 logging {
  level = "info"
 }

-// The alloy UI service listens on 127.0.0.1:12345 by default; the
-// in-container healthcheck probes it there. alloy 1.x refuses to
-// configure the ui service from a config block, so there is no block.
+// The alloy UI listens on 127.0.0.1:12345; the in-container healthcheck probes
+// it there. alloy 1.x refuses to configure the ui service from a config block,
+// so there is no ui block.

-// Ship logs to Loki (container name on the podman network).
+// Loki is published on the host's loopback (127.0.0.1:3100). Alloy shares the
+// host network, so it pushes by loopback, never by container name — a root
+// container is not on the rootless dankclaw_default network.
 loki.write "default" {
  endpoint {
-    url = "http://loki:3100/loki/api/v1/push"
+    url = "http://127.0.0.1:3100/loki/api/v1/push"
  }
 }

-// Systemd journal from the host socket.
+// The systemd journal. As root with /etc/machine-id + the journal dirs mounted,
+// this one source reads system.journal AND user-*.journal, so it carries both
+// host services and the rootless Quadlet containers' logs. There is deliberately
+// no container-file source: podman container stdout lands in the journal, and
+// the old /var/lib/containers/.../ctr.log and /var/log/containers globs pointed
+// at a root-storage layout that does not exist on this host.
 loki.source.journal "journal" {
  format_as_json = true
  labels = {
@@ -24,12 +36,3 @@ loki.source.journal "journal" {
  }
  forward_to = [loki.write.default.receiver]
 }
-
-// Container logs (podman storage path; adjust for docker).
-loki.source.file "containers" {
-  targets = [
-    {__path__ = "/var/lib/containers/storage/overlay-containers/*/userdata/ctr.log", job = "podman"},
-    {__path__ = "/var/log/containers/*.log", job = "containers"},
-  ]
-  forward_to = [loki.write.default.receiver]
-}
@@ -0,0 +1,17 @@
+# Grafana datasource for Loki, declared in git rather than clicked into
+# grafana.db — same reason as the Prometheus datasource: a UI-configured
+# datasource is invisible to review, lost with the data volume, and unknown
+# to the repository. Loki OSS ships no web UI of its own, so Explore through
+# this datasource is the only way a human reads the log store.
+#
+# URL is the container name on dankclaw_default (the same address Caddy
+# proxies to), not the published loopback port.
+apiVersion: 1
+
+datasources:
+  - name: Loki
+    uid: loki
+    type: loki
+    access: proxy
+    url: http://loki:3100
+    editable: false
@@ -30,9 +30,9 @@
  },
  {
    "name": "open-notebook",
-    "category": "user",
+    "category": "admin",
    "section": "AI",
-    "description": "Podcast generator — notes in, spoken episode out: script written by frank-zealot, voices rendered by frank-tts. Guarded twice: Authentik at the edge, then its own login.",
+    "description": "Podcast generator — notes in, spoken episode out: script written by frank-archon, voices rendered by frank-tts. Guarded twice: Authentik at the edge, then its own login. Single shared password, no per-user accounts — admin-only.",
    "url": "http://notebook.franksdank.cloud",
    "docs": "",
    "probe": "127.0.0.1:8502",
@@ -1,10 +1,15 @@
 # Alerting and recording rules for the LLM fleet.
 #
-# One metric namespace: llamacpp:* — frank-zealot and frank-dragoon, both
-# llama.cpp with /metrics (the family verified from /metrics on 2026-09-16
-# and 2026-09-20). frank-archon LEFT the family 2026-10-05: FreeToken (the
-# PR587 ROCm engine) has no /metrics endpoint, so its scrape job was
-# deleted rather than left never-green.
+# Two metric namespaces, because two runtime families are scraped:
+#
+#   * llamacpp:* — frank-zealot (llama.cpp) and frank-dragoon (halogen, which
+#     re-exports the llama.cpp metric names). Verified from /metrics on
+#     2026-09-16 / 2026-09-20.
+#   * vllm:* — frank-archon, the radiance engine on Dankserver since
+#     2026-10-05. It exports vLLM-compatible names, a DIFFERENT namespace, so
+#     it is a separate scrape job in prometheus.yml and the rules below are a
+#     separate group. (The old claim that its scrape job was deleted is false:
+#     it re-joined the day it left the FreeToken engine and is `up` today.)
 # (The netavark published-port quirk that forces the Dankclaw endpoints
 # onto the gateway is same-host only.)
 #
@@ -22,7 +27,8 @@
 #     cached by cached + prompt.
 #   * Speculative decoding: drafts and accepted tokens are both counters,
 #     so LlmSpecDecodeOff watches the pair.
-#   * No latency histograms exist, so there is no p95 alert.
+#   * No latency histograms in llama.cpp, so no p95 alert there; radiance's
+#     vllm:* namespace DOES export one, so the vllm-fleet group alerts on it.
 #
 # IMPORTANT: delivery. Since 2026-09-17 a firing alert reaches Frank on
 # Signal: Prometheus pushes to Alertmanager (dankclaw/observability/quadlet/
@@ -112,6 +118,84 @@ groups:
          description: >-
            Prometheus cannot reach the LLM endpoint. frank-zealot and
            frank-dragoon are root units on Dankclaw: `systemctl is-active
-            frank-zealot` / `frank-dragoon`. Both scraped endpoints export
-            the llamacpp:* family. (frank-archon left the scrape set
-            2026-10-05 — FreeToken exports no /metrics.)
+            frank-zealot` / `frank-dragoon` (both export llamacpp:* —
+            dragoon via halogen's llama.cpp-compatible names). frank-archon
+            is a radiance endpoint on Dankserver (`systemctl show
+            frank-archon` there) exporting the vllm:* family; the
+            `frank-.*` selector covers all three regardless of namespace.
+
+  # frank-archon (radiance) exports the vLLM-compatible namespace, NOT
+  # llamacpp:*. Every rule here is keyed to a series the exporter actually
+  # emits (verified from /metrics 2026-10-20: num_requests_waiting,
+  # gpu_prefix_cache_hit_rate, num_preemptions_total, prompt_tokens_total,
+  # time_to_first_token_seconds). The llamacpp rules above evaluate nothing for
+  # this job and these evaluate nothing for llama.cpp/halogen — two clean
+  # namespaces, each keyed only to what its runtime exports.
+  - name: vllm-fleet
+    interval: 30s
+    rules:
+      # p95 time-to-first-token, shared with the dashboard. When the endpoint
+      # is idle every bucket rate is zero, histogram_quantile yields NaN, and
+      # neither this nor the alert below can fire on an idle engine.
+      - record: job:vllm_ttft_p95:rate5m
+        expr: |
+          histogram_quantile(0.95, sum by (job, le)
+              (rate(vllm:time_to_first_token_seconds_bucket[5m])))
+
+      # Radiance exports a real queue-depth gauge (llama.cpp only shows the
+      # wall as requests_deferred). --max-num-seqs is small, so a sustained
+      # wait is either real load or this endpoint carrying the fleet alone.
+      - alert: VllmRequestsQueueing
+        expr: vllm:num_requests_waiting > 0
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          summary: "{{ $labels.job }}: requests waiting for 10 minutes"
+          description: >-
+            The radiance scheduler has requests queued. Either the fleet is
+            genuinely busy or this endpoint is carrying it alone.
+
+      # Radiance reports prefix reuse directly as a hit-rate gauge — no
+      # counter division needed. The token-rate floor keeps an idle engine
+      # (hit rate 0 with no traffic) from firing.
+      - alert: VllmPrefixCacheCollapse
+        expr: >-
+          vllm:gpu_prefix_cache_hit_rate < 0.20
+            and rate(vllm:prompt_tokens_total[5m]) > 0.05
+        for: 15m
+        labels:
+          severity: info
+        annotations:
+          summary: "{{ $labels.job }}: prefix hit rate below 20 % for 15 minutes"
+          description: >-
+            Nearly every request prefilling from scratch. With several large
+            contexts in flight this is the capacity wall, not a routing fault.
+
+      # Preemption is the KV-thrash signature radiance reports that llama.cpp
+      # cannot: the engine evicts a running request to admit another. Any
+      # sustained rate means the KV cache cannot hold the concurrency in flight.
+      - alert: VllmPreempting
+        expr: rate(vllm:num_preemptions_total[5m]) > 0
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "{{ $labels.job }}: engine preempting requests"
+          description: >-
+            The engine is evicting running requests to admit new ones — the KV
+            cache is undersized for the concurrency in flight.
+
+      # p95 first-token latency, kept HIGH on purpose: a large-prompt prefill
+      # legitimately spends many seconds before token 1, so this fires only on
+      # a wedged scheduler or a queue backed up behind one request.
+      - alert: VllmTtftSlow
+        expr: job:vllm_ttft_p95:rate5m > 120
+        for: 10m
+        labels:
+          severity: warning
+        annotations:
+          summary: "{{ $labels.job }}: p95 time-to-first-token over 120 s"
+          description: >-
+            First-token latency is pathological, not an honest big prefill.
+            Likely a wedged scheduler or one request stalling the queue.
@@ -240,14 +240,12 @@ CADDYFILE="$REPO_ROOT/dankclaw/Caddyfile"
 }

@test "alloy's UI address comes from the unit, not the config file" {
-    # alloy 1.x refuses to configure the ui service from a config block
-    # ("service ui does not support being configured"), so the unit moves the
-    # UI off the container's loopback with a command-line flag. Without that
-    # flag the port cannot be published and the service has no reachable
-    # health endpoint at all — which is why the homepage status card for
-    # alloy read "down" while it ran under compose.
+    # alloy 1.x refuses to configure the ui service from a config block, so the
+    # unit sets the UI address on the command line. As a Network=host root unit
+    # that address is the host loopback directly (no PublishPort) — reachable
+    # from the host, from nothing off-host.
    ! grep -qE '^(ui|server) \{' "$REPO_ROOT/shared/alloy/config.alloy"
    local unit="$REPO_ROOT/dankclaw/observability/quadlet/alloy.container"
-    grep -q -- '--server.http.listen-addr=0.0.0.0:12345' "$unit"
-    grep -q '^PublishPort=127.0.0.1:12345:12345$' "$unit"
+    grep -q -- '--server.http.listen-addr=127.0.0.1:12345' "$unit"
+    ! grep -q '^PublishPort=' "$unit"
 }
@@ -30,6 +30,7 @@ setup() {
    HOST_UNIT="$ROOT/dankclaw/host/halogen-fabric-clock"
    HOST_SERVICE="$ROOT/dankclaw/host/halogen-fabric-clock.service"
    NPU_FETCH="$ROOT/scripts/download-frank-dragoon-npu.sh"
+    NPU_RULE="$ROOT/dankclaw/udev/98-halogen-npu-access.rules"
    BUILD="$ROOT/scripts/build-rocmfpx.sh"
    TEST_TMP="$(mktemp -d)"
    touch "$TEST_TMP/empty.env"
@@ -357,6 +358,9 @@ $(cat "$TEST_TMP/serve.log")"
    npu_models="$(sed -nE 's/^HALOGEN_NPU_MODELS=(.*)$/\1/p' "$ENV_FILE")"
    grep -qF "LLM_DRAGOON_NPU_DIR=\"\${LLM_DRAGOON_NPU_DIR:-$npu_dir}\"" "$DEPLOYER"
    grep -qF "LLM_DRAGOON_NPU_MODELS=\"\${LLM_DRAGOON_NPU_MODELS:-$npu_models}\"" "$DEPLOYER"
+    # The deployer's own copy of the payload dir is what the lane's containment
+    # gate compares against — a stale twin would wave a bad staging path through.
+    grep -qF "LLM_DRAGOON_MODEL_DIR=\"\${LLM_DRAGOON_MODEL_DIR:-$dir}\"" "$DEPLOYER"
    net="$(sed -nE 's/^FRANK_DRAGOON_NETWORK=(.*)$/\1/p' "$ENV_FILE")"
    grep -qF "LLM_DRAGOON_NETWORK=\"\${LLM_DRAGOON_NETWORK:-$net}\"" "$DEPLOYER"
 }
@@ -415,7 +419,10 @@ $(cat "$TEST_TMP/serve.log")"
 # refused BY NAME instead of surfacing as an engine crash-loop.

 npu_fixture() {
+    # 0666 like the installed node: a rootless container's namespace cannot open
+    # the stock root:render 0660 mode, and the serve gate checks exactly that.
    touch "$TEST_TMP/accel0"
+    chmod 666 "$TEST_TMP/accel0"
    mkdir -p "$TEST_TMP/xrt"
    for l in libxrt_coreutil.so.2 libxrt_core.so.2 libxrt_driver_xdna.so.2; do
        touch "$TEST_TMP/xrt/$l"
@@ -523,6 +530,32 @@ $(cat "$TEST_TMP/serve.log")"
    [[ "$output" == *"iommu=pt"* ]]
 }

+@test "the lane refuses an NPU node the container namespace could not open, and names the rule" {
+    # Stock Debian mode is root:render 0660. The engine's own first words on such
+    # a node are "this process cannot open it (crw-rw---- nobody:nogroup)" and no
+    # flag fixes a mode an unmapped gid cannot cross (keep-groups measured dead
+    # 2026-10-19), so the unit refuses instead of crash-looping 189 times.
+    podman_stub
+    good_env
+    npu_fixture
+    chmod 660 "$TEST_TMP/accel0"
+    run_with_log
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"$TEST_TMP/accel0 is 660"* ]]
+    [[ "$output" == *"98-halogen-npu-access.rules"* ]]
+    [[ "$output" != *"--device"* ]]
+}
+
+@test "the NPU access rule is the accel counterpart of the GPU compute rule" {
+    [ -f "$NPU_RULE" ]
+    grep -qE '^KERNEL=="accel\[0-9\]\*", SUBSYSTEM=="accel", MODE="0666"$' "$NPU_RULE"
+    # The deployer installs the rule and re-triggers the node BEFORE it trusts
+    # the mode, so a machine that never had it self-heals instead of refusing.
+    grep -qE '98-halogen-npu-access\.rules' "$DEPLOYER"
+    grep -qE 'udevadm control --reload-rules' "$DEPLOYER"
+    grep -qE 'udevadm trigger --sysname-match=' "$DEPLOYER"
+}
+
@test "the lane refuses when an XRT NPU library is missing" {
    podman_stub
    good_env
@@ -41,13 +41,17 @@ service_section() {
 }

@test "every observability unit is boot-persistent" {
-    # Generated units cannot be enabled on this host's podman 5.x v1 quadlet
-    # generator; [Install] + linger is the entire boot-persistence story.
+    # Generated units cannot be enabled on this host's podman quadlet generator;
+    # [Install] + linger is the boot-persistence story for the rootless five.
+    # Alloy is a ROOT system unit, so its [Install] targets multi-user.target.
    local svc
    for svc in "${UNITS[@]}"; do
+        [[ "$svc" == alloy ]] && continue
        grep -q '^WantedBy=default.target$' "$UNIT_DIR/$svc.container" \
            || { echo "$svc is not boot-persistent"; return 1; }
    done
+    grep -q '^WantedBy=multi-user.target$' "$UNIT_DIR/alloy.container" \
+        || { echo "alloy (root unit) is not boot-persistent"; return 1; }
 }

@test "every observability unit is self-healing" {
@@ -140,15 +144,20 @@ service_section() {
    grep -q 'reverse_proxy prometheus:9090' "$REPO_ROOT/dankclaw/Caddyfile"
    grep -q 'reverse_proxy loki:3100' "$REPO_ROOT/dankclaw/Caddyfile"
    local svc
-    for svc in prometheus grafana loki alloy; do
+    for svc in prometheus grafana loki; do
        grep -q "^ContainerName=$svc$" "$UNIT_DIR/$svc.container" \
            || { echo "$svc unit renames its container"; return 1; }
        grep -q '^Network=dankclaw_default$' "$UNIT_DIR/$svc.container" \
            || { echo "$svc is not on the network Caddy and its peers use"; return 1; }
    done
+    # Alloy is not on that network: it is a root container on Network=host, the
+    # only way to reach Loki's 127.0.0.1:3100 publish from outside a bridge.
+    grep -q '^ContainerName=alloy$' "$UNIT_DIR/alloy.container"
+    grep -q '^Network=host$' "$UNIT_DIR/alloy.container" \
+        || { echo "alloy must be Network=host to reach Loki on the host loopback"; return 1; }
    grep -q '"grafana:3000"' "$REPO_ROOT/shared/prometheus/prometheus.yml"
    grep -q '"loki:3100"' "$REPO_ROOT/shared/prometheus/prometheus.yml"
-    grep -q 'url = "http://loki:3100/loki/api/v1/push"' "$REPO_ROOT/shared/alloy/config.alloy"
+    grep -q 'url = "http://127.0.0.1:3100/loki/api/v1/push"' "$REPO_ROOT/shared/alloy/config.alloy"
 }

@test "every observability unit has a health check" {
@@ -239,12 +248,14 @@ service_section() {
 }

@test "startup order follows the data dependencies" {
-    # Grafana is useless without Prometheus and Alloy drops logs without Loki;
-    # both pairs are expressed as systemd ordering, not launch luck.
+    # Grafana is useless without Prometheus (both rootless, same manager). Alloy
+    # used to order on loki.service; as a ROOT unit it lives on the SYSTEM
+    # manager and cannot order against a rootless user unit, so it depends only
+    # on the network and retries its push until Loki answers.
    grep -q 'After=network-online.target prometheus.service' "$UNIT_DIR/grafana.container"
    grep -q 'Wants=network-online.target prometheus.service' "$UNIT_DIR/grafana.container"
-    grep -q 'After=network-online.target loki.service' "$UNIT_DIR/alloy.container"
-    grep -q 'Wants=network-online.target loki.service' "$UNIT_DIR/alloy.container"
+    grep -q '^After=network-online.target$' "$UNIT_DIR/alloy.container"
+    ! grep -q 'loki.service' "$UNIT_DIR/alloy.container"
 }

@test "the deployer installs, restarts and health-checks each unit" {
@@ -281,6 +292,24 @@ PAIRS
    grep -q 'build_compose_image alloy' "$REMOTE_DEPLOY"
 }

+@test "alloy is a root host-network unit that reads every journal" {
+    # The whole point of the root cutover: a rootless container cannot open the
+    # 0640 journal files. Pin the properties that make the pipeline carry data.
+    local u="$UNIT_DIR/alloy.container"
+    grep -q '^Network=host$' "$u"
+    grep -q '^Volume=/etc/machine-id:/etc/machine-id:ro$' "$u" \
+        || { echo "alloy missing the load-bearing machine-id mount"; return 1; }
+    grep -q '^Volume=/var/log/journal:/var/log/journal:ro$' "$u"
+    grep -q '^Volume=/run/log/journal:/run/log/journal:ro$' "$u"
+    grep -q -- '--server.http.listen-addr=127.0.0.1:12345' "$u"
+    # Deployer: system install + root image mirror, not the user quadlet loop.
+    grep -q '^deploy_alloy() {' "$REMOTE_DEPLOY"
+    grep -q 'install_system_quadlet dankclaw/observability/quadlet alloy' "$REMOTE_DEPLOY"
+    grep -q '/etc/containers/systemd/' "$REMOTE_DEPLOY"
+    grep -q 'mirror_image_to_root localhost/franksdank/alloy:latest' "$REMOTE_DEPLOY"
+    grep -q 'alloy) deploy_alloy ;;' "$REMOTE_DEPLOY"
+}
+
@test "no dankclaw compose file competes with the quadlet units" {
    # Two owners for one container name is how a compose run resurrects a
    # stale duplicate next to the quadlet-managed one. This file was retired on
@@ -313,33 +342,33 @@ PAIRS
    done
 }

-@test "prometheus scrapes every LLM endpoint that exports metrics" {
-    # The LLM endpoints are the services whose behaviour this fleet argues about
-    # most (KV pressure, prefix-cache reuse, queue depth). Both Dankclaw
-    # endpoints export the llamacpp:* family. frank-archon LEFT the set
-    # 2026-10-05: the FreeToken PR587 ROCm engine has NO /metrics endpoint —
-    # scraping it was a structural never-green (404), so the job was deleted
-    # rather than left vacuous (the Lucebox DS4.1 precedent, 2026-09-26).
-    # The two Dankclaw endpoints are HOST processes reached through
-    # host.containers.internal: this Prometheus is a bridge-networked
-    # container, and on this netavark host a container can only reach the
-    # host's LAN IP at a PUBLISHED port — a host-process bind like
-    # llama-server's 0.0.0.0:7770 is refused at 192.168.0.143:7770 while it
-    # answers at 169.254.1.2:7770 (why frank-zealot stayed red by construction
-    # for days until the 2026-09-20 /targets audit).
+@test "prometheus scrapes every LLM endpoint, in its real metric namespace" {
+    # Every LLM endpoint that exports /metrics must have a scrape job. As of
+    # 2026-10-05 there are TWO runtime namespaces scraped:
+    #   * llamacpp:* — frank-zealot (llama.cpp) and frank-dragoon (halogen,
+    #     re-exporting the llama.cpp names), both root host processes on
+    #     Dankclaw reached via host.containers.internal — this Prometheus is a
+    #     bridge container and a host-process bind is refused at the LAN IP,
+    #     which is why frank-zealot read red by construction for days.
+    #   * vllm:* — frank-archon (radiance) on Dankserver, a different machine,
+    #     so plain LAN routing reaches 192.168.0.235:7770. It RE-JOINED the
+    #     scrape set with radiance; the old "job deleted, exports no /metrics"
+    #     note belonged to the retired FreeToken engine and is false today, so
+    #     this seam now asserts archon IS scraped (a bare `! grep` here was
+    #     set -e-exempt and never enforced the opposite).
    local cfg="$REPO_ROOT/shared/prometheus/prometheus.yml"
    grep -q '"host.containers.internal:7771"' "$cfg" || { echo "frank-zealot not scraped"; return 1; }
    grep -q '"host.containers.internal:7770"' "$cfg" || { echo "frank-dragoon not scraped"; return 1; }
-    ! grep -q '"192.168.0.235:7770"' "$cfg"
-    grep -q '"host.containers.internal:7770,192.168.0.235:7770"' "$cfg" \
-        && { echo "frank-archon folded into a Dankclaw gateway target"; return 1; }
-    for job in frank-zealot frank-dragoon; do
+    grep -q '"192.168.0.235:7770"' "$cfg" || { echo "frank-archon not scraped"; return 1; }
+    for job in frank-zealot frank-dragoon frank-archon; do
        grep -q "job_name: \"$job\"" "$cfg" || { echo "no scrape job for $job"; return 1; }
    done
-    ! grep -q 'job_name: "frank-archon"' "$cfg"
-    # None of them is reached through a guarded vhost: Prometheus cannot solve
-    # an Anubis challenge, so a franksdank.cloud target is red by construction.
-    ! grep -qE '^\s*-\s*"?https?://[^"]*franksdank' "$cfg"
+    # No LLM endpoint is reached through a guarded vhost: Prometheus cannot
+    # solve an Anubis challenge, so a franksdank.cloud target is red by
+    # construction.
+    grep -qE '^[[:space:]]*-[[:space:]]*"?https?://[^"]*franksdank' "$cfg" \
+        && { echo "a scrape target points at a guarded vhost"; return 1; }
+    true
 }

@test "grafana provisioning is declared in git and installed by the deployer" {
@@ -378,28 +407,44 @@ names = [r.get('alert') or r.get('record') for g in d['groups'] for r in g['rule
 for want in ('job:llm_prefix_cache_reuse:rate5m',
             'job:llm_spec_accept_length:rate5m',
             'LlmRequestsQueueing', 'LlmPrefixCacheCollapse',
-             'LlmSpecDecodeOff', 'LlmEndpointDown'):
+             'LlmSpecDecodeOff', 'LlmEndpointDown',
+             'job:vllm_ttft_p95:rate5m', 'VllmRequestsQueueing',
+             'VllmPrefixCacheCollapse', 'VllmPreempting', 'VllmTtftSlow'):
    assert want in names, want
 # Every rule must be keyed to a series that actually exists. A rule naming a
 # series nobody exports evaluates nothing forever and errors never — the
-# failure this seam exists for. Since the 2026-09-26 cutover the only scraped
-# LLM runtime is llama.cpp (frank-zealot, frank-dragoon); the vllm: family left
-# with frank-medium's radiance runtime and must not survive in any expr.
+# failure this seam exists for. Two runtimes are scraped in two namespaces:
+# llama.cpp/halogen (llamacpp:*) for frank-zealot and frank-dragoon, and
+# radiance (vllm:*) for frank-archon. Both families were read straight off the
+# endpoints' /metrics; anything else is a guess.
 exprs = [r.get('expr', '') for g in d['groups'] for r in g['rules']]
-# llama.cpp families measured from /metrics on 2026-09-16 and 2026-09-20
-# (frank-zealot, frank-dragoon). Anything else is a guess.
-live = {'llamacpp:prompt_tokens_total', 'llamacpp:prompt_tokens_cached_total',
-        'llamacpp:requests_deferred', 'llamacpp:requests_processing',
-        'llamacpp:spec_decode_num_drafts_total',
-        'llamacpp:spec_decode_num_accepted_tokens_total',
-        'llamacpp:spec_decode_num_draft_tokens_total',
-        'llamacpp:tokens_predicted_total',
-        'llamacpp:predicted_tokens_seconds', 'llamacpp:prompt_tokens_seconds',
-        'llamacpp:n_decode_total', 'llamacpp:n_busy_slots_per_decode'}
+live = {
+    # llama.cpp / halogen, measured from /metrics 2026-09-16 and 2026-09-20.
+    'llamacpp:prompt_tokens_total', 'llamacpp:prompt_tokens_cached_total',
+    'llamacpp:requests_deferred', 'llamacpp:requests_processing',
+    'llamacpp:spec_decode_num_drafts_total',
+    'llamacpp:spec_decode_num_accepted_tokens_total',
+    'llamacpp:spec_decode_num_draft_tokens_total',
+    'llamacpp:tokens_predicted_total',
+    'llamacpp:predicted_tokens_seconds', 'llamacpp:prompt_tokens_seconds',
+    'llamacpp:n_decode_total', 'llamacpp:n_busy_slots_per_decode',
+    # radiance (frank-archon), measured from /metrics 2026-10-20.
+    'vllm:num_requests_waiting', 'vllm:gpu_prefix_cache_hit_rate',
+    'vllm:num_preemptions_total', 'vllm:prompt_tokens_total',
+    'vllm:time_to_first_token_seconds_bucket',
+}
 used = {m for e in exprs
        for m in re.findall(r'(?:llamacpp|vllm):[a-z_0-9]+', e)}
 assert used <= live, used - live
-assert not any(':vllm' in e for e in exprs), 'vllm: series in rules but no vllm endpoint is scraped'
+# The two namespaces must not bleed across groups: a llamacpp: selector
+# evaluates nothing for the radiance job and vice versa, so each group may
+# reference only its own family.
+per = {g['name']: {m.split(':')[0] for r in g['rules']
+                   for m in re.findall(r'(?:llamacpp|vllm):[a-z_0-9]+', r.get('expr', ''))}
+       for g in d['groups']}
+for name, fams in per.items():
+    allowed = {'llamacpp'} if name == 'llm-fleet' else {'vllm'}
+    assert fams <= allowed, (name, fams)
 print('rules ok:', len(names))
 "
    # A rule file that nothing loads evaluates nothing, and Prometheus reports
@@ -290,7 +290,9 @@ vhost = sys.argv[2]
 cards = [s for s in data if s["name"] == "open-notebook"]
 assert len(cards) == 1, f"expected exactly one open-notebook card, got {len(cards)}"
 card = cards[0]
-assert card["category"] == "user" and card["section"] == "AI", card
+# category moved to "admin": single shared password, no per-user accounts, so it
+# is positioned as an admin-side tool, not a user-facing app (see the vault page).
+assert card["category"] == "admin" and card["section"] == "AI", card
 assert card["probe"] == "127.0.0.1:8502", card["probe"]
 assert card["method"] == "http"
 # Deployed and guarded: the card is live and its link is the vhost, so the
Author	SHA1	Message	Date
dank	980491668c	Merge pull request 'Alloy ships logs: root host-network service so it can read the journal' (#53 ) from feat/alloy-root-journal into main	2026-10-20 20:05:00 -07:00
Frank	e494617599	docs(alloy): document the root journal-read cutover Alloy page rewritten to the shipped root design (why root, the two mounts that make it work, deploy-pending status); Observability current state marks the log pipeline fixed-in-code, awaiting deploy.	2026-10-20 19:50:15 -07:00
Frank	9809ed9c74	test(observability): pin the root alloy shape The seams encoded alloy-as-rootless (default.target, dankclaw_default, loki:3100 by name, 0.0.0.0+PublishPort UI, After=loki.service). Update them to the root/system/host shape and add a test pinning the load-bearing properties: Network=host, the /etc/machine-id mount, the journal mounts, loopback UI, and the system-install + root-mirror deploy path.	2026-10-20 19:50:15 -07:00
Frank	634fef242b	feat(alloy): run it as a root host-network service so it reads the journal A rootless Alloy shipped zero lines: the journal files are root:systemd-journal 0640 (other::---) and a rootless container carries no supplementary groups, so sd_journal_open sees an empty view silently; the rootless quadlet container logs also live in user-1000.journal, not the ctr.log paths the old config tailed. As root, one loki.source.journal reads system.journal + user-*.journal and carries both. Network=host because Loki publishes 127.0.0.1:3100 and no bridge container can reach the host loopback; the UI binds loopback directly (no PublishPort). The /etc/machine-id mount is load-bearing — sd_journal finds the journal dir by the local machine id; without it the source reads 0 lines, health green. deploy_alloy routes around the rootless loop: build rootless, mirror the image into the root store (a root build fails on DNS in the build container), retire the old rootless alloy unit, install to /etc/containers/systemd. Validated with a throwaway root probe: 31,825 lines read, 26,966 pushed to Loki in 20s, zero drops.	2026-10-20 19:50:15 -07:00
dank	2fe68e69af	Merge pull request 'Observability: vllm rule group for frank-archon + Loki datasource; correct stale log-pipeline facts' (#52 ) from fix/observability-vllm-loki into main	2026-10-20 19:39:38 -07:00
Frank	70dd7fbe6d	docs(observability): correct current state and log-pipeline root cause AGENTS.md: the rule set is now two groups (llm-fleet + vllm-fleet) and a Loki datasource is provisioned beside Prometheus. Observability.md: current-state census refreshed (three LLM targets up, two rule groups, Loki datasource); logs still carry no data. Alloy.md: replace the 'wrong glob' theory with the proven cause - a rootless container cannot read the 0640 systemd-journal files (a rootless busybox read returns DENIED), and rootless quadlet logs land in the user journal the system-only source never reads. Closing it is a permission/architecture decision, not a config glob.	2026-10-20 19:28:20 -07:00
Frank	2626332828	feat(grafana): provision the Loki datasource Adds shared/grafana/provisioning/datasources/loki.yml at http://loki:3100 so logs are readable through Grafana Explore. Loki ships no web UI of its own and the guarded vhost serves only the API; this is the human surface.	2026-10-20 19:28:20 -07:00
Frank	eb4d9910cb	feat(observability): add vllm-fleet rule group for frank-archon radiance exports vllm:* metrics (verified on the live endpoint), not llamacpp:: nothing in llm-fleet could ever see frank-archon. Add a vllm-fleet group - 1 recording + 4 alert (queueing, prefix-cache collapse, preemption, p95 time-to-first-token) keyed to vllm:. Split the observability seam so llm-fleet rules bind llamacpp:* and vllm-fleet rules bind vllm:*, and neither may evaluate the other's namespace. Drops the retired no-metrics-for-archon doctrine the seam still asserted and the vacuous namespace grep.	2026-10-20 19:28:20 -07:00
dank	5cc2b3c25a	Merge pull request 'docs(openwebui): TTS reflects live frank-tts wiring (wilwheaton, LAN IP)' (#51 ) from docs/openwebui-tts-voice into main	2026-10-20 18:20:04 -07:00
Frank	18a29c40d2	docs(openwebui): TTS config reflects the live frank-tts wiring The Text-to-speech section recorded audio.tts base_url host.containers.internal and voice narrator1. Re-keyed the live config DB 2026-10-20: voice wilwheaton (narrator1 retired), api_base_url the canonical host LAN IP 192.168.0.143:7772. 7772 is published so host.containers.internal also reaches it, but the LAN IP matches this page's own on-host law. Browser-kokoro example voice updated to the current pack; the dated 2026-10-18 verification is left as history.	2026-10-20 18:18:01 -07:00
dank	73a7ec3fc1	Merge pull request 'chore(frank-tts): repoint narrator slots to live voice packs' (#50 ) from chore/narrator-voice-repoint into main	2026-10-20 18:12:52 -07:00
Frank	130e9d739f	chore(frank-tts): repoint narrator slots to the live voice packs Frank replaced the narrator1/2/3 packs with cloned character voices (wilwheaton, vaati, dr.kern, garrickhagon, jeffhays). The three narrator slots are now carried by real packs: narrator1=wilwheaton, narrator2=vaati, narrator3=dr.kern. - frank-tts.env DEFAULT_VOICE narrator1 -> wilwheaton (a voice-less request 400'd on the dead name; wilwheaton is narrator1's successor). - Docs: correct the /health census and the podcast-speaker references in Frank TTS.md and Open Notebook.md to the live packs. Dated 2026-10-18/19 measurement logs are left as history. Open Notebook's live dank-duo speaker profile was repointed over REST (Frank->wilwheaton, Comrade->vaati) - app-side, not in this commit. Bats fixtures keep narrator1 as a self-contained sample pack; untouched.	2026-10-20 18:11:40 -07:00
dank	854890a155	Merge pull request 'chore(open-notebook): admin-side positioning + single-tenant note' (#49 ) from chore/open-notebook-admin-tenant into main	2026-10-20 17:44:45 -07:00
Frank	d9f3d1be1e	chore(open-notebook): position as admin-side; document single-tenant open-notebook has no user accounts upstream (PasswordAuthMiddleware = one shared password; SurrealDB has no user/account/owner table — verified in the running image). One shared password = one shared workspace; the guard chain gates access but cannot partition data, and Authentik identity is never consumed. Multi-user is approved-but-unbuilt upstream (#712, post-v2); OIDC declined (#607). So move the homepage card from the user grid to the admin tier and add the single-tenant boundary to the vault page. Fix the card's stale 'frank-zealot' -> frank-archon. Update the seam's incidental category pin to the new intentional value. Password rotated on the host (not repo).	2026-10-20 17:38:22 -07:00
dank	afa3a4521c	Merge pull request 'docs(authentik): record 7-day login-session TTL + setter script' (#48 ) from chore/authentik-session-ttl into main	2026-10-20 16:57:29 -07:00
Frank	9cc8ca1c88	docs(authentik): record the 7-day login-session TTL and add a setter script The login-session lifetime (raised to days=7 on 2026-10-20) is owned by the default-authentication-login stage's session_duration, not a global setting and not SESSION_COOKIE_AGE. Authentik exposes no SESSION_SAVE_EVERY_REQUEST, so the window is absolute. session_duration is a STRING idiom; a timedelta stores a form timedelta_from_string rejects, crashing login. Add scripts/authentik-set-login-session-duration.sh that validates the form before writing (DB state, not repo-managed, re-run after a blueprint re-import). Trim the page back under the OKF 500-line cap. tests/run.sh green.	2026-10-20 16:55:44 -07:00
dank	9c05c40aa0	Merge pull request 'docs(agent): scaffold agent-skills config in the vault' (#47 ) from chore/agent-skills-vault-setup into main	2026-10-20 16:11:47 -07:00
Frank	63219787af	docs(agent): scaffold the agent-skills config as three vault pages Point the mattpocock-lineage skills (triage, to-tickets, to-spec, wayfinder) at repo facts instead of a parallel docs/ tree: Gitea is the issue tracker (credential-safe curl shape, token-only writes), the five canonical triage roles keep their default label strings (created on the tracker today), and GLOSSARY.md/docs/adr/ stay lazy with this vault owning operational facts. AGENTS.md gains the ## Agent skills pointer block; indices regenerated.	2026-10-20 16:09:30 -07:00
Frank	e914ecbca6	docs(pi-hole): the record-load trap rides a container restart, not a boot The 2026-10-19 write-up inferred a host restart from FTL's 16:25 start line. It was wrong: uptime -s reads 2026-10-18 01:24 (two days up) and the user journal shows 'Stopping pihole.service' at 16:25:23 with StartedAt 16:25:27 — a plain container restart inside the open-notebook deploy window. That moves where the verification belongs: after every pihole restart, not just after a reboot.	2026-10-20 10:56:39 -07:00
Frank	1eebf3b129	docs(netns): the by-name path to frank-dragoon is verified from the consumer open-notebook resolves frank-dragoon-halogen to 10.89.1.39, GET :8731/v1/models answers, and the engine logs the app's POST /v1/embeddings 200. That makes the 2026-10-19 dial census a pre-move record: pasta's forwarding rule is unchanged, the endpoint was moved out of pasta, so the 'menu is frank-tts + frank-archon' line is now past tense. Frontmatter date was still 2026-09-02 against October content; bumped.	2026-10-19 22:48:44 -07:00
Frank	af5a1f69a2	docs(pi-hole): FTL can boot with the record file mounted and serve none of it The 2026-10-19 reboot left every franksdank.cloud name NXDOMAIN for LAN clients while external names kept answering from cache; the file was mounted and readable and dns.hosts was intact, and 'pihole restartdns reload' restored every name in seconds. Mechanism not established. Instrument and remedy now on the owner page, plus a failure-mode row for the FTL rate-limit case seen beside it (10.0.2.x = a pasta container's forwarder).	2026-10-19 22:46:57 -07:00
Frank	0930510f9b	docs(vault): the embedding lane has a consumer, and two shapes that bite open-notebook's embedding default now points at qwen3-embedding-0.6b on frank-dragoon's XDNA2 lane, proven through the app's own paths (model test 1024 dims, worker 'Successfully embedded', vector search ranking two unrelated notes 0.659/0.292 and 0.906/0.318). Records the two app-side traps: defaults store the model ID and a name is accepted then fails in the worker, and POST /api/embed 500s on an upstream RecordID typing bug after the vector is already written. frank-dragoon's lane section moves from 'off until the mount fix' to armed-and-serving with the boot-ledger contract and the measured idle RSS; the page's stale 'not on dankclaw_default' reachability claims are retired.	2026-10-19 22:46:57 -07:00
dank	af327bcc5e	Merge pull request 'feat(frank-dragoon): open the NPU node the way rootless can actually reach it' (#46 ) from feat/frank-dragoon-npu-lane into main	2026-10-19 22:35:49 -07:00
Frank	fcf94562d1	feat(frank-dragoon): open the NPU node the way rootless can actually reach it The armed boot reached the engine and died in its own preflight: halogen npu: /dev/accel/accel0 is here, but this process cannot open it (crw-rw---- nobody:nogroup). The node is root:render 0660 and a rootless container maps exactly one uid/gid, so `render` is unmapped inside it and `--group-add keep-groups` — the vendor's advice, and the ComfyUI pattern for /dev/kfd — cannot grant a group the namespace does not map. Measured 2026-10-19 from a `dank` shell already in `render`: without the flag denied, with the flag denied, with the udev mode 0666 the open succeeds (OPEN_OK). Same wall 99-comfyui-gpu-access.rules already solved for the GPU compute nodes, so the accel node gets its counterpart: dankclaw/udev/98-halogen-npu-access.rules — KERNEL=="accel[0-9]*", SUBSYSTEM=="accel", MODE="0666" The deployer installs the rule, reloads rules, re-triggers the node and only then checks the mode; the serve script refuses any node that is not other-writable, naming the rule, so this cannot return as a crash loop (189 restarts while chat was down is the failure mode being closed). Both checks read the permission bit directly: `[ -o FILE ]` is not "other-writable" — it means "exists" and `-o` doubles as the OR operator inside test, which is why the first cut of the gate refused everything. Dragoon seam: 33 -> 35 tests. The fixture node is 0666 like the installed one, a 0660 node is refused with the rule named and nothing rendered, and the rule file's shape plus the deployer's install/trigger are asserted.	2026-10-19 22:35:06 -07:00
dank	861920797d	Merge pull request 'fix(deploy): the dragoon gate reads the deployer's own payload-dir twin' (#45 ) from feat/frank-dragoon-npu-lane into main	2026-10-19 21:49:16 -07:00
Frank	3ef711f146	fix(deploy): the dragoon gate reads the deployer's own payload-dir twin The containment gate added in #44 parsed scripts/frank-dragoon.env through a $CHECKOUT variable that does not exist in deploy-remote.sh, so the very next deploy died at `set -u` before it touched the unit: ./scripts/deploy-remote.sh: line 1445: CHECKOUT: unbound variable deploy-remote.sh runs from the checkout root and every other path in that block is a parity-twin default (LLM_DRAGOON_CKPT/_VISION/_TOK_DIR), so the payload dir gets the same treatment: LLM_DRAGOON_MODEL_DIR, defaulted to the deployed path, compared directly. The seam now asserts that twin against the env file next to the NPU twins it already checks, and the containment comparison itself is verified both ways (inside accepted; the old outside path and a shared-prefix sibling both refused).	2026-10-19 21:48:47 -07:00
dank	823fd52d0e	Merge pull request 'fix(frank-dragoon): stage the NPU lane inside the payload mount' (#44 ) from feat/frank-dragoon-npu-lane into main	2026-10-19 21:46:25 -07:00
Frank	bda011d26f	docs(vault): the NPU lane's one door into the container, and the pin that landed frank-dragoon's lane section: the staging dir must live inside the payload dir (the image ships no /models, so a nested :ro bind has no mountpoint to land on), the entrypoint's rejection of HALOGEN_NPU_DIR, and the fabric-clock paragraph corrected — the vendor's root host unit is installed and reports `held (manual)` as of today, so "nothing is held" is retired along with the hand-write it replaced. dankclaw-iommu-translation-off: the deferred `pp_dpm_fclk` pin is no longer deferred; the fact is owned by [[frank-dragoon]] and this page links it.	2026-10-19 21:43:36 -07:00