Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d4ce76b68b | ||
|
|
65bd5ccea7 | ||
|
|
5976ad0550 | ||
|
|
5ae6b7b928 | ||
|
|
cc0bc06288 | ||
|
|
bdead570e2 | ||
|
|
156ea71879 | ||
|
|
ab80c1366d | ||
|
|
c5a816e6a8 | ||
|
|
cb25796a6b | ||
|
|
f31e7c5ed8 | ||
|
|
028afdeb58 | ||
|
|
acf2a5cee5 |
@@ -35,11 +35,16 @@ the [runbook index](#runbook-index). The full swap history lives in
|
|||||||
|
|
||||||
## Repository Layout
|
## Repository Layout
|
||||||
|
|
||||||
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
|
- `dankserver/` — quadlet units + Caddyfile for the media stack (the
|
||||||
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
|
compose.yaml inventory retired 2026-10-13; its mounts had diverged from
|
||||||
- `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
|
the quadlets)
|
||||||
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
|
- `dankclaw/` — quadlet units + Caddyfile for the AI stack (compose retired 2026-09-16)
|
||||||
scripts, systemd units, env tunables, bench harnesses
|
- `shared/` — the service inventory (`services.yml`), homepage, Prometheus,
|
||||||
|
Loki, Alloy, SearXNG configs
|
||||||
|
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh` and
|
||||||
|
`service-map.py` (the inventory's only reader), LLM serve scripts + the
|
||||||
|
shared artifact gates (`scripts/lib/llm-gates.sh`), systemd units, env
|
||||||
|
tunables, bench harnesses
|
||||||
- `old/` — retired LLM scripts (reference only)
|
- `old/` — retired LLM scripts (reference only)
|
||||||
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
||||||
The managed skill library is distilled there one page per skill-cluster:
|
The managed skill library is distilled there one page per skill-cluster:
|
||||||
@@ -70,8 +75,13 @@ All edits flow through git. Never edit files on a remote machine directly.
|
|||||||
./scripts/deploy.sh dankclaw --dry-run
|
./scripts/deploy.sh dankclaw --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
The planner maps changed paths to services; the remote runner restarts only
|
The inventory (`shared/services.yml`, read only through
|
||||||
the selected service and records its revision outside the checkout.
|
`scripts/service-map.py`) maps changed paths to services; the remote runner
|
||||||
|
restarts only the selected service and records its revision outside the
|
||||||
|
checkout. Both `--service` allow-lists, the planner, and the changed-test
|
||||||
|
seam mapper ARE the inventory — adding, moving, or retiring a service is a
|
||||||
|
data edit, not a script edit
|
||||||
|
(`serverAppConfigs-docs/Architecture/Service Inventory.md`).
|
||||||
Documentation-only changes are a no-op. LLM services are selected only by
|
Documentation-only changes are a no-op. LLM services are selected only by
|
||||||
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
||||||
by its unit/env/serve trio in `scripts/` — so
|
by its unit/env/serve trio in `scripts/` — so
|
||||||
@@ -164,12 +174,17 @@ Tests are mandatory before commit (`bats` required on the dev machine):
|
|||||||
./tests/run.sh
|
./tests/run.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Seams: compose validity, env consistency (no leaked secrets), Caddyfile
|
Seams: env consistency (no leaked secrets), Caddyfile
|
||||||
validity, quadlet units, docs completeness **including vault link integrity**,
|
validity, quadlet units, docs completeness **including vault link integrity**,
|
||||||
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
||||||
deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
|
LLM artifact gates (`llm_gates`/`llm_deploy_gates` — the shared
|
||||||
and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
|
`scripts/lib/llm-gates.sh` every endpoint's serve and deploy gates run on),
|
||||||
no project, because such a spec silently never runs.
|
deployment flow, service-scoped planning, the **service inventory**
|
||||||
|
(`service_inventory.bats`: inventory soundness, the pinned path->service
|
||||||
|
plan fixture, render drift for the Pi-hole hosts file), homepage
|
||||||
|
config/pages/deploy/checker, and `e2e_scaffold` — which fails if a
|
||||||
|
Playwright spec file's number prefix is in no project, because such a spec
|
||||||
|
silently never runs.
|
||||||
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
||||||
|
|
||||||
## Secret Rules
|
## Secret Rules
|
||||||
@@ -1041,7 +1056,7 @@ Active:
|
|||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, Swift1.5-Qwen3.8-Flash-Next GPTQ w4 + bf16-protected experts, int8 trunk, bf16 KV, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1050,6 +1065,7 @@ Retired (rollback targets and history):
|
|||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
|
| radiance StillDeadcode Qwen3.8-Flash-Next native-FP8, 2×R9700 TP=2 (`frank-archon`) | Dankserver | 2026-10-08 → w4dn8 swap (rollback target — `.rad` payload on disk) | `frank-archon-radiance-qwen38-native-fp8.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||||
|
|||||||
+24
-27
@@ -56,6 +56,21 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════════════════
|
||||||
|
# Named snippet: the pre-auth guard chain. Anubis (:8923) challenges,
|
||||||
|
# the embedded Authentik outpost authenticates, and authenticated
|
||||||
|
# traffic returns to the http://:8017 listener's post-auth handlers.
|
||||||
|
# Every guarded vhost is `import guarded` — the chain is EDITED HERE,
|
||||||
|
# never inlined per site (rank 6: it used to be nine verbatim copies).
|
||||||
|
# Exemptions live in their own vhosts (auth. is the IdP itself; git
|
||||||
|
# smart-HTTP and self-authenticating services document their reason).
|
||||||
|
# ═══════════════════════════════════════════════════════════════
|
||||||
|
(guarded) {
|
||||||
|
reverse_proxy anubis:8923 {
|
||||||
|
header_up X-Real-Ip {remote_host}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
# --- Bare-IP access (health checks, convenience) ---
|
# --- Bare-IP access (health checks, convenience) ---
|
||||||
:80 {
|
:80 {
|
||||||
redir https://www.franksdank.cloud/ 308
|
redir https://www.franksdank.cloud/ 308
|
||||||
@@ -72,9 +87,7 @@ franksdank.cloud {
|
|||||||
}
|
}
|
||||||
|
|
||||||
www.franksdank.cloud {
|
www.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Legacy name: the old homepage name redirects straight onto www ---
|
# --- Legacy name: the old homepage name redirects straight onto www ---
|
||||||
@@ -100,9 +113,7 @@ auth.franksdank.cloud {
|
|||||||
|
|
||||||
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
|
# --- SearXNG (quadlet; behind the Gluetun tunnel, published 8081) ---
|
||||||
search.franksdank.cloud {
|
search.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
|
# --- OpenWebUI (quadlet on slirp4netns; published 3005) ---
|
||||||
@@ -126,30 +137,22 @@ gitea.franksdank.cloud {
|
|||||||
# never reach Grafana — RP mode needs none and works regardless of topology.
|
# never reach Grafana — RP mode needs none and works regardless of topology.
|
||||||
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
|
# The basicauth edge this replaces (CADDY_BASICAUTH_HASH) is retired.
|
||||||
grafana.franksdank.cloud {
|
grafana.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Prometheus ---
|
# --- Prometheus ---
|
||||||
prometheus.franksdank.cloud {
|
prometheus.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
|
# --- Loki (guarded: Anubis → Authentik; loki-proxy provider, internal_host caddy:8017) ---
|
||||||
loki.franksdank.cloud {
|
loki.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
|
# --- Seerr (guarded: Anubis → Authentik; seerr provider, internal_host caddy:8017) ---
|
||||||
seerr.franksdank.cloud {
|
seerr.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
|
# --- Marreta (guarded: Anubis → Authentik; marreta provider, internal_host caddy:8017) ---
|
||||||
@@ -157,9 +160,7 @@ seerr.franksdank.cloud {
|
|||||||
# group gate). Egress rides the searxng gluetun tunnel; reached via the
|
# group gate). Egress rides the searxng gluetun tunnel; reached via the
|
||||||
# host-published port 8082 (gluetun netns), not by container name.
|
# host-published port 8082 (gluetun netns), not by container name.
|
||||||
marreta.franksdank.cloud {
|
marreta.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
# --- Pi-hole (quadlet on slirp4netns; published 8083) ---
|
||||||
@@ -233,16 +234,12 @@ books.franksdank.cloud {
|
|||||||
|
|
||||||
# --- RomM ---
|
# --- RomM ---
|
||||||
romm.franksdank.cloud {
|
romm.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Copyparty ---
|
# --- Copyparty ---
|
||||||
files.franksdank.cloud {
|
files.franksdank.cloud {
|
||||||
reverse_proxy anubis:8923 {
|
import guarded
|
||||||
header_up X-Real-Ip {remote_host}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# --- Jellyfin ---
|
# --- Jellyfin ---
|
||||||
|
|||||||
@@ -1,30 +1,32 @@
|
|||||||
# Quadlet unit: node-exporter (rootless, boot-persistent).
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankclaw
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
#
|
#
|
||||||
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
|
# Install path on dankclaw: ~/.config/containers/systemd/node-exporter.container
|
||||||
# Unit name: node-exporter.service (container name: node-exporter)
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
#
|
#
|
||||||
# Host metrics source for Prometheus. This replaces the compose service of the
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
# same name in dankclaw/compose.yaml ("Observability" block).
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
#
|
#
|
||||||
# HOST NAMESPACE. It must run in the host network namespace to report the
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
# host's own interfaces/connections, and --pid=host for the processes and
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
# cgroups collectors. "localhost" inside the container is therefore the host's
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
# loopback, which is what the homepage status timer probes.
|
# loopback, which is what the homepage status timer probes.
|
||||||
#
|
#
|
||||||
# LISTEN ADDRESSES. Until 2026-09-11 this bound :9100 on every interface. It
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
# now binds exactly two: loopback for the homepage status probe, and the LAN
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
# address because Prometheus — which runs in a container on dankclaw_default —
|
# dankclaw_default — scrapes http://192.168.0.143:9100/metrics
|
||||||
# scrapes http://192.168.0.143:9100 (shared/prometheus/prometheus.yml,
|
# (shared/prometheus/prometheus.yml, job "node-dankclaw"). node_exporter has no
|
||||||
# job "node-dankclaw"). node_exporter itself has no authentication, so keeping
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
# the wildcard bind off the table matters: this is the second half of the
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
# pairing that removes the unauthenticated observability surface, and both
|
# host, and both consumers are accounted for.
|
||||||
# consumers are accounted for. Dankserver runs its own copy of this container
|
|
||||||
# (scraped by the "node-dankserver" job) — since 2026-09-16 as a Quadlet unit in
|
|
||||||
# the same shape (dankserver/observability/quadlet/), because the compose-only
|
|
||||||
# version it replaced had been dead for a day and a half before anyone noticed.
|
|
||||||
#
|
#
|
||||||
# MOUNTS. The container reads the host's /proc, /sys and / as /host/* read-only
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
# and is pointed at them with --path.*; it writes nothing anywhere.
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Prometheus node exporter (Dankclaw host metrics)
|
Description=Prometheus node exporter (Dankclaw host metrics)
|
||||||
|
|||||||
+30
-34
@@ -1,47 +1,43 @@
|
|||||||
# A records for the flat franksdank.name scheme (dnsmasq addn-hosts format:
|
# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts
|
||||||
# <IP> <name> [<name> ...]). Served by the Pi-hole quadlet (FTL is a
|
# format: <IP> <name>). RENDERED — do not edit by hand:
|
||||||
# dnsmasq fork): the quadlet mounts this file into /etc/pihole/hosts/ read-only.
|
|
||||||
#
|
#
|
||||||
# Caddy is consolidated on dankclaw (192.168.0.143): it fronts every service
|
# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts
|
||||||
# on both hosts, so every name resolves to that one IP. Keep in sync with the
|
|
||||||
# Caddyfile (tests/pihole.bats cross-checks this).
|
|
||||||
#
|
#
|
||||||
# 192.168.0.143 = dankclaw (runs the Caddy container + the AI/dev stack)
|
# The source of truth is the `dns:` field of shared/services.yml; the
|
||||||
# 192.168.0.235 = dankserver (media stack; reached by Caddy via published
|
# drift gate in tests/service_inventory.bats fails when this file and
|
||||||
# LAN ports — no DNS records point here)
|
# the inventory disagree. Caddy is consolidated on Dankclaw, so every
|
||||||
|
# name — including the services running on dankserver and
|
||||||
|
# frankbook-pro — resolves at the Dankclaw edge.
|
||||||
|
#
|
||||||
|
# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing
|
||||||
|
# comment on a data line becomes a HOSTNAME token. No inline notes
|
||||||
|
# here; the service attribution lives in the inventory.
|
||||||
|
|
||||||
# --- Homepage (naked domain + www) + legacy names ---
|
192.168.0.143 search.franksdank.cloud
|
||||||
|
192.168.0.143 marreta.franksdank.cloud
|
||||||
|
192.168.0.143 pihole.franksdank.cloud
|
||||||
|
192.168.0.143 ai.franksdank.cloud
|
||||||
|
192.168.0.143 gitea.franksdank.cloud
|
||||||
|
192.168.0.143 comfyui.franksdank.cloud
|
||||||
|
192.168.0.143 seerr.franksdank.cloud
|
||||||
|
192.168.0.143 hermes.franksdank.cloud
|
||||||
|
192.168.0.143 hermesadmin.franksdank.cloud
|
||||||
|
192.168.0.143 prometheus.franksdank.cloud
|
||||||
|
192.168.0.143 grafana.franksdank.cloud
|
||||||
|
192.168.0.143 loki.franksdank.cloud
|
||||||
|
192.168.0.143 auth.franksdank.cloud
|
||||||
192.168.0.143 franksdank.cloud
|
192.168.0.143 franksdank.cloud
|
||||||
192.168.0.143 www.franksdank.cloud
|
192.168.0.143 www.franksdank.cloud
|
||||||
192.168.0.143 dankclaw.franksdank.cloud
|
192.168.0.143 dankclaw.franksdank.cloud
|
||||||
192.168.0.143 dankserver.franksdank.cloud
|
192.168.0.143 dankserver.franksdank.cloud
|
||||||
|
|
||||||
# --- Authentik (SSO / OIDC provider) ---
|
|
||||||
192.168.0.143 auth.franksdank.cloud
|
|
||||||
|
|
||||||
# --- Media services (dankserver, proxied) ---
|
|
||||||
192.168.0.143 plex.franksdank.cloud
|
|
||||||
192.168.0.143 plex.franksdank.casa
|
|
||||||
192.168.0.143 books.franksdank.cloud
|
|
||||||
192.168.0.143 romm.franksdank.cloud
|
192.168.0.143 romm.franksdank.cloud
|
||||||
192.168.0.143 files.franksdank.cloud
|
192.168.0.143 files.franksdank.cloud
|
||||||
192.168.0.143 jelly.franksdank.cloud
|
192.168.0.143 jelly.franksdank.cloud
|
||||||
|
192.168.0.143 books.franksdank.cloud
|
||||||
|
192.168.0.143 plex.franksdank.cloud
|
||||||
|
192.168.0.143 plex.franksdank.casa
|
||||||
|
192.168.0.143 prowlarr.franksdank.cloud
|
||||||
|
192.168.0.143 qbittorrent.franksdank.cloud
|
||||||
192.168.0.143 radarr.franksdank.cloud
|
192.168.0.143 radarr.franksdank.cloud
|
||||||
192.168.0.143 lidarr.franksdank.cloud
|
192.168.0.143 lidarr.franksdank.cloud
|
||||||
192.168.0.143 readarr.franksdank.cloud
|
192.168.0.143 readarr.franksdank.cloud
|
||||||
192.168.0.143 prowlarr.franksdank.cloud
|
|
||||||
192.168.0.143 qbittorrent.franksdank.cloud
|
|
||||||
|
|
||||||
# --- AI / search / dev services (dankclaw, local) ---
|
|
||||||
192.168.0.143 search.franksdank.cloud
|
|
||||||
192.168.0.143 comfyui.franksdank.cloud
|
|
||||||
192.168.0.143 ai.franksdank.cloud
|
|
||||||
192.168.0.143 hermes.franksdank.cloud
|
|
||||||
192.168.0.143 hermesadmin.franksdank.cloud
|
|
||||||
192.168.0.143 gitea.franksdank.cloud
|
|
||||||
192.168.0.143 grafana.franksdank.cloud
|
|
||||||
192.168.0.143 prometheus.franksdank.cloud
|
|
||||||
192.168.0.143 loki.franksdank.cloud
|
|
||||||
192.168.0.143 pihole.franksdank.cloud
|
|
||||||
192.168.0.143 seerr.franksdank.cloud
|
|
||||||
192.168.0.143 marreta.franksdank.cloud
|
|
||||||
|
|||||||
@@ -1,34 +1,32 @@
|
|||||||
# Quadlet unit: node-exporter (rootless, boot-persistent) — DANKSERVER.
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host dankserver
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
#
|
#
|
||||||
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
|
# Install path on dankserver: ~/.config/containers/systemd/node-exporter.container
|
||||||
# Unit name: node-exporter.service (container name: node-exporter)
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
#
|
#
|
||||||
# WHY THIS FILE EXISTS. Until 2026-09-16 the Dankserver node-exporter ran only
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
# from `dankserver/compose.yaml` (the "Observability" block, `restart:
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
# unless-stopped`). Rootless Podman does not restart bare containers after the
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
# explicit stop that shutdown performs, so it died at a reboot and stayed dead:
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
# `podman ps -a` showed it `Exited (143)` and Prometheus' `node-dankserver` job
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
# had been scraping nothing since. The compose entry also bound the default
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
# wildcard address, which is the unauthenticated-metrics-API-on-every-interface
|
|
||||||
# posture that was already removed on Dankclaw. This unit replaces both: one
|
|
||||||
# owner per container name (the compose service is deleted), explicit listen
|
|
||||||
# addresses, and a generator-provided boot path.
|
|
||||||
#
|
#
|
||||||
# HOST NAMESPACE. Networking and PID in the host's namespaces: the collectors
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
# report the host's own interfaces, connections, processes and cgroups. The
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
# host's /proc, /sys and / are mounted read-only under /host and /rootfs, and
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
# node_exporter writes nothing.
|
# loopback, which is what the homepage status timer probes.
|
||||||
#
|
#
|
||||||
# LISTEN ADDRESSES. Exactly two:
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
# - 127.0.0.1:9100 for host-local probes
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
# - 192.168.0.235:9100 for Prometheus, which scrapes this host from Dankclaw
|
# dankclaw_default — scrapes http://192.168.0.235:9100/metrics
|
||||||
# (shared/prometheus/prometheus.yml, job "node-dankserver"). It cannot use
|
# (shared/prometheus/prometheus.yml, job "node-dankserver"). node_exporter has no
|
||||||
# loopback across hosts, and node_exporter has no authentication, so the
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
# wildcard bind stays off the table. This is the same deliberate LAN
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
# listener posture the observability runbook documents.
|
# host, and both consumers are accounted for.
|
||||||
#
|
#
|
||||||
# MIRROR. Keep this file in step with dankclaw/observability/quadlet/
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
# node-exporter.container; only the Description and the LAN address differ.
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Prometheus node exporter (Dankserver host metrics)
|
Description=Prometheus node exporter (Dankserver host metrics)
|
||||||
|
|||||||
@@ -26,8 +26,8 @@ RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn
|
|||||||
CALIB=/home/dank/corpora/calib
|
CALIB=/home/dank/corpora/calib
|
||||||
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
||||||
LOG=/home/dank/logs/swift15_convert.log
|
LOG=/home/dank/logs/swift15_convert.log
|
||||||
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
|
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert
|
||||||
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header).
|
# table; one encoding per expert table rules out a block-fp8 down tier -- see the recipe header).
|
||||||
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
|
||||||
|
|
||||||
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
|
||||||
|
|||||||
+54
-461
@@ -1,475 +1,68 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Plan runtime services affected by repository changes.
|
"""deploy-plan.py — map changed paths to deployable services.
|
||||||
|
|
||||||
The planner is intentionally side-effect free. It accepts either an explicit
|
Thin wrapper over scripts/service-map.py, which owns the path->service
|
||||||
path list or two Git revisions. The latter lets compose-file changes resolve
|
semantics in the shared inventory (shared/services.yml): per-service path
|
||||||
to only the service definitions that actually changed instead of restarting an
|
rules, the quadlet netns/network coupling graph, host scoping (a host's
|
||||||
entire compose project.
|
directory can only select that host's services), and canonical restart
|
||||||
|
order. This file keeps the deployer's CLI contract only.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
deploy-plan.py --target NAME --repo DIR --base REV --head REV
|
||||||
|
deploy-plan.py --target NAME [paths...] (paths also on stdin)
|
||||||
|
|
||||||
|
Prints one service name per line (canonical restart order); exit 0 with no
|
||||||
|
output when nothing relevant changed.
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
import os
|
||||||
|
|
||||||
import argparse
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
from pathlib import PurePosixPath
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import yaml
|
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
SERVICE_MAP = os.path.join(SCRIPT_DIR, "service-map.py")
|
||||||
# Dependency-sensitive services come before their consumers. Independent
|
|
||||||
# compose services follow. LLM services are deliberately last and are never
|
|
||||||
# implied by a generic host, stack, documentation, or config change.
|
|
||||||
SERVICE_ORDER = (
|
|
||||||
"gluetun",
|
|
||||||
"searxng-valkey",
|
|
||||||
"searxng",
|
|
||||||
"marreta",
|
|
||||||
"pihole",
|
|
||||||
"open-webui",
|
|
||||||
"gitea-postgres",
|
|
||||||
"gitea-server",
|
|
||||||
"comfyui",
|
|
||||||
"seerr",
|
|
||||||
"hermes",
|
|
||||||
"hermes-dashboard",
|
|
||||||
"hermes-webui",
|
|
||||||
"prometheus",
|
|
||||||
"grafana",
|
|
||||||
"loki",
|
|
||||||
"alloy",
|
|
||||||
"node-exporter",
|
|
||||||
"alertmanager",
|
|
||||||
"authentik-postgres",
|
|
||||||
"authentik-server",
|
|
||||||
"authentik-worker",
|
|
||||||
"authentik-outpost",
|
|
||||||
"authentik-outpost-ldap",
|
|
||||||
"caddy",
|
|
||||||
"homepage",
|
|
||||||
"romm-mariadb",
|
|
||||||
"romm-redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
"calibre-server",
|
|
||||||
"plex",
|
|
||||||
"tautulli",
|
|
||||||
"prowlarr",
|
|
||||||
"qbittorrent",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
"frank-dragoon",
|
|
||||||
"frank-zealot",
|
|
||||||
"frank-archon",
|
|
||||||
)
|
|
||||||
|
|
||||||
DANKCLAW_SERVICES = {
|
|
||||||
"hermes-webui",
|
|
||||||
"gluetun",
|
|
||||||
"searxng-valkey",
|
|
||||||
"searxng",
|
|
||||||
"marreta",
|
|
||||||
"pihole",
|
|
||||||
"open-webui",
|
|
||||||
"gitea-postgres",
|
|
||||||
"gitea-server",
|
|
||||||
"comfyui",
|
|
||||||
"seerr",
|
|
||||||
"hermes",
|
|
||||||
"hermes-dashboard",
|
|
||||||
"prometheus",
|
|
||||||
"grafana",
|
|
||||||
"loki",
|
|
||||||
"alloy",
|
|
||||||
"node-exporter",
|
|
||||||
"alertmanager",
|
|
||||||
"authentik-postgres",
|
|
||||||
"authentik-server",
|
|
||||||
"authentik-worker",
|
|
||||||
"authentik-outpost",
|
|
||||||
"authentik-outpost-ldap",
|
|
||||||
"caddy",
|
|
||||||
"homepage",
|
|
||||||
"frank-dragoon",
|
|
||||||
"frank-zealot",
|
|
||||||
}
|
|
||||||
|
|
||||||
DANKSERVER_SERVICES = {
|
|
||||||
"romm-mariadb",
|
|
||||||
"romm-redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
"calibre-server",
|
|
||||||
"plex",
|
|
||||||
"tautulli",
|
|
||||||
"prowlarr",
|
|
||||||
"qbittorrent",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
"frank-archon",
|
|
||||||
"node-exporter",
|
|
||||||
}
|
|
||||||
|
|
||||||
# FrankBook-Pro runs the *arr stack + qBittorrent inside a podman machine on
|
|
||||||
# macOS. Service names overlap the Linux hosts (gluetun, qbittorrent, ...);
|
|
||||||
# the per-target sets and the frankbook-pro intercept in deploy-remote.sh
|
|
||||||
# keep dispatch unambiguous.
|
|
||||||
FRANKBOOK_SERVICES = {
|
|
||||||
"gluetun",
|
|
||||||
"qbittorrent",
|
|
||||||
"prowlarr",
|
|
||||||
"radarr",
|
|
||||||
"sonarr",
|
|
||||||
"lidarr",
|
|
||||||
"readarr",
|
|
||||||
}
|
|
||||||
|
|
||||||
DANKCLAW_COMPOSE_SERVICES = set()
|
|
||||||
# Dankserver's compose file is still the source of truth for its media stack;
|
|
||||||
# Dankclaw's is gone (deleted 2026-09-16). Every service on Dankclaw is a
|
|
||||||
# Quadlet unit now — the last compose services were the hermes trio, whose units
|
|
||||||
# have owned their containers since 2026-09-12 and whose compose blocks were
|
|
||||||
# dead weight that could only create a duplicate container.
|
|
||||||
|
|
||||||
DANKSERVER_COMPOSE_SERVICES = {
|
|
||||||
"mariadb",
|
|
||||||
"redis",
|
|
||||||
"romm",
|
|
||||||
"copyparty",
|
|
||||||
"jellyfin",
|
|
||||||
"calibre-web",
|
|
||||||
}
|
|
||||||
COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
|
|
||||||
|
|
||||||
|
|
||||||
|
def git_paths(repo: str, base: str, head: str) -> list[str]:
|
||||||
def _add(services: set[str], *names: str) -> None:
|
proc = subprocess.run(
|
||||||
services.update(names)
|
["git", "-C", repo, "diff", "--name-only", base, head],
|
||||||
|
text=True, capture_output=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
def target_services(target: str) -> set[str]:
|
sys.stderr.write(f"deploy-plan: git diff failed: {proc.stderr}")
|
||||||
if target == "dankclaw":
|
raise SystemExit(1)
|
||||||
return DANKCLAW_SERVICES
|
return [line for line in proc.stdout.splitlines() if line]
|
||||||
if target == "dankserver":
|
|
||||||
return DANKSERVER_SERVICES
|
|
||||||
if target == "frankbook-pro":
|
|
||||||
return FRANKBOOK_SERVICES
|
|
||||||
raise ValueError(f"unknown target: {target}")
|
|
||||||
|
|
||||||
|
|
||||||
def _compose_services(target: str) -> set[str]:
|
|
||||||
return DANKCLAW_COMPOSE_SERVICES if target == "dankclaw" else DANKSERVER_COMPOSE_SERVICES
|
|
||||||
|
|
||||||
|
|
||||||
def _git_show(repo: str, revision: str, path: str) -> str | None:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", repo, "show", f"{revision}:{path}"],
|
|
||||||
text=True,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
return result.stdout if result.returncode == 0 else None
|
|
||||||
|
|
||||||
|
|
||||||
def _yaml_document(contents: str | None) -> dict[str, Any]:
|
|
||||||
if contents is None:
|
|
||||||
return {}
|
|
||||||
try:
|
|
||||||
data = yaml.safe_load(contents) or {}
|
|
||||||
except yaml.YAMLError:
|
|
||||||
return {}
|
|
||||||
return data if isinstance(data, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def _yaml_services(contents: str | None) -> dict[str, Any]:
|
|
||||||
services = _yaml_document(contents).get("services", {})
|
|
||||||
return services if isinstance(services, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def compose_changes(repo: str, target: str, base: str, head: str) -> set[str]:
|
|
||||||
"""Return only compose services whose YAML definitions or dependencies changed.
|
|
||||||
|
|
||||||
A target with no compose file (Dankclaw since 2026-09-16) plans nothing:
|
|
||||||
its services are Quadlet units, selected by their own paths.
|
|
||||||
"""
|
|
||||||
path = f"{target}/compose.yaml"
|
|
||||||
if not _git_show(repo, head, path) and not _git_show(repo, base, path):
|
|
||||||
return set()
|
|
||||||
old_doc = _yaml_document(_git_show(repo, base, path))
|
|
||||||
new_doc = _yaml_document(_git_show(repo, head, path))
|
|
||||||
old = old_doc.get("services", {}) if isinstance(old_doc.get("services", {}), dict) else {}
|
|
||||||
new = new_doc.get("services", {}) if isinstance(new_doc.get("services", {}), dict) else {}
|
|
||||||
names = set(old) | set(new)
|
|
||||||
allowed = _compose_services(target)
|
|
||||||
|
|
||||||
# An unavailable or malformed revision is safer as a broad compose plan
|
|
||||||
# than as a silent no-op. It still cannot include LLM units.
|
|
||||||
if (old_doc and not new_doc) or (new_doc and not old_doc and names):
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
|
||||||
|
|
||||||
# Top-level networks/configuration can affect every compose service even
|
|
||||||
# when no individual service mapping changed. Ignore the services key while
|
|
||||||
# comparing the rest of the Compose document.
|
|
||||||
old_globals = {key: value for key, value in old_doc.items() if key != "services"}
|
|
||||||
new_globals = {key: value for key, value in new_doc.items() if key != "services"}
|
|
||||||
if old_globals != new_globals:
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
|
||||||
|
|
||||||
changed = {name for name in names if name in allowed and old.get(name) != new.get(name)}
|
|
||||||
return {COMPOSE_NAME_MAP.get(n, n) for n in changed}
|
|
||||||
|
|
||||||
|
|
||||||
def services_for_path(path: str, target: str) -> set[str]:
|
|
||||||
"""Return services affected by one repository-relative path."""
|
|
||||||
path = path.replace("\\", "/")
|
|
||||||
p = PurePosixPath(path)
|
|
||||||
parts = p.parts
|
|
||||||
services: set[str] = set()
|
|
||||||
if not parts:
|
|
||||||
return services
|
|
||||||
|
|
||||||
# Non-runtime material is deliberately a no-op.
|
|
||||||
if parts[0] in {"docs", "tests", "old", ".githooks"}:
|
|
||||||
return services
|
|
||||||
if path in {"README.md", "AGENTS.md"} or parts[0] == ".obsidian":
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "shared":
|
|
||||||
if len(parts) < 2:
|
|
||||||
return services
|
|
||||||
group = parts[1]
|
|
||||||
if group == "searxng":
|
|
||||||
_add(services, "searxng")
|
|
||||||
elif group == "prometheus":
|
|
||||||
_add(services, "prometheus")
|
|
||||||
if len(parts) >= 3 and parts[2] == "alertmanager.yml":
|
|
||||||
# The Alertmanager config ships from this directory: a change
|
|
||||||
# to it must redeploy the service that installs it.
|
|
||||||
_add(services, "alertmanager")
|
|
||||||
elif group == "loki":
|
|
||||||
_add(services, "loki")
|
|
||||||
elif group == "alloy":
|
|
||||||
_add(services, "alloy")
|
|
||||||
elif group == "grafana":
|
|
||||||
# Provisioned datasources and dashboards: git is the source of
|
|
||||||
# truth, so a change here must restart the service that reads them.
|
|
||||||
_add(services, "grafana")
|
|
||||||
elif group == "homepage":
|
|
||||||
# Homepage is a Dankclaw-owned deployment. Shared homepage files
|
|
||||||
# are intentionally not copied during a Dankserver-only run.
|
|
||||||
if target == "dankclaw":
|
|
||||||
_add(services, "homepage")
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "dankclaw":
|
|
||||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
|
||||||
# Dankclaw has no compose file since 2026-09-16: every service is a
|
|
||||||
# Quadlet unit, each selected by its own path. A change (or a
|
|
||||||
# deletion) of this path therefore restarts nothing.
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "Caddyfile":
|
|
||||||
_add(services, "caddy")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "Containerfile.comfyui":
|
|
||||||
_add(services, "comfyui")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] in {"caddy", "comfyui", "loki", "alloy", "hermes", "anubis", "seerr"}:
|
|
||||||
_add(services, parts[1])
|
|
||||||
return services
|
|
||||||
if len(parts) >= 4 and parts[1] == "authentik" and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name in {"authentik-postgres", "authentik-server", "authentik-worker", "authentik-outpost", "authentik-outpost-ldap"}:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "searxng":
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3]
|
|
||||||
if name == "gluetun.container":
|
|
||||||
# SearXNG and Marreta (+ its browser companion) share
|
|
||||||
# Gluetun's network namespace. A Gluetun recreation
|
|
||||||
# tears the netns down, so every sharer must restart.
|
|
||||||
_add(services, "gluetun", "searxng", "marreta")
|
|
||||||
elif name == "searxng.container":
|
|
||||||
_add(services, "searxng")
|
|
||||||
elif name == "searxng-valkey.container":
|
|
||||||
_add(services, "searxng-valkey", "searxng")
|
|
||||||
elif name == "searxng.network":
|
|
||||||
_add(services, "gluetun", "searxng-valkey", "searxng", "marreta")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "marreta":
|
|
||||||
# One deployable owns both the app and its browser companion;
|
|
||||||
# both live in gluetun's netns. A change to either unit selects
|
|
||||||
# the marreta service, which restarts browser-then-app.
|
|
||||||
_add(services, "marreta")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "pihole":
|
|
||||||
_add(services, "pihole")
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "open-webui":
|
|
||||||
_add(services, "open-webui")
|
|
||||||
return services
|
|
||||||
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
|
||||||
# build, root unit): its files live in scripts/ and map there.
|
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
|
||||||
# The five observability containers became Quadlet units on
|
|
||||||
# 2026-09-11. A unit change selects exactly its own service;
|
|
||||||
# SERVICE_ORDER already puts each dependency ahead of its
|
|
||||||
# consumer (prometheus before grafana, loki before alloy).
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter", "alertmanager"}:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
if len(parts) >= 2 and parts[1] == "gitea":
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3]
|
|
||||||
if name == "gitea-postgres.container":
|
|
||||||
_add(services, "gitea-postgres")
|
|
||||||
elif name == "gitea-server.container":
|
|
||||||
_add(services, "gitea-server")
|
|
||||||
elif name == "gitea-caddy.container":
|
|
||||||
# Removing the old split proxy requires the canonical
|
|
||||||
# Caddy handler to retire its installed unit/container
|
|
||||||
# before it claims ports 80/443.
|
|
||||||
_add(services, "caddy")
|
|
||||||
elif name == "gitea.network":
|
|
||||||
_add(services, "gitea-postgres", "gitea-server")
|
|
||||||
return services
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "dankserver":
|
|
||||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
|
||||||
# Compose service keys map onto the deployed unit names.
|
|
||||||
_add(services, *(COMPOSE_NAME_MAP.get(n, n) for n in DANKSERVER_COMPOSE_SERVICES))
|
|
||||||
return services
|
|
||||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
|
||||||
quadlet_services = {
|
|
||||||
"romm": {"romm", "romm-mariadb", "romm-redis"},
|
|
||||||
"jellyfin": {"jellyfin"},
|
|
||||||
"calibre-web": {"calibre-web"},
|
|
||||||
"calibre-server": {"calibre-server"},
|
|
||||||
"copyparty": {"copyparty"},
|
|
||||||
"plex": {"plex"},
|
|
||||||
"tautulli": {"tautulli"},
|
|
||||||
}
|
|
||||||
_add(services, *quadlet_services.get(parts[1], set()))
|
|
||||||
if len(parts) >= 2 and parts[1] == "observability":
|
|
||||||
# Dankserver's node-exporter became a Quadlet unit on 2026-09-16
|
|
||||||
# (serverAppConfigs-docs/Infrastructure/Observability.md): its
|
|
||||||
# compose service died at a reboot and stayed dead, so Prometheus'
|
|
||||||
# node-dankserver job had nothing to scrape.
|
|
||||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name == "node-exporter":
|
|
||||||
_add(services, "node-exporter")
|
|
||||||
return services
|
|
||||||
# frank-archon's quadlet-era directory left the tree on 2026-09-26
|
|
||||||
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
|
|
||||||
# the trio in scripts/ now maps it). Nothing lives under
|
|
||||||
# dankserver/frank-archon/ except dated trash, and trash plans
|
|
||||||
# nothing (the retire rule above the service blocks).
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "frankbook-pro":
|
|
||||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
|
||||||
name = parts[3].removesuffix(".container")
|
|
||||||
if name == "gluetun":
|
|
||||||
# Every *arr + qBittorrent container shares gluetun's
|
|
||||||
# network namespace; a gluetun recreation kills all of
|
|
||||||
# them, so the plan restarts the whole netns family.
|
|
||||||
_add(services, "gluetun", "qbittorrent",
|
|
||||||
"prowlarr", "sonarr", "radarr", "lidarr", "readarr")
|
|
||||||
elif name in FRANKBOOK_SERVICES:
|
|
||||||
_add(services, name)
|
|
||||||
return services
|
|
||||||
|
|
||||||
if parts[0] == "scripts":
|
|
||||||
name = p.name
|
|
||||||
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
|
|
||||||
_add(services, "homepage")
|
|
||||||
if name in {"signal-relay.py", "signal-relay.service"}:
|
|
||||||
# The Signal relay is installed by the alertmanager deploy, so a
|
|
||||||
# change to it redeploying alertmanager is the only path that
|
|
||||||
# reaches it on the host.
|
|
||||||
_add(services, "alertmanager")
|
|
||||||
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
|
|
||||||
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
|
|
||||||
# files here; each one selects frank-zealot only. The ik_llama.cpp
|
|
||||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
|
||||||
# which is archive material and plans nothing.
|
|
||||||
_add(services, "frank-zealot")
|
|
||||||
if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
|
|
||||||
# CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
|
|
||||||
# serve files here the same way: a root unit on a host-built
|
|
||||||
# runtime. The download script fetches the payload, which is
|
|
||||||
# host-side state, so it selects nothing.
|
|
||||||
_add(services, "frank-dragoon")
|
|
||||||
if name == "qwen-sharp-v22.4.0.jinja":
|
|
||||||
# The Qwen-Sharp chat template frank-dragoon loads via
|
|
||||||
# --chat-template-file. It lived under dankserver/frank-medium/ until
|
|
||||||
# the 2026-09-26 cutover retired that directory; editing it changes
|
|
||||||
# what the live frank-dragoon endpoint renders, so it plans
|
|
||||||
# frank-dragoon.
|
|
||||||
_add(services, "frank-dragoon")
|
|
||||||
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
|
|
||||||
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
|
|
||||||
# DS4.1 quadlet) keeps its unit, env and serve files here the
|
|
||||||
# same way: a root unit on the unsloth b11030-mix prebuilt
|
|
||||||
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
|
||||||
# the payload, which is host-side state, so it selects nothing.
|
|
||||||
_add(services, "frank-archon")
|
|
||||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
|
||||||
# restart a runtime service.
|
|
||||||
return services
|
|
||||||
|
|
||||||
# Examples and env templates are not deployed runtime state.
|
|
||||||
if p.name in {".env.example", "env.example"}:
|
|
||||||
return services
|
|
||||||
return services
|
|
||||||
|
|
||||||
def changed_paths(repo: str, base: str, head: str) -> list[str]:
|
|
||||||
result = subprocess.run(
|
|
||||||
["git", "-C", repo, "diff", "--name-only", "--diff-filter=ACDMRTUXB", base, head],
|
|
||||||
text=True,
|
|
||||||
stdout=subprocess.PIPE,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
return [line for line in result.stdout.splitlines() if line]
|
|
||||||
|
|
||||||
|
|
||||||
def plan(paths: list[str], target: str, repo: str | None = None,
|
|
||||||
base: str | None = None, head: str | None = None) -> list[str]:
|
|
||||||
services: set[str] = set()
|
|
||||||
for path in paths:
|
|
||||||
if path == f"{target}/compose.yaml" and repo and base and head:
|
|
||||||
services.update(compose_changes(repo, target, base, head))
|
|
||||||
else:
|
|
||||||
services.update(services_for_path(path, target))
|
|
||||||
allowed = target_services(target)
|
|
||||||
return [name for name in SERVICE_ORDER if name in services and name in allowed]
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
args = sys.argv[1:]
|
||||||
parser.add_argument("--target", choices=("dankclaw", "dankserver", "frankbook-pro"), required=True)
|
if not args or args[0] != "--target":
|
||||||
parser.add_argument("--paths", nargs="+")
|
print(__doc__, file=sys.stderr)
|
||||||
parser.add_argument("--repo", help="repository path for revision-aware planning")
|
return 2
|
||||||
parser.add_argument("--base", help="base Git revision")
|
if len(args) < 2:
|
||||||
parser.add_argument("--head", help="head Git revision")
|
print("missing target name after --target", file=sys.stderr)
|
||||||
args = parser.parse_args()
|
return 2
|
||||||
|
target = args[1]
|
||||||
if not args.paths and not (args.repo and args.base and args.head):
|
# `--paths` is the historic seam keyword: a flag, not an option — the
|
||||||
parser.error("provide --paths or all of --repo, --base, and --head")
|
# paths are the positional arguments that follow it.
|
||||||
paths = args.paths or changed_paths(args.repo, args.base, args.head)
|
rest = [a for a in args[2:] if a != "--paths"]
|
||||||
print("\n".join(plan(paths, args.target, args.repo, args.base, args.head)))
|
if "--repo" in rest:
|
||||||
|
try:
|
||||||
|
repo = rest[rest.index("--repo") + 1]
|
||||||
|
base = rest[rest.index("--base") + 1]
|
||||||
|
head = rest[rest.index("--head") + 1]
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
print("--repo, --base and --head must appear together", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
paths = git_paths(repo, base, head)
|
||||||
|
else:
|
||||||
|
paths = rest or [line.strip() for line in sys.stdin if line.strip()]
|
||||||
|
if not paths:
|
||||||
|
return 0
|
||||||
|
proc = subprocess.run([sys.executable, SERVICE_MAP, "plan", "--host", target] + paths,
|
||||||
|
text=True, capture_output=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
sys.stderr.write(proc.stderr)
|
||||||
|
return proc.returncode
|
||||||
|
sys.stdout.write(proc.stdout)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+163
-233
@@ -9,6 +9,13 @@
|
|||||||
# pending until its service is explicitly deployed.
|
# pending until its service is explicitly deployed.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
# The LLM handlers' artifact gates are the serve scripts' own library, so
|
||||||
|
# the deployer and the unit refuse the same missing payload with the same
|
||||||
|
# words — no deploy can pass a gate the unit would fail. GATE_PREFIX is set
|
||||||
|
# per handler.
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib/llm-gates.sh"
|
||||||
|
|
||||||
TARGET="${1:-}"
|
TARGET="${1:-}"
|
||||||
REVISION="${2:-}"
|
REVISION="${2:-}"
|
||||||
shift 2 || true
|
shift 2 || true
|
||||||
@@ -80,31 +87,10 @@ cd "$REMOTE_DIR"
|
|||||||
mkdir -p "$STATE_DIR"
|
mkdir -p "$STATE_DIR"
|
||||||
|
|
||||||
service_is_allowed() {
|
service_is_allowed() {
|
||||||
local service="$1"
|
# The inventory (shared/services.yml via scripts/service-map.py) is the
|
||||||
case "$TARGET:$service" in
|
# only allow-list: a service is deployable on a target iff the inventory
|
||||||
dankclaw:gluetun|dankclaw:searxng-valkey|dankclaw:searxng|dankclaw:pihole|\
|
# places it there. Retiring a service from a host is a data edit.
|
||||||
dankclaw:open-webui|dankclaw:gitea-postgres|dankclaw:gitea-server|\
|
python3 scripts/service-map.py allows --host "$TARGET" --service "$1"
|
||||||
dankclaw:comfyui|dankclaw:hermes|\
|
|
||||||
dankclaw:hermes-dashboard|dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|\
|
|
||||||
dankclaw:loki|dankclaw:alloy|dankclaw:node-exporter|dankclaw:alertmanager|\
|
|
||||||
dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
|
|
||||||
dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
|
|
||||||
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
|
||||||
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
|
|
||||||
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
|
|
||||||
dankserver:frank-archon|dankserver:node-exporter|\
|
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
|
||||||
frankbook-pro:readarr)
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
mark_status() {
|
mark_status() {
|
||||||
@@ -274,13 +260,11 @@ install_signal_relay() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
copy_loki_config() {
|
copy_loki_config() {
|
||||||
run_privileged mkdir -p /opt/loki/config
|
copy_config /opt/loki/config shared/loki/loki-config.yaml
|
||||||
run_privileged cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
|
|
||||||
}
|
}
|
||||||
|
|
||||||
copy_alloy_config() {
|
copy_alloy_config() {
|
||||||
run_privileged mkdir -p /opt/alloy/config
|
copy_config /opt/alloy/config shared/alloy/config.alloy
|
||||||
run_privileged cp shared/alloy/config.alloy /opt/alloy/config/config.alloy
|
|
||||||
}
|
}
|
||||||
|
|
||||||
copy_grafana_provisioning() {
|
copy_grafana_provisioning() {
|
||||||
@@ -289,14 +273,11 @@ copy_grafana_provisioning() {
|
|||||||
# datasource would be invisible to review, lost with the data volume, and
|
# datasource would be invisible to review, lost with the data volume, and
|
||||||
# unknown to anyone reading the repository. The unit already mounts
|
# unknown to anyone reading the repository. The unit already mounts
|
||||||
# /opt/grafana/provisioning read-only, so this only has to populate it.
|
# /opt/grafana/provisioning read-only, so this only has to populate it.
|
||||||
run_privileged mkdir -p /opt/grafana/provisioning/datasources \
|
copy_config /opt/grafana/provisioning/datasources \
|
||||||
/opt/grafana/provisioning/dashboards
|
shared/grafana/provisioning/datasources/prometheus.yml
|
||||||
run_privileged cp shared/grafana/provisioning/datasources/prometheus.yml \
|
copy_config /opt/grafana/provisioning/dashboards \
|
||||||
/opt/grafana/provisioning/datasources/prometheus.yml
|
shared/grafana/provisioning/dashboards/provider.yml \
|
||||||
run_privileged cp shared/grafana/provisioning/dashboards/provider.yml \
|
shared/grafana/provisioning/dashboards/llm-fleet.json
|
||||||
/opt/grafana/provisioning/dashboards/provider.yml
|
|
||||||
run_privileged cp shared/grafana/provisioning/dashboards/llm-fleet.json \
|
|
||||||
/opt/grafana/provisioning/dashboards/llm-fleet.json
|
|
||||||
}
|
}
|
||||||
|
|
||||||
render_caddyfile() {
|
render_caddyfile() {
|
||||||
@@ -441,31 +422,6 @@ deploy_caddy() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
deploy_compose_service() {
|
|
||||||
local service="$1"
|
|
||||||
if [[ "$TARGET" == dankserver ]]; then
|
|
||||||
# Dankserver's compose file is the source-of-truth inventory, but its
|
|
||||||
# production containers are individually systemd-managed/bare. Do not
|
|
||||||
# use compose up here: it can create a second container with a wrong
|
|
||||||
# name and can interfere with a running workload.
|
|
||||||
echo "WARN: $service is recorded in dankserver/compose.yaml; production container management is host-specific and no restart was attempted."
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
if [[ "$TARGET" == dankclaw ]]; then
|
|
||||||
# Dankclaw has no compose file since 2026-09-16: every service is a
|
|
||||||
# Quadlet unit with its own handler above. Reaching this means the
|
|
||||||
# dispatcher is missing a case, and the old fallback (`podman-compose
|
|
||||||
# up -d`) would now be a silent no-op or a duplicate container.
|
|
||||||
echo "no compose services on dankclaw; $service has no quadlet handler" >&2
|
|
||||||
return 2
|
|
||||||
fi
|
|
||||||
case "$service" in
|
|
||||||
*)
|
|
||||||
echo "unknown compose service: $service" >&2
|
|
||||||
return 2
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
# grafana is the one observability service that holds a credential: the
|
# grafana is the one observability service that holds a credential: the
|
||||||
# Authentik OAuth2 client secret. It is seeded into a per-service env file (the
|
# Authentik OAuth2 client secret. It is seeded into a per-service env file (the
|
||||||
@@ -568,6 +524,35 @@ quadlet_unit_dir="$HOME/.config/containers/systemd"
|
|||||||
# dankserver's user manager had no systemd container generator dir before the
|
# dankserver's user manager had no systemd container generator dir before the
|
||||||
# quadlet migration; every handler that installs a unit must find it present.
|
# quadlet migration; every handler that installs a unit must find it present.
|
||||||
mkdir -p "$quadlet_unit_dir"
|
mkdir -p "$quadlet_unit_dir"
|
||||||
|
# copy_config DEST SRC... — the privileged config copy every service uses:
|
||||||
|
# ensure DEST, copy each tracked SRC into it. The deployer owns /opt state;
|
||||||
|
# the files in git are the source of truth, so a service config that only
|
||||||
|
# exists on the host is a gap by policy.
|
||||||
|
copy_config() {
|
||||||
|
local dest="$1"; shift
|
||||||
|
run_privileged mkdir -p "$dest"
|
||||||
|
local src
|
||||||
|
for src in "$@"; do
|
||||||
|
run_privileged cp "$src" "$dest/"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# retire_unit_file PATH TRASH [UNIT] — the repo Deletion & Trash policy, in
|
||||||
|
# code: stop and disable the named unit, move the unit FILE into the home
|
||||||
|
# trash (never rm), report. The dragoon quadlet retirement and the
|
||||||
|
# compose-era comfyui.service retirement were the same six lines twice.
|
||||||
|
retire_unit_file() {
|
||||||
|
local path="$1" trash="$2" unit="${3:-}"
|
||||||
|
[[ -e "$path" ]] || return 0
|
||||||
|
if [[ -n "$unit" ]]; then
|
||||||
|
systemctl --user stop "$unit" 2>/dev/null || true
|
||||||
|
systemctl --user disable "$unit" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
mkdir -p "$trash"
|
||||||
|
mv -- "$path" "$trash/$(basename "$path")"
|
||||||
|
echo "retired $(basename "$path") -> $trash"
|
||||||
|
}
|
||||||
|
|
||||||
install_quadlet() {
|
install_quadlet() {
|
||||||
local service="$1"
|
local service="$1"
|
||||||
local source_dir="$2"
|
local source_dir="$2"
|
||||||
@@ -655,15 +640,8 @@ deploy_comfyui() {
|
|||||||
# generator produces, and it is what kept a stale, device-less container
|
# generator produces, and it is what kept a stale, device-less container
|
||||||
# alive through `restart: unless-stopped` (9,146 failures on
|
# alive through `restart: unless-stopped` (9,146 failures on
|
||||||
# "No HIP GPUs are available"). Never `rm`: it moves into the home trash.
|
# "No HIP GPUs are available"). Never `rm`: it moves into the home trash.
|
||||||
local legacy="$HOME/.config/systemd/user/comfyui.service"
|
retire_unit_file "$HOME/.config/systemd/user/comfyui.service" \
|
||||||
if [[ -e "$legacy" ]]; then
|
"$HOME/.AGENTTRASH/20260916-comfyui-compose-unit" comfyui.service
|
||||||
local trash="$HOME/.AGENTTRASH/20260916-comfyui-compose-unit"
|
|
||||||
mkdir -p "$trash"
|
|
||||||
systemctl --user stop comfyui.service 2>/dev/null || true
|
|
||||||
systemctl --user disable comfyui.service 2>/dev/null || true
|
|
||||||
mv -- "$legacy" "$trash/comfyui.service"
|
|
||||||
echo "retired the compose-era comfyui.service -> $trash"
|
|
||||||
fi
|
|
||||||
# The stale container carries the old config (no /dev/kfd, no /dev/dri) and
|
# The stale container carries the old config (no /dev/kfd, no /dev/dri) and
|
||||||
# would keep the quadlet's container from taking the name. It is replaced
|
# would keep the quadlet's container from taking the name. It is replaced
|
||||||
# by the unit in the next step, so nothing is lost.
|
# by the unit in the next step, so nothing is lost.
|
||||||
@@ -721,9 +699,15 @@ deploy_gitea_unit() {
|
|||||||
systemctl --user restart "$service.service"
|
systemctl --user restart "$service.service"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The root units are symlinks from the systemd system directory into the
|
||||||
|
# checkout. SYSTEMD_SYSTEM_DIR is production (/etc/systemd/system); the knob
|
||||||
|
# exists so the deploy seam can run a whole handler — relink included — in a
|
||||||
|
# sandbox and read the unit links as evidence, unprivileged.
|
||||||
|
SYSTEMD_SYSTEM_DIR="${SYSTEMD_SYSTEM_DIR:-/etc/systemd/system}"
|
||||||
|
|
||||||
relink_unit() {
|
relink_unit() {
|
||||||
local service="$1"
|
local service="$1"
|
||||||
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "/etc/systemd/system/$service.service"
|
run_privileged ln -sfn "$REMOTE_DIR/scripts/$service.service" "$SYSTEMD_SYSTEM_DIR/$service.service"
|
||||||
}
|
}
|
||||||
|
|
||||||
# Authentik runs as three boot-persistent Quadlet units (postgres -> server ->
|
# Authentik runs as three boot-persistent Quadlet units (postgres -> server ->
|
||||||
@@ -942,17 +926,17 @@ deploy_plex() {
|
|||||||
systemctl --user restart plex.service
|
systemctl --user restart plex.service
|
||||||
# Readiness gate: /identity answers MediaContainer XML once PMS is up.
|
# Readiness gate: /identity answers MediaContainer XML once PMS is up.
|
||||||
#
|
#
|
||||||
# HTTPS, NOT HTTP. PMS drops plain HTTP off-loopback: the probe's
|
# HTTPS, NOT HTTP. PMS serves plain HTTP on loopback only: a probe
|
||||||
# connection arrives on the container's eth0, so an http:// request is
|
# arriving on the container's interface gets its connection accepted and
|
||||||
# accepted and then answered with nothing (curl 52, Empty reply) against a
|
# then an empty reply (curl 52) from a perfectly healthy server, so an
|
||||||
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full
|
# http:// probe would burn the whole timeout and fail a service that
|
||||||
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered
|
# answers https://127.0.0.1:32400/identity in milliseconds. The Caddyfile
|
||||||
# 200 in 44 ms the whole time. The same law is why the Caddyfile dials the
|
# dials the same upstream over TLS for the same reason
|
||||||
# plex upstream over TLS (tests/caddyfile.bats pins it).
|
# (tests/caddyfile.bats pins it).
|
||||||
# -k: PMS presents its own self-signed cert on 32400.
|
# -k: PMS presents its own self-signed cert on 32400.
|
||||||
#
|
#
|
||||||
# Reality check from the 2026-09 cutover: the FIRST container boot of a
|
# The budget accounts for a first boot of a large pre-existing library,
|
||||||
# years-old library ran ~10 min, so the gate still allows 600s.
|
# which spends minutes indexing before it answers anything.
|
||||||
# Fail the deploy (pending state preserved) rather than mark it deployed blind.
|
# Fail the deploy (pending state preserved) rather than mark it deployed blind.
|
||||||
for ((i = 1; i <= 120; i++)); do
|
for ((i = 1; i <= 120; i++)); do
|
||||||
curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
|
curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
|
||||||
@@ -1125,13 +1109,8 @@ LLM_DRAGOON_TOK_DIR="${LLM_DRAGOON_TOK_DIR:-/home/dank/models/halogen-flashnext-
|
|||||||
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
LLM_DRAGOON_TOK_FILES="${LLM_DRAGOON_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json}"
|
||||||
|
|
||||||
retire_halogen_dragoon_quadlet() {
|
retire_halogen_dragoon_quadlet() {
|
||||||
local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
|
retire_unit_file "$quadlet_unit_dir/frank-dragoon.container" \
|
||||||
systemctl --user stop frank-dragoon.service 2>/dev/null || true
|
"$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet" frank-dragoon.service
|
||||||
if [[ -e "$quadlet_unit_dir/frank-dragoon.container" ]]; then
|
|
||||||
mkdir -p "$trash"
|
|
||||||
mv -- "$quadlet_unit_dir/frank-dragoon.container" "$trash/frank-dragoon.container"
|
|
||||||
echo "retired quadlet frank-dragoon.container -> $trash"
|
|
||||||
fi
|
|
||||||
systemctl --user daemon-reload
|
systemctl --user daemon-reload
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1153,73 +1132,71 @@ wait_port_free() {
|
|||||||
ss -tan "sport = :$port" >&2 || true
|
ss -tan "sport = :$port" >&2 || true
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
# assert_unit_owns_port SERVICE PORT — the false-green gate every LLM
|
||||||
|
# endpoint shares. A health answer proves the PORT replied, never WHO
|
||||||
|
# replied: on 2026-09-26 a leftover probe held :7770, satisfied the curl,
|
||||||
|
# and the deploy reported green over a stopped unit. Green requires the
|
||||||
|
# unit's own MainPID to be alive behind the answer.
|
||||||
|
assert_unit_owns_port() {
|
||||||
|
local service="$1" port="$2" main_pid
|
||||||
|
main_pid="$(run_privileged systemctl show -p MainPID --value "$service.service")"
|
||||||
|
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
||||||
|
echo "FAIL: :$port answered but $service.service has no MainPID — a stray process holds the port" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
echo "$service: health answered by unit MainPID $main_pid"
|
||||||
|
}
|
||||||
|
|
||||||
|
# start_root_unit — the tail every LLM root unit deploys: relink the unit
|
||||||
|
# file from the checkout into /etc/systemd/system, (re)enable, restart,
|
||||||
|
# wait for /health, and prove the answer came from the unit's own process.
|
||||||
|
# The three endpoints differ in gates, ports, and budgets, never in this.
|
||||||
|
start_root_unit() {
|
||||||
|
local service="$1" port="$2" health_limit="$3"
|
||||||
|
relink_unit "$service"
|
||||||
|
run_privileged systemctl daemon-reload
|
||||||
|
run_privileged systemctl enable "$service.service"
|
||||||
|
run_privileged systemctl restart "$service.service"
|
||||||
|
wait_for_health "$service" "http://127.0.0.1:${port}/health" "$health_limit" || return 1
|
||||||
|
assert_unit_owns_port "$service" "$port"
|
||||||
|
}
|
||||||
|
|
||||||
deploy_llm_dragoon() {
|
deploy_llm_dragoon() {
|
||||||
# Refuse rather than restart-loop; every gate below has a twin in
|
# Refuse rather than restart-loop; every gate here is the SAME library
|
||||||
# serve-frank-dragoon.sh and they must agree (2026-10-01 lesson).
|
# call the serve script makes, so the two twins cannot drift (the
|
||||||
# Capture podman output before grepping: `podman images | grep -q`
|
# 2026-10-01 crash-loop lesson).
|
||||||
# races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
|
# Capture podman output before grepping is moot: require_image is one
|
||||||
if ! grep -qF "${LLM_DRAGOON_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
|
# command, no pipe (the 2026-09-30 SIGPIPE lesson).
|
||||||
echo "ERROR: halogen image absent: $LLM_DRAGOON_IMAGE" >&2
|
export GATE_PREFIX=frank-dragoon
|
||||||
echo "frank-dragoon: pull it: podman pull ${LLM_DRAGOON_IMAGE%@*}" >&2
|
require_image "$LLM_DRAGOON_IMAGE" \
|
||||||
return 1
|
"pull it: podman pull docker://${LLM_DRAGOON_IMAGE%@*}" || return 1
|
||||||
fi
|
require_bytes "$LLM_DRAGOON_CKPT" checkpoint "$LLM_DRAGOON_CKPT_BYTES" \
|
||||||
if [[ ! -f "$LLM_DRAGOON_CKPT" ]]; then
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
echo "ERROR: halogen checkpoint absent at $LLM_DRAGOON_CKPT" >&2
|
require_bytes "$LLM_DRAGOON_NGRAM" n-gram "$LLM_DRAGOON_NGRAM_BYTES" \
|
||||||
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
return 1
|
require_bytes "$LLM_DRAGOON_VISION" vision "$LLM_DRAGOON_VISION_BYTES" \
|
||||||
fi
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh; re-check: --verify-only" || return 1
|
||||||
if [[ "$(stat -c %s "$LLM_DRAGOON_CKPT")" != "$LLM_DRAGOON_CKPT_BYTES" ]]; then
|
# shellcheck disable=SC2086
|
||||||
echo "ERROR: halogen checkpoint is not $LLM_DRAGOON_CKPT_BYTES bytes (partial download?)" >&2
|
require_dir_files "$LLM_DRAGOON_TOK_DIR" tokenizer \
|
||||||
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
|
||||||
return 1
|
$LLM_DRAGOON_TOK_FILES || return 1
|
||||||
fi
|
|
||||||
for gate in "$LLM_DRAGOON_NGRAM:$LLM_DRAGOON_NGRAM_BYTES:n-gram table" \
|
|
||||||
"$LLM_DRAGOON_VISION:$LLM_DRAGOON_VISION_BYTES:vision tower"; do
|
|
||||||
IFS=: read -r gpath gbytes gname <<<"$gate"
|
|
||||||
if [[ ! -f "$gpath" ]]; then
|
|
||||||
echo "ERROR: halogen $gname absent at $gpath" >&2
|
|
||||||
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ "$(stat -c %s "$gpath")" != "$gbytes" ]]; then
|
|
||||||
echo "ERROR: halogen $gname is not $gbytes bytes (partial download?)" >&2
|
|
||||||
echo "frank-dragoon: re-check: bash scripts/download-frank-dragoon-flashnext.sh --verify-only" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
if [[ -L "$LLM_DRAGOON_TOK_DIR" || ! -d "$LLM_DRAGOON_TOK_DIR" ]]; then
|
|
||||||
echo "ERROR: tokenizer dir missing or a symlink at $LLM_DRAGOON_TOK_DIR" >&2
|
|
||||||
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
for tf in $LLM_DRAGOON_TOK_FILES; do
|
|
||||||
if [[ ! -f "$LLM_DRAGOON_TOK_DIR/$tf" || -L "$LLM_DRAGOON_TOK_DIR/$tf" ]]; then
|
|
||||||
echo "ERROR: tokenizer file missing or a symlink: $LLM_DRAGOON_TOK_DIR/$tf" >&2
|
|
||||||
echo "frank-dragoon: fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
retire_halogen_dragoon_quadlet
|
retire_halogen_dragoon_quadlet
|
||||||
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
# The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
|
||||||
# the container) before waiting on the port. The stray sweep is keyed
|
# the container) before waiting on the port. The stray sweep is keyed
|
||||||
# on the FULL container name, which appears in the argv of a manual
|
# on the FULL container name: it appears in the argv of a manual
|
||||||
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
|
# `podman run --name frank-dragoon-halogen` / `podman wait`, while a
|
||||||
# this deployer's own command line (`bash deploy-remote.sh
|
# pattern naming only the runtime would ALSO match this deployer's own
|
||||||
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet
|
# command line — a pkill whose pattern hits its own argv takes the
|
||||||
# pkill law), and it can never match frank-zealot's llama-server.
|
# running deploy down with the target. The full name can never match
|
||||||
|
# frank-zealot's llama-server either.
|
||||||
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
|
||||||
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
|
||||||
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
|
||||||
# Nothing may start until the predecessor has released the port
|
# Nothing may start until the predecessor has released the port
|
||||||
# (2026-09-16: "Deactivated successfully" was not "port free").
|
# (2026-09-16: "Deactivated successfully" was not "port free").
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-dragoon
|
start_root_unit frank-dragoon 7770 1200
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-dragoon.service
|
|
||||||
run_privileged systemctl restart frank-dragoon.service
|
|
||||||
wait_for_health frank-dragoon http://127.0.0.1:7770/health 1200
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
|
# deploy_llm_zealot — frank-zealot is llama.cpp (master build, CUDA sm_86) on
|
||||||
@@ -1243,37 +1220,18 @@ LLM_ZEALOT_MMPROJ="${LLM_ZEALOT_MMPROJ:-/home/dank/models/Qwen35-9B-HauhauCS-Q6K
|
|||||||
deploy_llm_zealot() {
|
deploy_llm_zealot() {
|
||||||
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
# Refuse rather than restart-loop: a unit whose runtime or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why. Same library calls
|
||||||
if [[ ! -x "$LLM_ZEALOT_RUNTIME" ]]; then
|
# as serve-frank-zealot.sh, so the deployer's twin refuses identically.
|
||||||
echo "ERROR: llama.cpp runtime absent at $LLM_ZEALOT_RUNTIME" >&2
|
export GATE_PREFIX=frank-zealot
|
||||||
echo "frank-zealot: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
|
require_exec "$LLM_ZEALOT_RUNTIME" runtime \
|
||||||
echo "frank-zealot: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
|
"build it: cmake -B build-cuda -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86" \
|
||||||
echo "frank-zealot: && cmake --build build-cuda --target llama-server) first." >&2
|
|| return 1
|
||||||
return 1
|
require_file "$LLM_ZEALOT_MODEL_FILE" gguf \
|
||||||
fi
|
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
|
||||||
for f in "$LLM_ZEALOT_MODEL_FILE" "$LLM_ZEALOT_MMPROJ"; do
|
require_file "$LLM_ZEALOT_MMPROJ" mmproj \
|
||||||
if [[ ! -f "$f" ]]; then
|
"fetch it: bash scripts/download-frank-zealot.sh" || return 1
|
||||||
echo "ERROR: frank-zealot payload file absent at $f" >&2
|
|
||||||
echo "frank-zealot: run scripts/download-frank-zealot.sh first;" >&2
|
start_root_unit frank-zealot 7771 900
|
||||||
echo "frank-zealot: refusing to start a unit that cannot load." >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
relink_unit frank-zealot
|
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-zealot.service
|
|
||||||
run_privileged systemctl restart frank-zealot.service
|
|
||||||
wait_for_health frank-zealot http://127.0.0.1:7771/health 900 || return 1
|
|
||||||
# False-green hardening (frank-dragoon's lesson): the port answering
|
|
||||||
# proves nothing about WHO answers — the unit must be active with a
|
|
||||||
# live MainPID, or a stray SGLang holdover is masquerading as deployed.
|
|
||||||
local main_pid
|
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-zealot.service)"
|
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
|
||||||
echo "FAIL: :7771 answered but frank-zealot.service has no MainPID — a stray process holds the port" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "frank-zealot: health answered by unit MainPID $main_pid"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
||||||
@@ -1300,11 +1258,16 @@ LLM_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-pr
|
|||||||
deploy_llm_archon() {
|
deploy_llm_archon() {
|
||||||
# Refuse rather than restart-loop: a unit whose image or payload is
|
# Refuse rather than restart-loop: a unit whose image or payload is
|
||||||
# missing never becomes healthy, and a green deploy would leave the
|
# missing never becomes healthy, and a green deploy would leave the
|
||||||
# endpoint dark with nothing in the log to say why.
|
# endpoint dark with nothing in the log to say why. The gates are the
|
||||||
local env_file="/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
# shared library's, called with the values the unit's own env file
|
||||||
local image model_dir model_file rad_min mib
|
# declares — so serve-frank-archon.sh and this handler cannot disagree,
|
||||||
|
# and the payload path lives in exactly one file.
|
||||||
|
export GATE_PREFIX=frank-archon
|
||||||
|
local env_file="${LLM_ARCHON_ENV_FILE:-$REMOTE_DIR/scripts/frank-archon.env}"
|
||||||
|
local image model_dir model_file rad_min
|
||||||
if [[ ! -f "$env_file" ]]; then
|
if [[ ! -f "$env_file" ]]; then
|
||||||
echo "ERROR: frank-archon env file absent: $env_file" >&2
|
echo "frank-archon: env: missing at $env_file" >&2
|
||||||
|
echo " hint: deploy the repo checkout first ($REMOTE_DIR)" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
|
image="$(sed -n 's/^FRANK_ARCHON_IMAGE=//p' "$env_file" | tail -1)"
|
||||||
@@ -1312,28 +1275,16 @@ deploy_llm_archon() {
|
|||||||
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
|
model_file="$(sed -n 's/^FRANK_ARCHON_MODEL_FILE=//p' "$env_file" | tail -1)"
|
||||||
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
rad_min="$(sed -n 's/^FRANK_ARCHON_RAD_MIN_MIB=//p' "$env_file" | tail -1)"
|
||||||
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
if [[ -z "$image" || -z "$model_dir" || -z "$model_file" || -z "$rad_min" ]]; then
|
||||||
echo "ERROR: frank-archon.env missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
echo "frank-archon: env: missing FRANK_ARCHON_IMAGE/MODEL_DIR/MODEL_FILE/RAD_MIN_MIB" >&2
|
||||||
return 1
|
echo " hint: scripts/frank-archon.env" >&2
|
||||||
fi
|
|
||||||
# The image gate is the digest ref itself: a name@sha256 ref exists in
|
|
||||||
# storage only if THAT manifest was pulled. One command — no
|
|
||||||
# pipefail/grep race (the 2026-09-30 lesson).
|
|
||||||
if ! podman image exists "$image"; then
|
|
||||||
echo "ERROR: runtime image absent from storage: $image" >&2
|
|
||||||
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ ! -f "$model_dir/$model_file" ]]; then
|
|
||||||
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
|
||||||
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
|
||||||
if [[ "$mib" -lt "$rad_min" ]]; then
|
|
||||||
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
|
||||||
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
require_image "$image" \
|
||||||
|
"pull it: podman pull docker://stilldeadcode/radiance:latest (as dank)" \
|
||||||
|
|| return 1
|
||||||
|
require_min_mib "$model_dir/$model_file" ".rad" "$rad_min" \
|
||||||
|
"convert it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" \
|
||||||
|
|| return 1
|
||||||
|
|
||||||
# A stale container with the same name makes `podman run --name` fail at
|
# A stale container with the same name makes `podman run --name` fail at
|
||||||
# start; remove any leftover from a failed previous attempt.
|
# start; remove any leftover from a failed previous attempt.
|
||||||
@@ -1352,41 +1303,15 @@ deploy_llm_archon() {
|
|||||||
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
|
# (2026-09-26) and the frank-dragoon cutover (2026-09-16) both failed
|
||||||
# exactly here when the next engine bound 7770 a second too early.
|
# exactly here when the next engine bound 7770 a second too early.
|
||||||
wait_port_free 7770 || return 1
|
wait_port_free 7770 || return 1
|
||||||
relink_unit frank-archon
|
|
||||||
run_privileged systemctl daemon-reload
|
|
||||||
run_privileged systemctl enable frank-archon.service
|
|
||||||
run_privileged systemctl restart frank-archon.service
|
|
||||||
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
# Cold start: .rad load + PLE pin + expert host-pool upload (the
|
||||||
# publisher's compose healthcheck allows a 15-minute start_period).
|
# publisher's compose healthcheck allows a 15-minute start_period) —
|
||||||
# Parity with every predecessor on this port. The HTTP listener binds
|
# the 900-iteration budget is 1800s, parity with every predecessor on
|
||||||
# AFTER the full load (main.cpp: engine.start() then engine.serve()),
|
# this port. The HTTP listener binds AFTER the full load (main.cpp:
|
||||||
# so during load the port REFUSES connections — a bare curl is honest
|
# engine.start() then engine.serve()), so during load the port REFUSES
|
||||||
# here, unlike FreeToken's 200-while-loading body. (The MainPID check
|
# connections — a bare curl is honest here, unlike FreeToken's
|
||||||
# below still guards the stray-port case.)
|
# 200-while-loading body. (start_root_unit's MainPID check still
|
||||||
local i ok=0
|
# guards the stray-port case.)
|
||||||
for ((i=1; i<=900; i++)); do
|
start_root_unit frank-archon 7770 900
|
||||||
if curl -fsS http://127.0.0.1:7770/health 2>/dev/null >/dev/null; then
|
|
||||||
echo "frank-archon engine healthy after approximately $((i * 2))s"
|
|
||||||
ok=1
|
|
||||||
break
|
|
||||||
fi
|
|
||||||
sleep 2
|
|
||||||
done
|
|
||||||
if [[ "$ok" != 1 ]]; then
|
|
||||||
echo "FAIL: frank-archon /health never answered 200 in 1800s" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
# False-green hardening (the 2026-09-26 lesson): a bare curl against the
|
|
||||||
# port cannot tell WHO answers — a stray probe holding :7770 satisfies
|
|
||||||
# it while the unit is stopped. The deploy is only green if the unit
|
|
||||||
# itself is active with a live MainPID behind the health answer.
|
|
||||||
local main_pid
|
|
||||||
main_pid="$(run_privileged systemctl show -p MainPID --value frank-archon.service)"
|
|
||||||
if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
|
|
||||||
echo "FAIL: :7770 answered but frank-archon.service has no MainPID — a stray process holds the port" >&2
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
echo "frank-archon: health answered by unit MainPID $main_pid"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
wait_for_health() {
|
wait_for_health() {
|
||||||
@@ -1586,7 +1511,12 @@ deploy_service() {
|
|||||||
tautulli) deploy_tautulli ;;
|
tautulli) deploy_tautulli ;;
|
||||||
prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
|
prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
|
||||||
comfyui) deploy_comfyui ;;
|
comfyui) deploy_comfyui ;;
|
||||||
*) deploy_compose_service "$service" ;;
|
*)
|
||||||
|
# Every deployable service on every target has a handler above
|
||||||
|
# (inventory = allow-list, so nothing unreachable arrives).
|
||||||
|
echo "no handler for service: $service" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
esac
|
esac
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+7
-25
@@ -131,31 +131,13 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
valid_service() {
|
valid_service() {
|
||||||
local service="$1"
|
# The inventory is the only allow-list (see scripts/service-map.py);
|
||||||
case "$TARGET:$service" in
|
# service-map's "allows" exit code is the answer. The former literal
|
||||||
dankclaw:caddy|dankclaw:gluetun|dankclaw:searxng-network|dankclaw:searxng-valkey|\
|
# case list here drifted from reality (it still listed the *arr stack
|
||||||
dankclaw:searxng|dankclaw:marreta|dankclaw:pihole|dankclaw:open-webui|dankclaw:gitea-postgres|\
|
# under dankserver, which moved to frankbook-pro, and a
|
||||||
dankclaw:gitea-server|dankclaw:homepage|\
|
# "searxng-network" name that is a generated systemd unit, not a
|
||||||
dankclaw:comfyui|dankclaw:hermes|dankclaw:hermes-dashboard|\
|
# deployable service).
|
||||||
dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|dankclaw:loki|dankclaw:alloy|\
|
python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$TARGET" --service "$1" >/dev/null
|
||||||
dankclaw:alertmanager|\
|
|
||||||
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
|
||||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
|
||||||
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
|
||||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
|
||||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
|
||||||
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
|
|
||||||
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
|
|
||||||
dankserver:frank-archon|\
|
|
||||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
|
||||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
|
||||||
frankbook-pro:readarr)
|
|
||||||
return 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for service in "${EXPLICIT_SERVICES[@]}"; do
|
for service in "${EXPLICIT_SERVICES[@]}"; do
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
"""derive-protected-experts.py — rewrite the recipe's protected-expert section
|
||||||
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
|
||||||
|
|
||||||
The law (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
The selection rule (docs/MOE-W4.md "Protected experts", restated in the recipe):
|
||||||
an expert whose share of its layer's down-input energy is at least 3% AND
|
an expert whose share of its layer's down-input energy is at least 3% AND
|
||||||
at least three times its share of the routing is kept bf16; the count must
|
at least three times its share of the routing is kept bf16; the count must
|
||||||
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||||
|
|||||||
@@ -14,10 +14,11 @@
|
|||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
# Verify (default) is three independent agreements: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size (catches truncation), the shard count
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# matching the weight-map index (catches omissions), and every shard's
|
||||||
# only the public API.
|
# sha256 equal to the Hub's LFS oid (catches substituted content).
|
||||||
|
# Verify needs only the public API.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next"
|
||||||
|
|||||||
@@ -12,10 +12,11 @@
|
|||||||
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
|
||||||
# listing/resume downloader, curl+jq only): per-file size checks against
|
# listing/resume downloader, curl+jq only): per-file size checks against
|
||||||
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
# the Hub listing, `-C -` resume, pickle-format duplicates skipped.
|
||||||
# Verify (default) is the AGENTS.md law in three parts: every listed file
|
# Verify (default) is three independent agreements: every listed file
|
||||||
# present with the listed size, the shard count matches the weight-map
|
# present with the listed size (catches truncation), the shard count
|
||||||
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
|
# matching the weight-map index (catches omissions), and every shard's
|
||||||
# only the public API.
|
# sha256 equal to the Hub's LFS oid (catches substituted content).
|
||||||
|
# Verify needs only the public API.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REPO="Qwen/Qwen3.8-Flash-Next-FP8"
|
REPO="Qwen/Qwen3.8-Flash-Next-FP8"
|
||||||
|
|||||||
@@ -4,12 +4,15 @@
|
|||||||
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
# .hgn, the REQUIRED lookup table and the optional vision tower from
|
||||||
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
|
||||||
# card pins), and the flat tokenizer the two repos share. All public, no
|
# card pins), and the flat tokenizer the two repos share. All public, no
|
||||||
# token — and per the fleet secret law, none would ever touch argv.
|
# token needed — and a secret would never be passed via argv anyway: the
|
||||||
|
# command line is world-readable through /proc, so any token placed there
|
||||||
|
# leaks to every process-listing tool on the host.
|
||||||
#
|
#
|
||||||
# The fetch law, same as every sibling script: curl -f (an access error
|
# The fetch rules, uniform across every sibling script: curl -f so an
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# access error fails loudly instead of writing the error body as a
|
||||||
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an
|
# "successful" tiny artifact; the exact byte count from the Hub tree API;
|
||||||
# existing payload without downloading.
|
# LFS sha256 verification; --verify-only to re-check an existing payload
|
||||||
|
# without downloading.
|
||||||
#
|
#
|
||||||
# File manifest (verified against the Hub tree API on 2026-10-06):
|
# File manifest (verified against the Hub tree API on 2026-10-06):
|
||||||
# qwen38-flash-next-v2-swift15-abliterated.hgn 66,687,678,464 B sha256
|
# qwen38-flash-next-v2-swift15-abliterated.hgn 66,687,678,464 B sha256
|
||||||
@@ -26,7 +29,9 @@
|
|||||||
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
|
||||||
# already on disk (tokenizer.json LFS
|
# already on disk (tokenizer.json LFS
|
||||||
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
|
||||||
# fetched fresh rather than symlinked, per the no-symlink law.
|
# fetched fresh, never symlinked: a snapshot link points into the
|
||||||
|
# host-side cache directory, which does not exist inside the
|
||||||
|
# container mount, so the link dangles exactly where it is consumed.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
|
||||||
|
|||||||
@@ -1,13 +1,15 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
|
||||||
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
|
||||||
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
|
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and a token
|
||||||
# secret law, none would ever touch argv).
|
# would never travel in argv: command lines are world-readable through
|
||||||
|
# /proc, so a secret placed there leaks to any process-listing tool).
|
||||||
#
|
#
|
||||||
# The fetch law, same as every sibling script: curl -f (an access error
|
# The fetch rules, uniform across every sibling script: curl -f so an
|
||||||
# must not become a "successful" 153-byte artifact), exact byte count from
|
# access error fails loudly instead of writing the error body as a
|
||||||
# the model card, LFS sha256 verification, --verify-only to re-check an
|
# "successful" tiny artifact; the exact byte count from the model card;
|
||||||
# existing payload without downloading.
|
# LFS sha256 verification; --verify-only to re-check an existing payload
|
||||||
|
# without downloading.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
|
||||||
|
|||||||
@@ -11,13 +11,13 @@ Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
|
|||||||
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
type-size table (those are easy to get subtly wrong; a wrong per-element size
|
||||||
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
silently under-covers each tensor and dumps tail samples into a junk bucket).
|
||||||
|
|
||||||
Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed
|
What this instrument settles: per-BLOCK expert read volume is FLAT across
|
||||||
streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT
|
all CPU-resident blocks, because MoE top-k routing fires every block on
|
||||||
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
|
every token — there are no "most active layers" at `-ot` granularity. The
|
||||||
routing fires every block every token — there are no "most active layers" at
|
real skew is per-EXPERT: a small fraction of (block,expert) cells carries
|
||||||
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
|
most of the samples, and tensor-level overrides cannot reach that level.
|
||||||
cells carry ~40% of samples), which tensor-level overrides cannot reach.
|
Any fixed number of GPU-resident blocks is therefore equivalent; COUNT is
|
||||||
Any 8 GPU-resident blocks are therefore equivalent; COUNT is the only lever.
|
the only lever.
|
||||||
|
|
||||||
Usage (needs passwordless sudo for perf only; model files must be readable):
|
Usage (needs passwordless sudo for perf only; model files must be readable):
|
||||||
fp-expert-activity.py <pid> [duration_s] # dump + sample + analyze
|
fp-expert-activity.py <pid> [duration_s] # dump + sample + analyze
|
||||||
|
|||||||
+14
-12
@@ -115,15 +115,16 @@ FRANK_ARCHON_TP=2
|
|||||||
FRANK_ARCHON_TP_WIRE=wht6
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
|
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
|
||||||
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
||||||
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
# block below). Slot-count limits observed on other runtimes do NOT
|
||||||
# tcclaviger vLLM fork's torch.compile shape specialization on
|
# transfer: a "this slot count cannot boot" refusal is usually that
|
||||||
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
# runtime's compiler specializing shapes (e.g. a ConstraintViolationError
|
||||||
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
# from shape specialization on query_start_loc), not a property of the
|
||||||
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
# engine now running. This engine boots seqs 2, and seqs 2
|
||||||
# (30.9-33.1 t/s, inside the +-3% noise floor). Side benefit: the fused TP
|
# vs 3 decode measurements land inside each other's noise floor. Side
|
||||||
# all-reduce is declared over seqs x (1 + spec) = 6 rows, under the 25-row
|
# benefit: the fused TP all-reduce is declared over seqs x (1 + spec) = 6
|
||||||
# bound at which the wht6 wire stays EXACT (single-sequence decode and any
|
# rows, under the 25-row bound at which the wht6 wire stays EXACT
|
||||||
# step of <= 25 rows is bit-exact; only busier steps are lossy).
|
# (single-sequence decode and any step of <= 25 rows is bit-exact; only
|
||||||
|
# busier steps are lossy).
|
||||||
FRANK_ARCHON_MAX_SEQS=2
|
FRANK_ARCHON_MAX_SEQS=2
|
||||||
# Served window. Native is 262,144; 200,000 is the publisher's production
|
# Served window. Native is 262,144; 200,000 is the publisher's production
|
||||||
# value (compose) and what models.yml's contextWindow must quote — a longer
|
# value (compose) and what models.yml's contextWindow must quote — a longer
|
||||||
@@ -203,9 +204,10 @@ FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
|||||||
# (104 GiB available with the service loaded).
|
# (104 GiB available with the service loaded).
|
||||||
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
||||||
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
# The PLE n-gram table (47.7 GiB) serves from pinned host RAM the cards
|
||||||
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
|
# read. `ram` REFUSES the start if the memlock limit is too small — a
|
||||||
# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
|
# named refusal beats a mid-load allocator failure — and the unit's
|
||||||
|
# LimitMEMLOCK=infinity is the other half of the arrangement.
|
||||||
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
||||||
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
|
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
|
||||||
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
|
|||||||
@@ -42,10 +42,11 @@
|
|||||||
# form of the engine's default, so the seam test can pin the product.
|
# form of the engine's default, so the seam test can pin the product.
|
||||||
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
||||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
# drafter is NOT slot-locked on this engine. A "slots > 1 disables the
|
||||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
# drafter" restriction is a property of the engine generation that
|
||||||
# quickstart ships 2 slots WITH the head), so the second slot is
|
# imposed it, not of speculation itself — batching and speculation
|
||||||
# concurrent capacity, not a speculation tax.
|
# coexist here, and the card's own quickstart ships 2 slots WITH the
|
||||||
|
# head — so the second slot is concurrent capacity, not a speculation tax.
|
||||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
||||||
# card's own quickstart arithmetic does not bill it):
|
# card's own quickstart arithmetic does not bill it):
|
||||||
@@ -88,9 +89,10 @@
|
|||||||
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
|
||||||
# amdgpu.noretry=0 (check at deploy).
|
# amdgpu.noretry=0 (check at deploy).
|
||||||
|
|
||||||
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
|
# Image: digest-pinned, not tag-pinned — a tag can be re-pointed to a
|
||||||
# is what was qualified). 0.16.2 = the version the model card names as its
|
# build that was never qualified here, while the digest pins exactly the
|
||||||
# quickstart pin; RepoDigest captured at pull on 2026-10-06.
|
# build that was tested. 0.16.2 = the version the model card names as its
|
||||||
|
# quickstart pin; the RepoDigest was captured at a pull of that tag.
|
||||||
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
|
||||||
# Container identity: kept from the 27B era (unit ExecStop names it).
|
# Container identity: kept from the 27B era (unit ExecStop names it).
|
||||||
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
|
||||||
|
|||||||
@@ -16,11 +16,11 @@
|
|||||||
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
|
||||||
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
|
||||||
#
|
#
|
||||||
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
|
# The KV rate follows from the architecture: 8 full layers × 4 KV heads ×
|
||||||
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
|
# 256 dim = 8,192 elements per token per cache. f16 K + f16 V = 32,768
|
||||||
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
|
# B/token = 32 KiB/token, so the 401,408-token pool is 12.25 GiB. The 24
|
||||||
# recurrent state per SLOT (not per token) — that is why two 200k slots fit
|
# DeltaNet layers cost a fixed recurrent state per SLOT (not per token) —
|
||||||
# where the 26B MoE fit only one.
|
# that is why two 200k slots fit where the 26B MoE fit only one.
|
||||||
#
|
#
|
||||||
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
# Feasibility comes from the predecessor's MEASURED boots, not a formula:
|
||||||
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
# MiMo-V2.6-Distill-Qwen-9B (identical qwen35 geometry, 7,793,710,624 B) at
|
||||||
|
|||||||
@@ -12,7 +12,12 @@ Group=dank
|
|||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
|
Environment="FRANK_ZEALOT_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-zealot.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh
|
||||||
Restart=on-failure
|
# Restart=always is the right policy for a serving daemon: a process that
|
||||||
|
# exits on its own — exit 0 — is a CLEAN stop to systemd, so on-failure
|
||||||
|
# never fires and the endpoint stays dark (with the unit reporting
|
||||||
|
# success) until the next boot. frank-zealot is the fleet's compaction
|
||||||
|
# hub — every endpoint's compaction dies when it does.
|
||||||
|
Restart=always
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
|
|||||||
Executable
+133
@@ -0,0 +1,133 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# llm-gates.sh — the artifact gates every LLM endpoint runs before it starts.
|
||||||
|
#
|
||||||
|
# Why this file exists: every LLM endpoint gates its payload twice — the
|
||||||
|
# serve script at start, the deployer before dispatch. Two hand-written
|
||||||
|
# copies of the same checks drift apart in wording and strictness, and the
|
||||||
|
# drift surfaces as deploy-time disagreement: one side passes, the other
|
||||||
|
# refuses, and the failure lands in production instead of in the gate.
|
||||||
|
# Both sides source this library, so the refusal is one text and one rule.
|
||||||
|
#
|
||||||
|
# Contract (public interface):
|
||||||
|
# require_exec PATH LABEL HINT
|
||||||
|
# require_file PATH LABEL HINT
|
||||||
|
# require_bytes PATH LABEL WANT_BYTES HINT
|
||||||
|
# require_min_mib PATH LABEL MIN_MIB HINT
|
||||||
|
# require_dir_files DIR LABEL HINT FILE...
|
||||||
|
# require_image REF HINT
|
||||||
|
#
|
||||||
|
# Each returns 0 and prints nothing on success. On failure it prints two
|
||||||
|
# lines to stderr — the first naming the endpoint ($GATE_PREFIX) and the
|
||||||
|
# artifact, the second the remedy — and returns 1. It NEVER exits: the serve
|
||||||
|
# scripts run under `set -e` (so a refusal is terminal), while the deployer
|
||||||
|
# wants `|| return 1` so a failed gate reports a red deploy for that one
|
||||||
|
# service and the dispatcher's state marker stays pending.
|
||||||
|
# Lines are labelled by ARTIFACT CLASS, and the class is the whole label
|
||||||
|
# (the path rides in the same line): the class + path is what an operator
|
||||||
|
# acts on, so no wording order and no payload rename can make the two twins
|
||||||
|
# disagree. The deployer's old "halogen vision tower" and the serve script's
|
||||||
|
# "vision-tower" were one gate in two vocabularies that no grep could join.
|
||||||
|
|
||||||
|
# Size checks use equality, not a floor, for `require_bytes`: a fetch that
|
||||||
|
# truncates looks like a success (an HTTP error exit is often swallowed, and
|
||||||
|
# the error body is written in the artifact's place), so "exists and is big
|
||||||
|
# enough" is not proof of the right artifact. The declared byte count is a
|
||||||
|
# contract in both directions — a bigger file is a DIFFERENT artifact.
|
||||||
|
|
||||||
|
# GATE_PREFIX names the endpoint in every refusal (e.g. frank-dragoon). The
|
||||||
|
# caller exports it; shellcheck cannot follow it across the source boundary,
|
||||||
|
# so the read site carries the directive.
|
||||||
|
# shellcheck disable=SC2034
|
||||||
|
: "${GATE_PREFIX:=gate}"
|
||||||
|
|
||||||
|
# Two bash behaviors make the `return 1` at every call site load-bearing:
|
||||||
|
# * a function whose body ends inside an `if` returns the CONDITION's
|
||||||
|
# status (true = 0), so a gate that ends on its own refusal prints the
|
||||||
|
# refusal and reports SUCCESS;
|
||||||
|
# * `set -e` is suspended for any command in a `… || …` chain, so a gate
|
||||||
|
# that runs inside one must not be trusted to abort the caller.
|
||||||
|
# Every gate's refusal therefore carries an explicit `|| return 1`, and the
|
||||||
|
# caller consumes the status (`|| exit 1`, `|| return 1`).
|
||||||
|
_gate_fail() {
|
||||||
|
printf '%s: %s\n' "$GATE_PREFIX" "$1" >&2
|
||||||
|
printf ' hint: %s\n' "$2" >&2
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_file — PATH must be a regular file. A directory (an unpacked
|
||||||
|
# staging dir wearing the payload's name) and a symlink (a HuggingFace cache
|
||||||
|
# snapshot's entry, which dangles inside the container mount) both refuse.
|
||||||
|
require_file() {
|
||||||
|
local path="$1" label="$2" hint="$3"
|
||||||
|
if [[ -L "$path" || ! -f "$path" ]]; then
|
||||||
|
_gate_fail "$label: missing at $path" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_exec — PATH must be an executable regular file. The native runtimes
|
||||||
|
# (llama.cpp's llama-server) are binaries the operator builds; a tree that
|
||||||
|
# compiled but lost its +x, or a directory wearing the binary's name, must
|
||||||
|
# refuse the same way a missing one does.
|
||||||
|
require_exec() {
|
||||||
|
local path="$1" label="$2" hint="$3"
|
||||||
|
if [[ -L "$path" || ! -x "$path" ]]; then
|
||||||
|
_gate_fail "$label: missing at $path" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_bytes — PATH must exist AND be exactly WANT_BYTES bytes.
|
||||||
|
require_bytes() {
|
||||||
|
local path="$1" label="$2" want="$3" hint="$4" got
|
||||||
|
require_file "$path" "$label" "$hint" || return 1
|
||||||
|
got="$(stat -c %s "$path")"
|
||||||
|
if [[ "$got" != "$want" ]]; then
|
||||||
|
_gate_fail "$label: is $got bytes, expected $want" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_min_mib — PATH must be at least MIN_MIB mebibytes. The floor form
|
||||||
|
# is for build products (a .rad container is a range of legal sizes; a
|
||||||
|
# truncated convert is not).
|
||||||
|
require_min_mib() {
|
||||||
|
local path="$1" label="$2" min="$3" hint="$4" mib
|
||||||
|
require_file "$path" "$label" "$hint" || return 1
|
||||||
|
mib=$(( $(stat -c %s "$path") / 1048576 ))
|
||||||
|
if (( mib < min )); then
|
||||||
|
_gate_fail "$label: is $mib MiB, expected >= $min MiB" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_dir_files — DIR must be a directory and every named member a
|
||||||
|
# regular file, never a symlink: snapshot layouts link files into place, and
|
||||||
|
# a link dangles inside the container mount even when the directory itself
|
||||||
|
# is real. A symlinked DIRECTORY is refused too — the same layout can link
|
||||||
|
# the directory itself, and the link dangles identically.
|
||||||
|
require_dir_files() {
|
||||||
|
local dir="$1" label="$2" hint="$3"
|
||||||
|
shift 3
|
||||||
|
local f
|
||||||
|
if [[ -L "$dir" || ! -d "$dir" ]]; then
|
||||||
|
_gate_fail "$label: missing at $dir" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
for f in "$@"; do
|
||||||
|
if [[ -L "$dir/$f" || ! -f "$dir/$f" ]]; then
|
||||||
|
_gate_fail "$label: $dir/$f is missing or a symlink" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# require_image — the image ref must be in podman storage. A name@sha256 ref
|
||||||
|
# exists only if THAT manifest was pulled, so the gate is the ref itself, in
|
||||||
|
# one command: no pipefail/grep race (the 2026-09-30 lesson).
|
||||||
|
require_image() {
|
||||||
|
local ref="$1" hint="$2"
|
||||||
|
if ! podman image exists "$ref"; then
|
||||||
|
_gate_fail "image: $ref is not in podman storage" "$hint" || return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -29,10 +29,11 @@
|
|||||||
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
|
||||||
# getattr_static()s each one off the instance, breaking on the first miss.
|
# getattr_static()s each one off the instance, breaking on the first miss.
|
||||||
# So BOTH wrappers need all three attributes.
|
# So BOTH wrappers need all three attributes.
|
||||||
# * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with
|
# * Why not inheritance: adding SupportsEagle3 as a BASE class dies at
|
||||||
# "Cannot create a consistent method resolution order (MRO) for bases ..."
|
# class creation with "Cannot create a consistent method resolution
|
||||||
# (GLM's wrappers already carry four/five Protocol-derived bases). The
|
# order (MRO) for bases ..." (GLM's wrappers already carry four/five
|
||||||
# attribute route avoids the MRO entirely. Also proven by simulation:
|
# Protocol-derived bases; the base ordering cannot take another). The
|
||||||
|
# attribute route avoids the MRO entirely; simulation confirms that
|
||||||
# the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
|
# the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
|
||||||
# position.
|
# position.
|
||||||
# * Method placement: without inheriting, the Protocol's default
|
# * Method placement: without inheriting, the Protocol's default
|
||||||
|
|||||||
@@ -9,7 +9,9 @@ No scratch container, no mask: the service under test is the service.
|
|||||||
Three probes, in increasing cost:
|
Three probes, in increasing cost:
|
||||||
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
||||||
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
||||||
The bench law applies: >=128 generated tokens, greedy, real text.
|
Method: >=128 generated tokens (below that, first-token latency
|
||||||
|
dominates the decode rate), greedy (sampling noise is unattributable),
|
||||||
|
real text (synthetic text inflates draft acceptance).
|
||||||
B the capacity contract — TWO concurrent chats of ~197k prompt tokens
|
B the capacity contract — TWO concurrent chats of ~197k prompt tokens
|
||||||
(~394k tokens resident) each carrying a unique marker at its head. Both
|
(~394k tokens resident) each carrying a unique marker at its head. Both
|
||||||
must answer their own marker. This is what a 2 x 200,000 endpoint owes:
|
must answer their own marker. This is what a 2 x 200,000 endpoint owes:
|
||||||
@@ -17,11 +19,11 @@ Three probes, in increasing cost:
|
|||||||
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
|
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
|
||||||
to give the blocks back (cache_n > 0, TTFT collapses).
|
to give the blocks back (cache_n > 0, TTFT collapses).
|
||||||
|
|
||||||
PROBE LAW (learned the hard way, twice): radiance routes the thinking block to
|
PROBE PITFALL: radiance routes the thinking block to
|
||||||
`reasoning_content`, so a chat request whose budget is spent inside
|
`reasoning_content`, so a chat request whose budget is spent inside
|
||||||
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
|
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
|
||||||
reads `content` alone scores 0/2 on a healthy server. Every request here sets
|
reads `content` alone scores 0/2 on a healthy server. Every request here
|
||||||
chat_template_kwargs.enable_thinking=false and reads both fields.
|
sets chat_template_kwargs.enable_thinking=false and reads both fields.
|
||||||
|
|
||||||
A/B method for a shape change (no scratch container): write a variant env file
|
A/B method for a shape change (no scratch container): write a variant env file
|
||||||
on the host, point a transient systemd drop-in at it
|
on the host, point a transient systemd drop-in at it
|
||||||
|
|||||||
@@ -1,10 +1,12 @@
|
|||||||
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
||||||
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
||||||
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
||||||
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
# through GPTQ into the w4nl64a8h table. The converter declares ONE
|
||||||
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
# encoding per expert table (qwen4exp_fp8 rad_arch_declare): the served
|
||||||
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
# kernel covers the whole expert set, and the only legal mixed layout is a
|
||||||
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
# WHOLE expert kept plain bf16 beside it. That rules out the original
|
||||||
|
# split tier (down block-fp8 + gate_up w4), which asks one table for two
|
||||||
|
# encodings.
|
||||||
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
||||||
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
||||||
#
|
#
|
||||||
@@ -25,11 +27,14 @@
|
|||||||
|
|
||||||
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
||||||
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
||||||
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
# The Swift 1.5 finetune's routing distribution is not assumed equal: the
|
||||||
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
# derivation step rewrites this section from Swift's OWN calibration
|
||||||
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
# recording before this recipe is ever run. The selection rule is the base
|
||||||
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
# checkpoint's: an expert whose share of its layer's down-input energy is
|
||||||
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
# at least 3% AND at least three times its share of the routing; an even
|
||||||
|
# count on each rank's half of the layer (the four-bit GEMM takes the rest
|
||||||
|
# as two equal tables by parity), padded with that half's next most
|
||||||
|
# energetic expert.
|
||||||
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
||||||
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
||||||
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
||||||
|
|||||||
@@ -26,6 +26,14 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
|
# The artifact gates are the shared library: the serve script and the
|
||||||
|
# deployer gate the same payload, and two hand-written copies of the same
|
||||||
|
# checks drift apart until one side passes what the other refuses. One
|
||||||
|
# library, one refusal text, one rule.
|
||||||
|
export GATE_PREFIX=frank-archon
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
|
ENV_FILE="${FRANK_ARCHON_ENV_FILE:-$SCRIPT_DIR/frank-archon.env}"
|
||||||
|
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
@@ -65,38 +73,22 @@ set +a
|
|||||||
|
|
||||||
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
||||||
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
# in storage only if THAT manifest was pulled — a retagged look-alike or a
|
||||||
# newer latest does not satisfy it. `podman image exists` takes a
|
# newer latest does not satisfy it. One command, no pipefail/grep race (the
|
||||||
# name@digest ref in one command — no pipefail/grep race (the 2026-09-30
|
# 2026-09-30 lesson). Runs as dank: the unit's process uses dank's ROOTLESS
|
||||||
# lesson). Runs as dank: the unit's process uses dank's ROOTLESS podman
|
# podman storage (root unit, User=dank — the fleet's frank-archon shape).
|
||||||
# storage (root unit, User=dank — the fleet's frank-archon shape).
|
require_image "$FRANK_ARCHON_IMAGE" \
|
||||||
if ! podman image exists "$FRANK_ARCHON_IMAGE"; then
|
"podman pull docker://stilldeadcode/radiance:latest (as dank)" || exit 1
|
||||||
echo "ERROR: radiance image absent: $FRANK_ARCHON_IMAGE" >&2
|
|
||||||
echo "frank-archon: podman pull docker://stilldeadcode/radiance:latest (as dank)" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# The payload gate: the .rad container file, sized. A truncated convert
|
# The payload gate: the .rad container file, sized. A truncated convert
|
||||||
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
# would otherwise buy a guaranteed long-timeout boot instead of this fast
|
||||||
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
# failure. The staging dir / raw checkpoint are NOT this script's concern
|
||||||
# (scripts/build-frank-archon-rad.sh built the file; its rad-info output
|
# (the convert script built the file; its rad-info output is the record).
|
||||||
# is the record).
|
require_min_mib "$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE" \
|
||||||
RAD="$FRANK_ARCHON_MODEL_DIR/$FRANK_ARCHON_MODEL_FILE"
|
container "$FRANK_ARCHON_RAD_MIN_MIB" \
|
||||||
if [ ! -f "$RAD" ]; then
|
"build it: scripts/convert-swift15-w4dn8.sh (rad-info is the record)" || exit 1
|
||||||
echo "ERROR: .rad container missing: $RAD" >&2
|
|
||||||
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
RAD_MIB=$(( $(stat -c %s "$RAD") / 1048576 ))
|
|
||||||
if [ "$RAD_MIB" -lt "$FRANK_ARCHON_RAD_MIN_MIB" ]; then
|
|
||||||
echo "ERROR: .rad container too small: $RAD is ${RAD_MIB} MiB, want >= $FRANK_ARCHON_RAD_MIN_MIB" >&2
|
|
||||||
echo "frank-archon: the convert was truncated — rebuild it" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
|
require_file "$FRANK_ARCHON_TEMPLATE" template \
|
||||||
echo "ERROR: chat template missing: $FRANK_ARCHON_TEMPLATE" >&2
|
"the chat template is vendored: scripts/qwen38-froggeric-v22.5.jinja" || exit 1
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
|
mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
|
||||||
# The engine's create is NOT recursive: with only the state dir mounted, its
|
# The engine's create is NOT recursive: with only the state dir mounted, its
|
||||||
@@ -112,8 +104,12 @@ mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
|
|||||||
# that wrote it. The dir is pre-created for the same reason as kvcache
|
# that wrote it. The dir is pre-created for the same reason as kvcache
|
||||||
# (the engine's create is not recursive).
|
# (the engine's create is not recursive).
|
||||||
|
|
||||||
if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
# GPU access is mandatory; the node paths are knobs so the seam can drive
|
||||||
echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
|
# this gate on a box without AMD GPUs (production defaults).
|
||||||
|
DEV_KFD="${FRANK_ARCHON_DEV_KFD:-/dev/kfd}"
|
||||||
|
DEV_DRI="${FRANK_ARCHON_DEV_DRI:-/dev/dri}"
|
||||||
|
if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
||||||
|
echo "ERROR: GPU device nodes absent ($DEV_KFD, $DEV_DRI)" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -129,8 +125,9 @@ ttm_cap_mib() {
|
|||||||
# min over the knobs THIS kernel exposes — the current kernel has
|
# min over the knobs THIS kernel exposes — the current kernel has
|
||||||
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
||||||
# not a zero (a zero would refuse a healthy cap).
|
# not a zero (a zero would refuse a healthy cap).
|
||||||
local v cur=""
|
local v cur="" dir
|
||||||
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
|
dir="${FRANK_ARCHON_TTM_SYSFS:-/sys/module/ttm/parameters}"
|
||||||
|
for p in "$dir/pages_limit" "$dir/pages_max"; do
|
||||||
[ -r "$p" ] || continue
|
[ -r "$p" ] || continue
|
||||||
v="$(cat "$p" 2>/dev/null)" || continue
|
v="$(cat "$p" 2>/dev/null)" || continue
|
||||||
[ -n "$v" ] || continue
|
[ -n "$v" ] || continue
|
||||||
@@ -155,7 +152,7 @@ for CTR in $FRANK_ARCHON_LEGACY_CONTAINERS "$FRANK_ARCHON_CONTAINER"; do
|
|||||||
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
podman rm -f "$CTR" >/dev/null 2>&1 || true
|
||||||
done
|
done
|
||||||
|
|
||||||
LOG=/home/dank/logs/frank_archon.log
|
LOG="${FRANK_ARCHON_LOG:-/home/dank/logs/frank_archon.log}"
|
||||||
mkdir -p "$(dirname "$LOG")"
|
mkdir -p "$(dirname "$LOG")"
|
||||||
|
|
||||||
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
# Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
|
||||||
|
|||||||
@@ -29,7 +29,14 @@
|
|||||||
# exactly like frank-archon's.
|
# exactly like frank-archon's.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
# The artifact gates are the shared library — the serve script and the
|
||||||
|
# deployer refuse the same missing/truncated payload with the same words
|
||||||
|
# (the 2026-10-01 crash-loop was the two hand-written twins disagreeing).
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
export GATE_PREFIX=frank-dragoon
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
|
|
||||||
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
ENV_FILE="${FRANK_DRAGOON_ENV_FILE:-$SCRIPT_DIR/frank-dragoon.env}"
|
||||||
if [ ! -f "$ENV_FILE" ]; then
|
if [ ! -f "$ENV_FILE" ]; then
|
||||||
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
|
echo "frank-dragoon: env file not found: $ENV_FILE" >&2
|
||||||
@@ -57,55 +64,38 @@ set +a
|
|||||||
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
|
: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set}"
|
||||||
: "${HALOGEN_SPEC_ADAPT:?HALOGEN_SPEC_ADAPT must be set (0 = head always on)}"
|
: "${HALOGEN_SPEC_ADAPT:?HALOGEN_SPEC_ADAPT must be set (0 = head always on)}"
|
||||||
|
|
||||||
# Capture podman's output BEFORE grepping: under `set -o pipefail` the
|
# The image gate is the digest-pinned ref itself: `podman image exists` takes
|
||||||
# `podman images | grep -q` pattern races (grep exits on first match,
|
# a name@sha256 ref in one command, so a retagged look-alike or a newer
|
||||||
# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
|
# latest cannot satisfy it and there is no pipefail/SIGPIPE race with grep
|
||||||
# 2026-09-30 deploy refusal).
|
# (the 2026-09-30 deploy refusal). Runs as dank: the unit's process uses
|
||||||
DIGESTS="$(podman images --format '{{.RepoDigests}}')"
|
# dank's rootless storage.
|
||||||
if ! grep -qF "${FRANK_DRAGOON_IMAGE#*@}" <<<"$DIGESTS"; then
|
require_image "$FRANK_DRAGOON_IMAGE" \
|
||||||
echo "frank-dragoon: image $FRANK_DRAGOON_IMAGE not in podman storage;" >&2
|
"pull it: podman pull docker://$FRANK_DRAGOON_IMAGE" || exit 1
|
||||||
echo " pull it: podman pull ${FRANK_DRAGOON_IMAGE%@*}" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Size-exact gates on all three payload files: a truncated curl exits 0 on
|
# Size-exact gates on all three payload files: a truncated download can
|
||||||
# some failures and a partial .hgn loads far enough to look alive, so the
|
# exit 0, and a partial .hgn loads far enough to look alive, so existence
|
||||||
# byte counts from the Hub tree API are the contract (fleet fetch law).
|
# proves nothing. The byte count published by the Hub tree API is the
|
||||||
ck() {
|
# contract: any other size is a different (broken) artifact.
|
||||||
# ck <env-name> <path> <expected-bytes>
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" \
|
||||||
if [ ! -f "$2" ]; then
|
checkpoint "$FRANK_DRAGOON_CHECKPOINT_BYTES" \
|
||||||
echo "frank-dragoon: $1 $2 missing;" >&2
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" \
|
||||||
exit 1
|
n-gram "$FRANK_DRAGOON_NGRAM_BYTES" \
|
||||||
fi
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
local sz
|
require_bytes "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" \
|
||||||
sz="$(stat -c %s "$2")"
|
vision "$FRANK_DRAGOON_VISION_BYTES" \
|
||||||
if [ "$sz" != "$3" ]; then
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" || exit 1
|
||||||
echo "frank-dragoon: $1 is $sz bytes, expected $3 (partial download?)" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
ck checkpoint "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_CHECKPOINT" "$FRANK_DRAGOON_CHECKPOINT_BYTES"
|
|
||||||
ck n-gram-table "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_NGRAM_TABLE" "$FRANK_DRAGOON_NGRAM_BYTES"
|
|
||||||
ck vision-tower "$FRANK_DRAGOON_MODEL_DIR/$FRANK_DRAGOON_VISION_TOWER" "$FRANK_DRAGOON_VISION_BYTES"
|
|
||||||
|
|
||||||
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
# The payload ships the tokenizer as a FLAT DIRECTORY. A HuggingFace cache
|
||||||
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
# snapshot symlinks each file into a sibling blobs/, and those symlinks
|
||||||
# dangle inside the container mount — so the directory must be real and
|
# dangle inside the container mount — so the directory must be real and
|
||||||
# every named file must be a regular file, not a link.
|
# every named file must be a regular file, not a link.
|
||||||
TOK_DIR="$FRANK_DRAGOON_MODEL_DIR/tokenizer"
|
# The tokenizer file list is word-separated ON PURPOSE (one argument per
|
||||||
if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
|
# file), so the unquoted expansion below is the interface, not a slip.
|
||||||
echo "frank-dragoon: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
|
# shellcheck disable=SC2086
|
||||||
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
require_dir_files "$FRANK_DRAGOON_MODEL_DIR/tokenizer" tokenizer \
|
||||||
exit 1
|
"fetch it: bash scripts/download-frank-dragoon-flashnext.sh" \
|
||||||
fi
|
$FRANK_DRAGOON_TOKENIZER_FILES || exit 1
|
||||||
for TF in $FRANK_DRAGOON_TOKENIZER_FILES; do
|
|
||||||
if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
|
|
||||||
echo "frank-dragoon: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
|
|
||||||
echo " fetch it: bash scripts/download-frank-dragoon-flashnext.sh" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
# GPU access is mandatory but the node paths are variables so the seam can
|
# GPU access is mandatory but the node paths are variables so the seam can
|
||||||
# drive the gate on a box without an AMD GPU.
|
# drive the gate on a box without an AMD GPU.
|
||||||
@@ -115,6 +105,7 @@ if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
|
|||||||
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
echo "frank-dragoon: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
# Host-RAM preflight — the engine's pin gate, checked BEFORE podman run.
|
||||||
# The engine refuses to pin when MemAvailable is below the checkpoint plus
|
# The engine refuses to pin when MemAvailable is below the checkpoint plus
|
||||||
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
|
# a 16 GiB floor: pinned pages cannot be swapped, and under the floor the
|
||||||
|
|||||||
@@ -28,6 +28,14 @@
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
|
# The artifact gates are the shared library (scripts/lib/llm-gates.sh): the
|
||||||
|
# serve script and the deployer gate the same payload, and two hand-written
|
||||||
|
# copies of the same checks drift apart until one side passes what the
|
||||||
|
# other refuses.
|
||||||
|
export GATE_PREFIX=frank-zealot
|
||||||
|
# shellcheck disable=SC1091
|
||||||
|
source "$SCRIPT_DIR/lib/llm-gates.sh"
|
||||||
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
|
ENV_FILE="${FRANK_ZEALOT_ENV_FILE:-$SCRIPT_DIR/frank-zealot.env}"
|
||||||
if [[ -f "$ENV_FILE" ]]; then
|
if [[ -f "$ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
@@ -62,9 +70,9 @@ fi
|
|||||||
# FP8 class; no true fp8 KV dtype exists in this build).
|
# FP8 class; no true fp8 KV dtype exists in this build).
|
||||||
: "${CACHE_TYPE_K:=f16}"
|
: "${CACHE_TYPE_K:=f16}"
|
||||||
: "${CACHE_TYPE_V:=f16}"
|
: "${CACHE_TYPE_V:=f16}"
|
||||||
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
|
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors,
|
||||||
# there is no in-file head (the law: MTP is a property of the file, not of the
|
# so there is no in-file head — a speculative head is a property of the
|
||||||
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set);
|
# model file, not of the flags; no flag creates tensors that are absent.
|
||||||
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
|
||||||
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
|
||||||
: "${SPEC_TYPE:=none}"
|
: "${SPEC_TYPE:=none}"
|
||||||
@@ -81,13 +89,18 @@ fi
|
|||||||
|
|
||||||
# Fail loudly rather than half-loading: a missing file is a download that
|
# Fail loudly rather than half-loading: a missing file is a download that
|
||||||
# has not finished, and Restart=always would turn it into a crash loop that
|
# has not finished, and Restart=always would turn it into a crash loop that
|
||||||
# reads like a runtime fault.
|
# reads like a runtime fault. The refusal text is the shared gate library's,
|
||||||
for path in "$LLAMA_BIN" "$MODEL_FILE"; do
|
# so the deployer's twin refuses identically.
|
||||||
[[ -e "$path" ]] || { echo "frank-zealot: missing $path" >&2; exit 1; }
|
require_file "$LLAMA_BIN" runtime \
|
||||||
done
|
"build it: scripts/build-llama-master-cuda.sh" || exit 1
|
||||||
|
require_file "$MODEL_FILE" gguf \
|
||||||
|
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
|
||||||
|
if [[ -n "$MMPROJ_FILE" ]]; then
|
||||||
|
require_file "$MMPROJ_FILE" mmproj \
|
||||||
|
"fetch it: bash scripts/download-frank-zealot.sh" || exit 1
|
||||||
|
fi
|
||||||
EXTRA_ARGS=()
|
EXTRA_ARGS=()
|
||||||
if [[ -n "$MMPROJ_FILE" ]]; then
|
if [[ -n "$MMPROJ_FILE" ]]; then
|
||||||
[[ -e "$MMPROJ_FILE" ]] || { echo "frank-zealot: missing $MMPROJ_FILE" >&2; exit 1; }
|
|
||||||
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|||||||
Executable
+750
@@ -0,0 +1,750 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""service-map.py — the only reader of the service inventory.
|
||||||
|
|
||||||
|
Four lists used to agree by discipline (the planner's path rules, the
|
||||||
|
deployer's allow-list, the changed-test mapper, and the scrape/probe/DNS
|
||||||
|
facts written by hand in three config files). They now all read
|
||||||
|
`shared/services.yml` through this tool:
|
||||||
|
|
||||||
|
hosts the deploy targets
|
||||||
|
services [--host H] deployables a target accepts, canonical order
|
||||||
|
allows --host H --service S exit 0 when H accepts S (silent)
|
||||||
|
plan --host H PATH... services selected by changed paths, order-safe
|
||||||
|
seams [--host H] PATH... the Bats seams a changed path must re-run
|
||||||
|
owner --host H PATH the service that owns a file (empty: none)
|
||||||
|
units --host H --service S "<unit> <unit_dir>" lines the handler installs
|
||||||
|
stack --host H --service S the deployable + members one handler deploys
|
||||||
|
get --service S [--host H] F one field as JSON
|
||||||
|
render pihole-hosts the Pi-hole addn-hosts file
|
||||||
|
check [NAME...] integrity + drift gates (exit 1 on drift)
|
||||||
|
|
||||||
|
Two rules the engine enforces structurally:
|
||||||
|
|
||||||
|
* A path selects only services that RUN on the planned target. Path
|
||||||
|
matching that is host-blind (match first, filter later) lets a file on
|
||||||
|
one host plan a service on another; selection is scoped to the host the
|
||||||
|
path belongs to.
|
||||||
|
* The netns family is DERIVED, never declared: a unit file that says
|
||||||
|
`Network=gluetun.container` rides that tunnel, so a gluetun change selects
|
||||||
|
every sharer. The unit files are the source of that coupling.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from functools import lru_cache
|
||||||
|
from pathlib import Path, PurePosixPath
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import yaml
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
DEFAULT_INVENTORY = REPO_ROOT / "shared" / "services.yml"
|
||||||
|
|
||||||
|
REQUIRED_KEYS = ("name", "hosts", "kind", "tests")
|
||||||
|
KINDS = {"quadlet", "root-unit", "vm-quadlet", "host"}
|
||||||
|
UNIT_SUFFIXES = (".container", ".network")
|
||||||
|
|
||||||
|
# Repo material that is never a deployed runtime's state.
|
||||||
|
NON_RUNTIME_ROOTS = {"docs", "report", "old", ".githooks", ".obsidian", ".github",
|
||||||
|
"tests", "serverAppConfigs-docs"}
|
||||||
|
NON_RUNTIME_NAMES = {".env.example"}
|
||||||
|
NON_RUNTIME_PATHS = {"README.md", "AGENTS.md", ".gitignore"}
|
||||||
|
|
||||||
|
|
||||||
|
def is_trash(path: str) -> bool:
|
||||||
|
return ".AGENTTRASH" in PurePosixPath(path).parts
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# inventory
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
class Inventory:
|
||||||
|
def __init__(self, doc: dict[str, Any], origin: str) -> None:
|
||||||
|
self.origin = origin
|
||||||
|
self.hosts: dict[str, dict[str, Any]] = doc["hosts"]
|
||||||
|
self.services: list[dict[str, Any]] = doc["services"]
|
||||||
|
self.unowned: dict[str, str] = doc.get("unowned", {})
|
||||||
|
for index, svc in enumerate(self.services):
|
||||||
|
svc["_index"] = index
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def load(cls, path: str | Path = DEFAULT_INVENTORY) -> "Inventory":
|
||||||
|
doc = yaml.safe_load(Path(path).read_text())
|
||||||
|
if not isinstance(doc, dict) or "services" not in doc:
|
||||||
|
raise SystemExit(f"{path}: not a service inventory document")
|
||||||
|
return cls(doc, str(path))
|
||||||
|
|
||||||
|
def on(self, host: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if host in s["hosts"]]
|
||||||
|
|
||||||
|
def named(self, host: str, name: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if s["name"] == name and host in s["hosts"]]
|
||||||
|
|
||||||
|
def any_named(self, name: str) -> list[dict[str, Any]]:
|
||||||
|
return [s for s in self.services if s["name"] == name]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def order(self) -> list[str]:
|
||||||
|
seen: list[str] = []
|
||||||
|
for svc in self.services:
|
||||||
|
if svc["name"] not in seen:
|
||||||
|
seen.append(svc["name"])
|
||||||
|
return seen
|
||||||
|
|
||||||
|
def edge_ip(self) -> str:
|
||||||
|
"""The single address every DNS name resolves at (Caddy's edge)."""
|
||||||
|
for host in self.hosts.values():
|
||||||
|
if "dns_edge" in host:
|
||||||
|
return str(host["dns_edge"])
|
||||||
|
raise SystemExit(f"{self.origin}: no host declares dns_edge")
|
||||||
|
|
||||||
|
def service(self, name: str) -> dict | None:
|
||||||
|
for svc in self.services:
|
||||||
|
if svc["name"] == name:
|
||||||
|
return svc
|
||||||
|
return None
|
||||||
|
|
||||||
|
def unit_dirs(self, svc: dict[str, Any], host: str | None = None) -> list[str]:
|
||||||
|
"""The repo dirs holding this service's unit files, for one host."""
|
||||||
|
dirs = svc.get("unit_dir")
|
||||||
|
if not dirs:
|
||||||
|
return []
|
||||||
|
if isinstance(dirs, dict):
|
||||||
|
if host is not None:
|
||||||
|
return [dirs[host]] if host in dirs else []
|
||||||
|
return [dirs[h] for h in svc["hosts"] if h in dirs]
|
||||||
|
return [dirs]
|
||||||
|
|
||||||
|
def owns_unit(self, svc: dict[str, Any], host: str, unit: str, directory: str) -> bool:
|
||||||
|
return unit in svc.get("units", []) and directory in self.unit_dirs(svc, host)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# repo facts
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
@lru_cache(maxsize=1)
|
||||||
|
def tracked_files() -> tuple[str, ...]:
|
||||||
|
# Tracked + untracked (a check that only saw HEAD would ignore the very
|
||||||
|
# files a work-in-progress commit is adding).
|
||||||
|
out = subprocess.run(
|
||||||
|
["git", "-C", str(REPO_ROOT), "ls-files", "--cached", "--others",
|
||||||
|
"--exclude-standard"],
|
||||||
|
text=True, stdout=subprocess.PIPE, check=True)
|
||||||
|
return tuple(line for line in out.stdout.splitlines() if line)
|
||||||
|
|
||||||
|
|
||||||
|
def _network_values(container: Path) -> list[str]:
|
||||||
|
"""Every `Network=` value a quadlet unit declares (comma lists included)."""
|
||||||
|
values: list[str] = []
|
||||||
|
try:
|
||||||
|
text = container.read_text()
|
||||||
|
except OSError:
|
||||||
|
return values
|
||||||
|
for line in text.splitlines():
|
||||||
|
if line.startswith("Network="):
|
||||||
|
values.extend(v.strip() for v in line[len("Network="):].split(",") if v.strip())
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
class Graph:
|
||||||
|
"""The unit-file coupling, read out of the tree: netns + netns-net users.
|
||||||
|
|
||||||
|
Quadlet installs every user unit flat, so `Network=gluetun.container`
|
||||||
|
names a unit by name, not by path; the same host is the scope, which is
|
||||||
|
also the deployment scope.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, hosts: list[str]) -> None:
|
||||||
|
self.units: dict[tuple[str, str], str] = {} # (host, unit) -> path
|
||||||
|
self.networks: dict[tuple[str, str], str] = {} # (host, net) -> path
|
||||||
|
self.netns_children: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||||
|
self.network_users: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||||
|
for path in tracked_files():
|
||||||
|
parts = PurePosixPath(path).parts
|
||||||
|
if not parts or parts[0] not in hosts or is_trash(path):
|
||||||
|
continue
|
||||||
|
host, directory, name = parts[0], str(PurePosixPath(*parts[1:-1])), parts[-1]
|
||||||
|
if name.endswith(".container"):
|
||||||
|
self.units[(host, name[: -len(".container")])] = path
|
||||||
|
elif name.endswith(".network"):
|
||||||
|
self.networks[(host, name[: -len(".network")])] = path
|
||||||
|
for (host, unit), path in self.units.items():
|
||||||
|
for value in _network_values(REPO_ROOT / path):
|
||||||
|
if value.endswith(".container"):
|
||||||
|
parent = (host, value[: -len(".container")])
|
||||||
|
self.netns_children.setdefault(parent, set()).add((host, unit))
|
||||||
|
elif value.endswith(".network"):
|
||||||
|
net = (host, value[: -len(".network")])
|
||||||
|
self.network_users.setdefault(net, set()).add((host, unit))
|
||||||
|
|
||||||
|
def rides(self, start: list[tuple[str, str]]) -> set[tuple[str, str]]:
|
||||||
|
"""The transitive netns closure: a unit, everything riding it, and
|
||||||
|
everything riding those (a netns inside a netns)."""
|
||||||
|
found: set[tuple[str, str]] = set()
|
||||||
|
frontier = list(start)
|
||||||
|
while frontier:
|
||||||
|
node = frontier.pop()
|
||||||
|
if node in found:
|
||||||
|
continue
|
||||||
|
found.add(node)
|
||||||
|
frontier.extend(self.netns_children.get(node, ()))
|
||||||
|
return found
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=8)
|
||||||
|
def graph_for(hosts: tuple[str, ...]) -> Graph:
|
||||||
|
return Graph(list(hosts))
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# selection
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def _owner_index(inv: Inventory) -> dict[tuple[str, str, int]]:
|
||||||
|
"""(host, unit) -> service index, for unit-file ownership.
|
||||||
|
|
||||||
|
A quadlet unit name is unique per host (``check units`` enforces one
|
||||||
|
claim), so the coupling closure can resolve a riding unit that lives in
|
||||||
|
a DIFFERENT directory than the changed file — gluetun's netns children
|
||||||
|
are the point.
|
||||||
|
"""
|
||||||
|
index: dict[tuple[str, str], int] = {}
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
key = (host, unit)
|
||||||
|
if key in index and index[key] != svc["_index"]:
|
||||||
|
raise SystemExit(
|
||||||
|
f"{inv.origin}: unit {unit!r} on {host} claimed by "
|
||||||
|
f"{inv.services[index[key]]['name']} and {svc['name']}"
|
||||||
|
)
|
||||||
|
index[key] = svc["_index"]
|
||||||
|
return index
|
||||||
|
|
||||||
|
|
||||||
|
def select(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||||
|
"""Canonical-order service names the changed paths select for one host."""
|
||||||
|
if host not in inv.hosts:
|
||||||
|
raise SystemExit(f"{inv.origin}: no host {host!r}")
|
||||||
|
graph = graph_for(tuple(inv.hosts))
|
||||||
|
owners = _owner_index(inv)
|
||||||
|
hits: set[int] = set()
|
||||||
|
|
||||||
|
def add(index: int) -> None:
|
||||||
|
hits.add(index)
|
||||||
|
|
||||||
|
for raw in paths:
|
||||||
|
path = raw.replace("\\", "/")
|
||||||
|
parts = PurePosixPath(path).parts
|
||||||
|
if not parts or is_trash(path) or path in NON_RUNTIME_PATHS:
|
||||||
|
continue
|
||||||
|
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||||
|
continue
|
||||||
|
head = parts[0]
|
||||||
|
if head in NON_RUNTIME_ROOTS:
|
||||||
|
continue
|
||||||
|
# A host's directory is the scope: a frankbook-pro file can never
|
||||||
|
# select a dankserver service.
|
||||||
|
scoped_host = head if head in inv.hosts else None
|
||||||
|
if scoped_host is not None and scoped_host != host:
|
||||||
|
continue
|
||||||
|
|
||||||
|
name = parts[-1]
|
||||||
|
directory = str(PurePosixPath(*parts[1:-1])) if len(parts) > 1 else ""
|
||||||
|
|
||||||
|
if scoped_host and name.endswith(UNIT_SUFFIXES):
|
||||||
|
stem = name.rsplit(".", 1)[0]
|
||||||
|
key = (scoped_host, stem)
|
||||||
|
if name.endswith(".network") and key in graph.networks:
|
||||||
|
users = graph.network_users.get(key, set())
|
||||||
|
for unit in graph.rides(sorted(users)):
|
||||||
|
index = owners.get(unit)
|
||||||
|
if index is not None:
|
||||||
|
add(index)
|
||||||
|
elif key in graph.units:
|
||||||
|
for unit in graph.rides([key]):
|
||||||
|
index = owners.get(unit)
|
||||||
|
if index is not None:
|
||||||
|
add(index)
|
||||||
|
|
||||||
|
for index in list(hits):
|
||||||
|
for dependent in inv.services[index].get("dependents", []):
|
||||||
|
for svc in inv.named(host, dependent):
|
||||||
|
add(svc["_index"])
|
||||||
|
|
||||||
|
for svc in inv.on(host):
|
||||||
|
if path in svc.get("files", []):
|
||||||
|
add(svc["_index"])
|
||||||
|
if path in svc.get("retires", []):
|
||||||
|
add(svc["_index"])
|
||||||
|
for directory_rule in svc.get("dirs", []):
|
||||||
|
if path == directory_rule or path.startswith(directory_rule + "/"):
|
||||||
|
add(svc["_index"])
|
||||||
|
|
||||||
|
return [svc["name"] for svc in sorted((inv.services[i] for i in hits),
|
||||||
|
key=lambda s: s["_index"])]
|
||||||
|
|
||||||
|
|
||||||
|
def seams_for(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||||
|
"""The Bats seams the selected services require, deduplicated, in order."""
|
||||||
|
seams: list[str] = []
|
||||||
|
for name in select(inv, host, paths):
|
||||||
|
for svc in inv.named(host, name):
|
||||||
|
for seam in svc.get("tests", []):
|
||||||
|
if seam not in seams:
|
||||||
|
seams.append(seam)
|
||||||
|
if seams and "tests/deployment_scoping.bats" not in seams:
|
||||||
|
seams.append("tests/deployment_scoping.bats")
|
||||||
|
return seams
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# renderers + facts
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def render_pihole_hosts(inv: Inventory) -> str:
|
||||||
|
edge = inv.edge_ip()
|
||||||
|
lines = [
|
||||||
|
"# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts",
|
||||||
|
"# format: <IP> <name>). RENDERED — do not edit by hand:",
|
||||||
|
"#",
|
||||||
|
"# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts",
|
||||||
|
"#",
|
||||||
|
"# The source of truth is the `dns:` field of shared/services.yml; the",
|
||||||
|
"# drift gate in tests/service_inventory.bats fails when this file and",
|
||||||
|
"# the inventory disagree. Caddy is consolidated on Dankclaw, so every",
|
||||||
|
"# name — including the services running on dankserver and",
|
||||||
|
"# frankbook-pro — resolves at the Dankclaw edge.",
|
||||||
|
"#",
|
||||||
|
"# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing",
|
||||||
|
"# comment on a data line becomes a HOSTNAME token. No inline notes",
|
||||||
|
"# here; the service attribution lives in the inventory.",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
for svc in inv.services:
|
||||||
|
for name in svc.get("dns", []):
|
||||||
|
lines.append(f"{edge} {name}")
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def scrape_facts(inv: Inventory) -> list[dict[str, str]]:
|
||||||
|
facts = []
|
||||||
|
for svc in inv.services:
|
||||||
|
for job in svc.get("scrape", []):
|
||||||
|
facts.append({"job": job["job"], "target": job["target"],
|
||||||
|
"metrics_path": job.get("metrics_path", "/")})
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def prometheus_facts(path: Path) -> list[dict[str, str]]:
|
||||||
|
doc = yaml.safe_load(path.read_text())
|
||||||
|
facts = []
|
||||||
|
for job in doc.get("scrape_configs", []):
|
||||||
|
for sc in job.get("static_configs", []):
|
||||||
|
for target in sc.get("targets", []):
|
||||||
|
facts.append({"job": job["job_name"], "target": target,
|
||||||
|
"metrics_path": job.get("metrics_path", "/")})
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def probe_facts(inv: Inventory) -> list[str]:
|
||||||
|
facts: list[str] = []
|
||||||
|
for svc in inv.services:
|
||||||
|
probe = svc.get("probe")
|
||||||
|
if isinstance(probe, dict):
|
||||||
|
facts.extend(str(v) for v in probe.values())
|
||||||
|
elif probe:
|
||||||
|
facts.append(str(probe))
|
||||||
|
return facts
|
||||||
|
|
||||||
|
|
||||||
|
def homepage_probes(path: Path) -> list[str]:
|
||||||
|
return sorted({s["probe"] for s in json.loads(path.read_text()) if s.get("probe")})
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# checks
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def check_ownership(inv: Inventory, graph: Graph) -> list[str]:
|
||||||
|
"""Every repo file belongs to a service, a rule, or the unowned list."""
|
||||||
|
problems: list[str] = []
|
||||||
|
exact: set[str] = set()
|
||||||
|
directories: set[str] = set()
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
exact.add(f"{directory}/{unit}.container")
|
||||||
|
for field in ("files", "retires"):
|
||||||
|
exact.update(svc.get(field, []))
|
||||||
|
directories.update(svc.get("dirs", []))
|
||||||
|
# A shared network file is claimed by the units that ride it; the unit
|
||||||
|
# graph is the referrer, and check_units proves every reference resolves.
|
||||||
|
exact.update(graph.networks.values())
|
||||||
|
|
||||||
|
def is_claimed(path: str) -> bool:
|
||||||
|
if path in exact:
|
||||||
|
return True
|
||||||
|
first = PurePosixPath(path).parts[0]
|
||||||
|
if first in NON_RUNTIME_ROOTS or path in NON_RUNTIME_PATHS or is_trash(path):
|
||||||
|
return True
|
||||||
|
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||||
|
return True
|
||||||
|
for owned in directories | set(inv.unowned):
|
||||||
|
if path == owned or path.startswith(owned + "/"):
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
for path in tracked_files():
|
||||||
|
if not is_claimed(path):
|
||||||
|
problems.append(f"file belongs to no service and is not in unowned: {path}")
|
||||||
|
|
||||||
|
# The reverse direction: an unowned entry that names nothing is stale.
|
||||||
|
for directory in inv.unowned:
|
||||||
|
if not any(p == directory or p.startswith(directory + "/") for p in tracked_files()):
|
||||||
|
problems.append(f"unowned entry names no tracked file: {directory}")
|
||||||
|
|
||||||
|
# Every inventory file/dir claim names a tracked file.
|
||||||
|
for svc in inv.services:
|
||||||
|
# `retires` names a file whose DELETION selects the service, so by
|
||||||
|
# definition it is not in the tree.
|
||||||
|
for field in ("files", "dirs"):
|
||||||
|
for entry in svc.get(field, []):
|
||||||
|
if entry not in tracked_files() and not any(
|
||||||
|
p.startswith(entry + "/") for p in tracked_files()):
|
||||||
|
problems.append(f"{svc['name']}: {field} entry names no tracked file: {entry}")
|
||||||
|
del graph
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def check_units(inv: Inventory, graph: Graph) -> list[str]:
|
||||||
|
"""Every unit file has exactly one owner; every unit reference resolves."""
|
||||||
|
problems: list[str] = []
|
||||||
|
owners: dict[str, list[str]] = {}
|
||||||
|
for svc in inv.services:
|
||||||
|
for host in svc["hosts"]:
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
path = f"{directory}/{unit}.container"
|
||||||
|
owners.setdefault(path, []).append(f"{svc['name']}@{host}")
|
||||||
|
if path not in tracked_files():
|
||||||
|
problems.append(f"{svc['name']}: unit file is not tracked: {path}")
|
||||||
|
for path in tracked_files():
|
||||||
|
if not path.endswith(".container") or is_trash(path):
|
||||||
|
continue
|
||||||
|
found = owners.get(path, [])
|
||||||
|
if not found:
|
||||||
|
problems.append(f"unit file claimed by no service: {path}")
|
||||||
|
elif len(found) > 1:
|
||||||
|
problems.append(f"unit file claimed by {len(found)} services: {path}: {found}")
|
||||||
|
|
||||||
|
# Root units (scripts/*.service) are claimed by exactly one `files:` list.
|
||||||
|
for path in tracked_files():
|
||||||
|
if not (path.startswith("scripts/") and path.endswith(".service")) or is_trash(path):
|
||||||
|
continue
|
||||||
|
found = [s["name"] for s in inv.services if path in s.get("files", [])]
|
||||||
|
if not found:
|
||||||
|
problems.append(f"root unit claimed by no service: {path}")
|
||||||
|
elif len(found) > 1:
|
||||||
|
problems.append(f"root unit claimed by {found}: {path}")
|
||||||
|
|
||||||
|
# Every Network= reference in a unit file names a file on the same host.
|
||||||
|
for (host, unit), path in graph.units.items():
|
||||||
|
for value in _network_values(REPO_ROOT / path):
|
||||||
|
if value.endswith(".container"):
|
||||||
|
target = (host, value[: -len(".container")])
|
||||||
|
if target not in graph.units:
|
||||||
|
problems.append(f"{path}: Network={value} names no unit on {host}")
|
||||||
|
elif value.endswith(".network"):
|
||||||
|
target = (host, value[: -len(".network")])
|
||||||
|
if target not in graph.networks:
|
||||||
|
problems.append(f"{path}: Network={value} names no network on {host}")
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
def check_config(inv: Inventory) -> list[str]:
|
||||||
|
problems: list[str] = []
|
||||||
|
for svc in inv.services:
|
||||||
|
label = f"{'+'.join(svc['hosts'])}/{svc['name']}"
|
||||||
|
for key in REQUIRED_KEYS:
|
||||||
|
if key not in svc:
|
||||||
|
problems.append(f"{label}: missing required key {key}")
|
||||||
|
if svc.get("kind") not in KINDS:
|
||||||
|
problems.append(f"{label}: kind {svc.get('kind')!r} not one of {sorted(KINDS)}")
|
||||||
|
unknown = set(svc.get("hosts", [])) - set(inv.hosts)
|
||||||
|
if unknown:
|
||||||
|
problems.append(f"{label}: undeclared hosts: {sorted(unknown)}")
|
||||||
|
if svc.get("units") and not svc.get("unit_dir"):
|
||||||
|
problems.append(f"{label}: units declared without unit_dir")
|
||||||
|
if svc.get("deployed_by") and not inv.any_named(svc["deployed_by"]):
|
||||||
|
problems.append(f"{label}: deployed_by names no service: {svc['deployed_by']}")
|
||||||
|
if (svc.get("health") and svc.get("wait") is None) or \
|
||||||
|
(svc.get("wait") is not None and not svc.get("health")):
|
||||||
|
problems.append(f"{label}: health and wait are a pair")
|
||||||
|
for seam in svc.get("tests", []):
|
||||||
|
if seam not in tracked_files():
|
||||||
|
problems.append(f"{label}: seam does not exist: {seam}")
|
||||||
|
for name in svc.get("dns", []):
|
||||||
|
if not str(name).replace(".", "").replace("-", "").isalnum():
|
||||||
|
problems.append(f"{label}: dns name is not a plain hostname: {name}")
|
||||||
|
if svc.get("kind") == "root-unit" and svc.get("unit_dir"):
|
||||||
|
problems.append(f"{label}: a root unit lives in scripts/, not unit_dir")
|
||||||
|
seen: set[str] = set()
|
||||||
|
for svc in inv.services:
|
||||||
|
key = (svc["name"], tuple(svc["hosts"]))
|
||||||
|
if key in seen:
|
||||||
|
problems.append(f"duplicate inventory entry: {svc['name']} on {svc['hosts']}")
|
||||||
|
seen.add(key)
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
# --- node-exporter quadlet template (rank 6) --------------------------------
|
||||||
|
# Parallel per-host copies kept in step by a comment drift apart; the
|
||||||
|
# template is the single source, and `check` drift-gates every tracked
|
||||||
|
# rendering against it.
|
||||||
|
NODE_EXPORTER_TEMPLATE = """\
|
||||||
|
# Quadlet unit: node-exporter (rootless, boot-persistent) — RENDERED ARTIFACT.
|
||||||
|
# Regenerate: python3 scripts/service-map.py render node-exporter-unit --host {host}
|
||||||
|
# Source: shared/services.yml (entry node-exporter) + this template.
|
||||||
|
#
|
||||||
|
# Install path on {host}: ~/.config/containers/systemd/node-exporter.container
|
||||||
|
# Unit name: node-exporter.service (container name: node-exporter)
|
||||||
|
#
|
||||||
|
# Host metrics source for Prometheus. Replaces the compose service of the same
|
||||||
|
# name (the "Observability" block): Dankclaw moved to this quadlet on
|
||||||
|
# 2026-09-11, Dankserver on 2026-09-16 — its compose-only copy died at a reboot
|
||||||
|
# and stayed dead (rootless Podman does not restart bare containers after the
|
||||||
|
# explicit stop shutdown performs), dead for a day and a half before anyone
|
||||||
|
# noticed. One owner per container name; no compose file remains anywhere.
|
||||||
|
#
|
||||||
|
# HOST NAMESPACE. It must run in the host network namespace to report the
|
||||||
|
# host's own interfaces/connections, and --pid=host for the processes and
|
||||||
|
# cgroups collectors. "localhost" inside the container is therefore the host's
|
||||||
|
# loopback, which is what the homepage status timer probes.
|
||||||
|
#
|
||||||
|
# LISTEN ADDRESSES. Exactly two: loopback for the homepage status probe, and
|
||||||
|
# the LAN address, because Prometheus — which runs in a container on
|
||||||
|
# dankclaw_default — scrapes http://{lan}:{port}/metrics
|
||||||
|
# (shared/prometheus/prometheus.yml, job "node-{host}"). node_exporter has no
|
||||||
|
# authentication, so the wildcard bind stays off the table; this is the
|
||||||
|
# pairing that removes the unauthenticated observability surface on every
|
||||||
|
# host, and both consumers are accounted for.
|
||||||
|
#
|
||||||
|
# MOUNTS. The container reads the host's /proc, /sys and / as /host/*
|
||||||
|
# read-only and is pointed at them with --path.*; it writes nothing anywhere.
|
||||||
|
|
||||||
|
[Unit]
|
||||||
|
Description=Prometheus node exporter ({title} host metrics)
|
||||||
|
After=network-online.target
|
||||||
|
Wants=network-online.target
|
||||||
|
|
||||||
|
[Container]
|
||||||
|
ContainerName=node-exporter
|
||||||
|
Image=docker.io/prom/node-exporter:latest
|
||||||
|
Network=host
|
||||||
|
PodmanArgs=--pid=host
|
||||||
|
Exec=--path.procfs=/host/proc --path.sysfs=/host/sys --path.rootfs=/rootfs --web.listen-address=127.0.0.1:{port} --web.listen-address={lan}:{port}
|
||||||
|
Volume=/proc:/host/proc:ro
|
||||||
|
Volume=/sys:/host/sys:ro
|
||||||
|
Volume=/:/rootfs:ro
|
||||||
|
HealthCmd=wget --spider -q http://localhost:{port}/metrics
|
||||||
|
HealthInterval=30s
|
||||||
|
HealthTimeout=10s
|
||||||
|
HealthRetries=3
|
||||||
|
HealthStartPeriod=30s
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=default.target
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def render_node_exporter_unit(inv: Inventory, host: str) -> str:
|
||||||
|
"""The node-exporter quadlet for one host: shared template, per-host
|
||||||
|
title, LAN address from the hosts map, port from the scrape job."""
|
||||||
|
svc = inv.service("node-exporter")
|
||||||
|
if svc is None:
|
||||||
|
raise SystemExit("inventory has no node-exporter entry")
|
||||||
|
job = f"node-{host}"
|
||||||
|
port = next((str(e["target"]).rsplit(":", 1)[1] for e in svc.get("scrape", [])
|
||||||
|
if e.get("job") == job), None)
|
||||||
|
if port is None:
|
||||||
|
raise SystemExit(f"node-exporter scrape declares no job {job}")
|
||||||
|
lan = inv.hosts[host]["address"]
|
||||||
|
return NODE_EXPORTER_TEMPLATE.format(host=host, title=host.capitalize(),
|
||||||
|
lan=lan, port=port)
|
||||||
|
|
||||||
|
|
||||||
|
def check_drift(inv: Inventory) -> list[str]:
|
||||||
|
problems: list[str] = []
|
||||||
|
rendered = render_pihole_hosts(inv)
|
||||||
|
hosts_file = REPO_ROOT / "dankclaw" / "pihole" / "addn-hosts"
|
||||||
|
if hosts_file.read_text() != rendered:
|
||||||
|
problems.append("dankclaw/pihole/addn-hosts is not the render of the inventory "
|
||||||
|
"(scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts)")
|
||||||
|
|
||||||
|
want = sorted(json.dumps(f, sort_keys=True) for f in scrape_facts(inv))
|
||||||
|
have = sorted(json.dumps(f, sort_keys=True) for f in
|
||||||
|
prometheus_facts(REPO_ROOT / "shared" / "prometheus" / "prometheus.yml"))
|
||||||
|
for extra in [f for f in have if f not in want]:
|
||||||
|
problems.append(f"prometheus scrapes a target the inventory does not declare: {extra}")
|
||||||
|
for missing in [f for f in want if f not in have]:
|
||||||
|
problems.append(f"inventory declares a scrape prometheus.yml lacks: {missing}")
|
||||||
|
|
||||||
|
svc = inv.service("node-exporter")
|
||||||
|
for host in (svc or {}).get("hosts", []):
|
||||||
|
for directory in inv.unit_dirs(svc, host):
|
||||||
|
path = REPO_ROOT / directory / "node-exporter.container"
|
||||||
|
if path.exists() and path.read_text() != render_node_exporter_unit(inv, host):
|
||||||
|
problems.append(
|
||||||
|
f"{directory}/node-exporter.container is not the render of the "
|
||||||
|
f"template (scripts/service-map.py render node-exporter-unit --host {host})")
|
||||||
|
|
||||||
|
probes_want = sorted(set(probe_facts(inv)))
|
||||||
|
probes_have = homepage_probes(REPO_ROOT / "shared" / "homepage" / "services.json")
|
||||||
|
for extra in [p for p in probes_have if p not in probes_want]:
|
||||||
|
problems.append(f"homepage probes a target the inventory does not declare: {extra}")
|
||||||
|
for missing in [p for p in probes_want if p not in probes_have]:
|
||||||
|
problems.append(f"inventory declares a probe homepage does not list: {missing}")
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
CHECKS = {
|
||||||
|
"config": lambda inv, graph: check_config(inv),
|
||||||
|
"units": check_units,
|
||||||
|
"ownership": check_ownership,
|
||||||
|
"drift": lambda inv, graph: check_drift(inv),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def check(inv: Inventory, names: list[str]) -> list[str]:
|
||||||
|
graph = graph_for(tuple(inv.hosts))
|
||||||
|
problems: list[str] = []
|
||||||
|
for name in names or list(CHECKS):
|
||||||
|
problems.extend(CHECKS[name](inv, graph))
|
||||||
|
return problems
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# CLI
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__,
|
||||||
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
parser.add_argument("--inventory", type=Path, default=DEFAULT_INVENTORY)
|
||||||
|
sub = parser.add_subparsers(dest="command", required=True)
|
||||||
|
|
||||||
|
sub.add_parser("hosts")
|
||||||
|
p = sub.add_parser("services")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p = sub.add_parser("allows")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("plan")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("paths", nargs="+")
|
||||||
|
p = sub.add_parser("seams")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p.add_argument("paths", nargs="+")
|
||||||
|
p = sub.add_parser("owner")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("path")
|
||||||
|
p = sub.add_parser("units")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("stack")
|
||||||
|
p.add_argument("--host", required=True)
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p = sub.add_parser("get")
|
||||||
|
p.add_argument("--host")
|
||||||
|
p.add_argument("--service", required=True)
|
||||||
|
p.add_argument("field")
|
||||||
|
p = sub.add_parser("render")
|
||||||
|
p.add_argument("artifact", choices=("pihole-hosts", "node-exporter-unit"))
|
||||||
|
p.add_argument("--host", choices=list(Hosts) if False else None)
|
||||||
|
p = sub.add_parser("check")
|
||||||
|
p.add_argument("names", nargs="*")
|
||||||
|
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
inv = Inventory.load(args.inventory)
|
||||||
|
|
||||||
|
if args.command == "hosts":
|
||||||
|
print("\n".join(inv.hosts))
|
||||||
|
elif args.command == "services":
|
||||||
|
services = inv.on(args.host) if args.host else inv.services
|
||||||
|
seen: list[str] = []
|
||||||
|
for svc in services:
|
||||||
|
if svc["name"] not in seen:
|
||||||
|
seen.append(svc["name"])
|
||||||
|
print("\n".join(seen))
|
||||||
|
elif args.command == "allows":
|
||||||
|
return 0 if inv.named(args.host, args.service) else 1
|
||||||
|
elif args.command == "plan":
|
||||||
|
print("\n".join(select(inv, args.host, args.paths)))
|
||||||
|
elif args.command == "seams":
|
||||||
|
if args.host is not None:
|
||||||
|
seams = seams_for(inv, args.host, args.paths)
|
||||||
|
else:
|
||||||
|
# No host: the union across every host (the dev machine's
|
||||||
|
# pre-commit mapper has no deployment target).
|
||||||
|
seams = []
|
||||||
|
for host in inv.hosts:
|
||||||
|
for seam in seams_for(inv, host, args.paths):
|
||||||
|
if seam not in seams:
|
||||||
|
seams.append(seam)
|
||||||
|
print("\n".join(seams))
|
||||||
|
elif args.command == "owner":
|
||||||
|
found = select(inv, args.host, [args.path])
|
||||||
|
print(found[-1] if found else "")
|
||||||
|
elif args.command == "units":
|
||||||
|
matches = inv.named(args.host, args.service)
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
for svc in matches:
|
||||||
|
for directory in inv.unit_dirs(svc, args.host):
|
||||||
|
for unit in svc.get("units", []):
|
||||||
|
print(f"{unit} {directory}")
|
||||||
|
elif args.command == "stack":
|
||||||
|
matches = inv.named(args.host, args.service)
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
owner = matches[0].get("deployed_by") or args.service
|
||||||
|
print(owner)
|
||||||
|
for svc in inv.on(args.host):
|
||||||
|
if svc.get("deployed_by") == owner and svc["name"] != owner:
|
||||||
|
print(svc["name"])
|
||||||
|
elif args.command == "get":
|
||||||
|
matches = inv.on(args.host) if args.host else inv.services
|
||||||
|
matches = [s for s in matches if s["name"] == args.service]
|
||||||
|
if not matches:
|
||||||
|
print(f"no service {args.service}" + (f" on {args.host}" if args.host else ""),
|
||||||
|
file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
values = [s.get(args.field) for s in matches if args.field in s]
|
||||||
|
print(json.dumps(values[0] if len(values) == 1 else values))
|
||||||
|
elif args.command == "render":
|
||||||
|
if args.artifact == "pihole-hosts":
|
||||||
|
sys.stdout.write(render_pihole_hosts(inv))
|
||||||
|
else:
|
||||||
|
if not args.host:
|
||||||
|
print("render node-exporter-unit requires --host", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
sys.stdout.write(render_node_exporter_unit(inv, args.host))
|
||||||
|
elif args.command == "check":
|
||||||
|
problems = check(inv, args.names)
|
||||||
|
if problems:
|
||||||
|
for problem in problems:
|
||||||
|
print(f"FAIL: {problem}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print("OK: the service inventory is sound (schema, units, ownership, drift)")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
@@ -293,12 +293,12 @@ def main():
|
|||||||
ddt, dsh = meta[f"{pre0}.mlp.experts.0.down_proj.weight"][3:5]
|
ddt, dsh = meta[f"{pre0}.mlp.experts.0.down_proj.weight"][3:5]
|
||||||
gsd, gssh = meta[f"{pre0}.mlp.experts.0.gate_proj.weight_scale_inv"][3:5]
|
gsd, gssh = meta[f"{pre0}.mlp.experts.0.gate_proj.weight_scale_inv"][3:5]
|
||||||
dsd, dssh = meta[f"{pre0}.mlp.experts.0.down_proj.weight_scale_inv"][3:5]
|
dsd, dssh = meta[f"{pre0}.mlp.experts.0.down_proj.weight_scale_inv"][3:5]
|
||||||
scale_law = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
|
scale_contract = (gdt == F8 and ddt == F8 and gsd == BF16 and dsd == BF16
|
||||||
and list(gssh) == [(gsh[0] + 127) // 128, (gsh[1] + 127) // 128]
|
and list(gssh) == [(gsh[0] + 127) // 128, (gsh[1] + 127) // 128]
|
||||||
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
|
||||||
if not scale_law:
|
if not scale_contract:
|
||||||
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
|
||||||
f"down {ddt}{dsh} scale {dsd}{dssh} (scale law 128x128)")
|
f"down {ddt}{dsh} scale {dsd}{dssh} (fp8 block scales cover 128x128 tiles)")
|
||||||
return 2
|
return 2
|
||||||
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
|
||||||
"gs": tuple(gssh), "ds": tuple(dssh)}
|
"gs": tuple(gssh), "ds": tuple(dssh)}
|
||||||
|
|||||||
+57
-190
@@ -62,234 +62,101 @@ declare -a TESTS=()
|
|||||||
add_test() {
|
add_test() {
|
||||||
local test="$1"
|
local test="$1"
|
||||||
local existing
|
local existing
|
||||||
|
MAPPED=1
|
||||||
for existing in "${TESTS[@]:-}"; do
|
for existing in "${TESTS[@]:-}"; do
|
||||||
[ "$existing" != "$test" ] || return 0
|
[ "$existing" != "$test" ] || return 0
|
||||||
done
|
done
|
||||||
TESTS+=("$test")
|
TESTS+=("$test")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# The path->test mapping has two layers:
|
||||||
|
# 1. cross-cutting seams that no single service owns (compose validity,
|
||||||
|
# env consistency, the deployer itself, docs, syntax);
|
||||||
|
# 2. the service inventory (shared/services.yml via service-map.py
|
||||||
|
# "seams"): every service the path selects on ANY host contributes its
|
||||||
|
# declared tests. Service seams are the services' own — the former
|
||||||
|
# shotgun fallbacks (every dankclaw quadlet unit ran the open-webui,
|
||||||
|
# pihole, and searxng seams) are retired; the inventory names each
|
||||||
|
# service's real seams, and adding a service is a data edit.
|
||||||
for path in "${CHANGED[@]}"; do
|
for path in "${CHANGED[@]}"; do
|
||||||
|
MAPPED=0
|
||||||
case "$path" in
|
case "$path" in
|
||||||
tests/*.bats)
|
tests/*.bats)
|
||||||
add_test "$path"
|
add_test "$path"
|
||||||
|
continue
|
||||||
;;
|
;;
|
||||||
tests/helpers.bash|tests/run.sh)
|
tests/helpers.bash|tests/run.sh)
|
||||||
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
||||||
;;
|
;;
|
||||||
.githooks/*)
|
|
||||||
add_test tests/changed_tests.bats
|
|
||||||
;;
|
|
||||||
dankserver/compose.yaml|dankclaw/compose.yaml)
|
|
||||||
add_test tests/compose_validity.bats
|
|
||||||
add_test tests/env_consistency.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankserver/Caddyfile|dankclaw/Caddyfile)
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
tests/e2e/*)
|
tests/e2e/*)
|
||||||
add_test tests/e2e_scaffold.bats
|
add_test tests/e2e_scaffold.bats
|
||||||
|
continue
|
||||||
;;
|
;;
|
||||||
|
.githooks/*)
|
||||||
|
add_test tests/changed_tests.bats
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
# Archive material and host-level facts: a path under any
|
||||||
|
# .AGENTTRASH/ directory is retired-by-policy (the planner plans
|
||||||
|
# nothing from it), udev rules are applied by manual sudo install,
|
||||||
|
# reports are frozen artifacts, and ignore-lists change nothing at
|
||||||
|
# runtime. They select no focused test; without this case they would
|
||||||
|
# trip the mapping gate below.
|
||||||
|
.AGENTTRASH/*|*/.AGENTTRASH/*|dankclaw/udev/*|report/*|.gitignore|.env.example)
|
||||||
|
continue
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Cross-cutting seams.
|
||||||
|
case "$path" in
|
||||||
dankserver/.env.example|dankclaw/.env.example)
|
dankserver/.env.example|dankclaw/.env.example)
|
||||||
add_test tests/env_consistency.bats
|
add_test tests/env_consistency.bats
|
||||||
;;
|
;;
|
||||||
shared/homepage/*)
|
dankserver/Caddyfile|dankclaw/Caddyfile)
|
||||||
add_test tests/homepage_caddy.bats
|
|
||||||
add_test tests/homepage_config.bats
|
|
||||||
add_test tests/homepage_deploy.bats
|
|
||||||
add_test tests/homepage_pages.bats
|
|
||||||
add_test tests/homepage_checker.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
shared/searxng/*)
|
|
||||||
add_test tests/searxng_limiter.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# The five observability containers left compose for Quadlet units on
|
|
||||||
# 2026-09-11. Their seam pins boot persistence and the loopback-only
|
|
||||||
# publish posture that keeps them off every host interface. This case
|
|
||||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
||||||
# would otherwise swallow it into unrelated seams.
|
|
||||||
dankclaw/observability/*)
|
|
||||||
add_test tests/observability_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
add_test tests/caddyfile.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
# Caddy left compose.yaml on 2026-08-26 for a quadlet; caddyfile.bats
|
|
||||||
# now asserts against the quadlet itself. Route every change under
|
|
||||||
# dankclaw/caddy/ to both seams (this block must precede the
|
|
||||||
# dankclaw/*.container glob, which would otherwise swallow the unit).
|
|
||||||
dankclaw/caddy/*)
|
|
||||||
add_test tests/caddy_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/*.container|dankclaw/*.network)
|
|
||||||
add_test tests/open_webui_quadlet.bats
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# Anubis is a quadlet + Caddyfile pair: the chain (public listener ->
|
|
||||||
# anubis:8923 -> internal :8016 listener) spans both, so any change
|
|
||||||
# under the service dir must re-run the caddyfile seam too. This case
|
|
||||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
|
||||||
# would otherwise swallow it into unrelated seams.
|
|
||||||
dankclaw/anubis/*)
|
|
||||||
add_test tests/anubis.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/homepage_caddy.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# frank-dragoon is a rootless quadlet again since 2026-09-16 (the
|
|
||||||
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
|
||||||
# podman posture, so it gets its own seam rather than the generic
|
|
||||||
# fallback below.
|
|
||||||
dankclaw/frank-dragoon/*)
|
|
||||||
add_test tests/frank_dragoon_unit.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
# Quadlets live under dankclaw/<svc>/quadlet/ (reorganized out of the
|
|
||||||
# top-level dankclaw/*.container globs above). Map the whole subtree so
|
|
||||||
# changes to the pihole, open-webui, searxng, and gitea quadlets hit
|
|
||||||
# their seam tests. The pihole-specific block must come first: shell
|
|
||||||
# case matches the first pattern, and dankclaw/pihole/quadlet/*.container
|
|
||||||
# also satisfies the dankclaw/*/quadlet/*.container fallback.
|
|
||||||
dankclaw/pihole/*)
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/*/quadlet/*.container|dankclaw/*/quadlet/*.network)
|
|
||||||
add_test tests/open_webui_quadlet.bats
|
|
||||||
add_test tests/pihole.bats
|
|
||||||
add_test tests/searxng_quadlet.bats
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
;;
|
||||||
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
||||||
add_test tests/deployment_flow.bats
|
add_test tests/deployment_flow.bats
|
||||||
add_test tests/deployment_scoping.bats
|
add_test tests/deployment_scoping.bats
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
|
scripts/service-map.py|shared/services.yml)
|
||||||
|
add_test tests/service_inventory.bats
|
||||||
|
add_test tests/deployment_scoping.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
scripts/lib/*.sh)
|
||||||
|
add_test tests/llm_gates.bats
|
||||||
|
add_test tests/scripts_syntax.bats
|
||||||
|
;;
|
||||||
|
serverAppConfigs-docs/*|README.md|AGENTS.md)
|
||||||
|
add_test tests/docs_completeness.bats
|
||||||
|
add_test tests/deployment_scoping.bats
|
||||||
|
;;
|
||||||
scripts/mount-entropy.sh)
|
scripts/mount-entropy.sh)
|
||||||
add_test tests/mount_entropy.bats
|
add_test tests/mount_entropy.bats
|
||||||
;;
|
|
||||||
# Archive material: a path under any .AGENTTRASH/ directory is
|
|
||||||
# retired-by-policy (deploy-plan.py plans nothing from it), so it
|
|
||||||
# selects no focused test. It still needs its own case: without one
|
|
||||||
# the wrapper refuses retirement commits, whose trash moves show up
|
|
||||||
# as additions, with "no focused-test mapping".
|
|
||||||
.AGENTTRASH/*|*/.AGENTTRASH/*)
|
|
||||||
;;
|
|
||||||
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
|
|
||||||
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
|
|
||||||
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
|
|
||||||
# live in scripts/, which is why this case has to precede the generic
|
|
||||||
# scripts/*.service and scripts/* fallbacks below.
|
|
||||||
scripts/frank-archon.*|scripts/serve-frank-archon.sh)
|
|
||||||
add_test tests/frank_archon_unit.bats
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
# frank-dragoon's chat template lives in scripts/ (moved there on
|
old/scripts/*)
|
||||||
# 2026-09-26 when its old home was retired): a template edit changes
|
|
||||||
# what the live endpoint renders, so it selects the large unit seam.
|
|
||||||
scripts/qwen-sharp-v22.4.0.jinja)
|
|
||||||
add_test tests/frank_dragoon_unit.bats
|
|
||||||
;;
|
|
||||||
# frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
|
||||||
# binary, so no generated quadlet is involved. Its env, unit and serve
|
|
||||||
# script all live in scripts/, which is why this case has to precede
|
|
||||||
# the generic scripts/*.service and scripts/* fallbacks below.
|
|
||||||
scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
|
|
||||||
add_test tests/frank_zealot_unit.bats
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
scripts/*.service)
|
|
||||||
add_test tests/llm_units.bats
|
|
||||||
add_test tests/scripts_syntax.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
scripts/*)
|
scripts/*)
|
||||||
add_test tests/scripts_syntax.bats
|
add_test tests/scripts_syntax.bats
|
||||||
;;
|
;;
|
||||||
serverAppConfigs-docs/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/hermes/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
;;
|
|
||||||
README.md|AGENTS.md)
|
|
||||||
# Project guidance changes can affect the documented deployment
|
|
||||||
# contract; validate the docs vault and scoped deployment seams.
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/comfyui/*)
|
|
||||||
add_test tests/docs_completeness.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# loki and alloy images are built from this repo's Containerfiles; the
|
|
||||||
# loki one carries the healthcheck command (see its comment), so both
|
|
||||||
# seams read it.
|
|
||||||
dankclaw/loki/*|dankclaw/alloy/*)
|
|
||||||
add_test tests/observability_quadlet.bats
|
|
||||||
add_test tests/caddyfile.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
dankclaw/udev/*)
|
|
||||||
# Host-level udev rules are applied by manual sudo install + udevadm
|
|
||||||
# trigger, not dispatched as a service; nothing here is bats-testable.
|
|
||||||
;;
|
|
||||||
.AGENTTRASH/*)
|
|
||||||
# Trash moves are not testable; skip.
|
|
||||||
;;
|
|
||||||
.gitignore|.env.example)
|
|
||||||
# Ignore/ignore-list changes change nothing at runtime; not testable.
|
|
||||||
;;
|
|
||||||
old/scripts/*|old/scripts/**)
|
|
||||||
add_test tests/scripts_syntax.bats
|
|
||||||
;;
|
|
||||||
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
|
|
||||||
# copyparty, plex, tautulli): the seam checks boot-persistence +
|
|
||||||
# v1-generator contract; deploy-flow seams guard the deployer routing.
|
|
||||||
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
|
|
||||||
add_test tests/dankserver_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# FrankBook-Pro units live inside the podman machine VM; the seam
|
|
||||||
# pins boot persistence, the gluetun netns invariant, and the
|
|
||||||
# plan-locked volume layout. Deployer changes route here too.
|
|
||||||
frankbook-pro/*)
|
|
||||||
add_test tests/frankbook_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
# Seerr unit on dankclaw: boot persistence, loopback-only port
|
|
||||||
# posture, and the deployer routing that installs it.
|
|
||||||
dankclaw/seerr/*)
|
|
||||||
add_test tests/seerr_quadlet.bats
|
|
||||||
add_test tests/deployment_flow.bats
|
|
||||||
add_test tests/deployment_scoping.bats
|
|
||||||
;;
|
|
||||||
report/*)
|
|
||||||
# Frozen audit reports are point-in-time artifacts; not testable.
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "no focused-test mapping for '$path'; add one before committing"
|
|
||||||
;;
|
|
||||||
esac
|
esac
|
||||||
|
|
||||||
|
# The inventory's declared seams for every service this path selects.
|
||||||
|
while IFS= read -r seam; do
|
||||||
|
[ -n "$seam" ] && add_test "$seam"
|
||||||
|
done < <(python3 "$REPO_ROOT/scripts/service-map.py" seams -- "$path" 2>/dev/null || true)
|
||||||
|
|
||||||
|
# Mapping gate: a path that selected no test at all — no service seam,
|
||||||
|
# no cross-cutting rule — is an unmapped area of the repo; refuse to
|
||||||
|
# commit it silently (the old catch-all `die`).
|
||||||
|
if [ "$MAPPED" -eq 0 ]; then
|
||||||
|
die "no focused-test mapping for '$path'; map it in the inventory (shared/services.yml) before committing"
|
||||||
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
||||||
|
|||||||
@@ -0,0 +1,100 @@
|
|||||||
|
---
|
||||||
|
type: reference
|
||||||
|
title: Service Inventory
|
||||||
|
description: shared/services.yml is the one inventory of deployable services; scripts/service-map.py is its only reader. The planner, both deploy allow-lists, the changed-test seam mapper, the Pi-hole hosts render, the Prometheus scrape facts, and the homepage probes all derive from it.
|
||||||
|
tags: [deployment, inventory, planner, quadlet, prometheus, pihole, homepage]
|
||||||
|
timestamp: 2026-10-13T00:00:00-07:00
|
||||||
|
---
|
||||||
|
|
||||||
|
# Service Inventory
|
||||||
|
|
||||||
|
Before 2026-10-13 the same facts lived in four hand-maintained lists — the
|
||||||
|
planner's path rules (`deploy-plan.py`), the `--service` allow-list in
|
||||||
|
`deploy.sh`, the near-copy in `deploy-remote.sh`, and the changed-test mapper
|
||||||
|
(`test-changed.sh`) — plus per-fact copies in `prometheus.yml`,
|
||||||
|
`addn-hosts`, and `services.json`. They drifted on every swap (the planner
|
||||||
|
still mapped the retired `qwen-sharp` template to frank-dragoon; the allow-list
|
||||||
|
still listed the *arr stack under Dankserver after it moved to FrankBook-Pro).
|
||||||
|
Now there is **one inventory and one reader**:
|
||||||
|
|
||||||
|
- `shared/services.yml` — the data: every deployable service, its hosts, unit
|
||||||
|
dir(s), unit files, path claims, tests, scrape/DNS/probe facts, aliases.
|
||||||
|
- `scripts/service-map.py` — the only reader. CLI subcommands:
|
||||||
|
`hosts`, `services`, `allows`, `plan`, `seams`, `owner`, `units`, `stack`,
|
||||||
|
`get`, `render`, `check`.
|
||||||
|
|
||||||
|
## 1. Consumers
|
||||||
|
|
||||||
|
|Consumer|Call|
|
||||||
|
|---|---|
|
||||||
|
|`scripts/deploy-plan.py`|thin wrapper → `plan --host TARGET [paths...]` (keeps the `--repo/--base/--head` and `--paths` CLIs)|
|
||||||
|
|`scripts/deploy.sh` (`--service` gate)|`allows --host T --service S`|
|
||||||
|
|`scripts/deploy-remote.sh` (`service_is_allowed`)|same|
|
||||||
|
|`scripts/test-changed.sh`|`seams [paths...]` — a service's declared `tests` are its seams|
|
||||||
|
|`dankclaw/pihole/addn-hosts`|`render pihole-hosts` — the tracked file is a rendered artifact|
|
||||||
|
|`*/observability/quadlet/node-exporter.container`|`render node-exporter-unit --host H` — the MIRROR pair became one template; per-host title, LAN address (hosts map), and port (scrape job) fill it|
|
||||||
|
|Prometheus scrape jobs, LAN DNS, homepage probes|`check` compares the tracked configs against the inventory's `scrape`/`dns`/`probe` fields (drift gate)|
|
||||||
|
|
||||||
|
Selection rules the engine encodes (see `tests/fixtures/service-plan-golden.tsv`,
|
||||||
|
the pinned characterization of every tracked path × every target):
|
||||||
|
|
||||||
|
- **Host scoping** — a host's top-level directory can only select that host's
|
||||||
|
services. `frankbook-pro/**` plans nothing on Dankserver (the old planner
|
||||||
|
had a stale Dankserver *arr list).
|
||||||
|
- **Quadlet coupling graph** — `Network=x.container` rides x's netns;
|
||||||
|
`Network=y.network` joins a shared network. Changing the parent (or the
|
||||||
|
network file) selects every rider transitively, in their own directories
|
||||||
|
(`dankclaw/searxng/quadlet/gluetun.container` → gluetun + searxng + marreta;
|
||||||
|
`frankbook-pro/gluetun/quadlet/gluetun.container` → the whole arr stack).
|
||||||
|
- **Stacks** — a service's `stack` members deploy and restart as one handler
|
||||||
|
(romm trio, hermes trio).
|
||||||
|
- **Retirement** — removing a service from `services.yml` removes it from the
|
||||||
|
planner, both allow-lists, and the seam mapper in one data edit; a pending
|
||||||
|
deploy-state entry for it is auto-triaged (see [[Deploy State Triage]]).
|
||||||
|
|
||||||
|
## 2. Integrity gates (`check`)
|
||||||
|
|
||||||
|
`python3 scripts/service-map.py check` runs, and
|
||||||
|
`tests/service_inventory.bats` pins:
|
||||||
|
|
||||||
|
- **schema** — required fields per entry, known keys only;
|
||||||
|
- **units** — every unit file named by an entry exists under its `unit_dir`
|
||||||
|
on the right host; each unit name is claimed by exactly one service per host;
|
||||||
|
- **ownership** — every tracked quadlet `.container`/`.network` file is claimed
|
||||||
|
by exactly one service (trash excluded);
|
||||||
|
- **paths** — every claimed `files`/`dirs`/`retires` glob matches ≥1 tracked file;
|
||||||
|
- **compose** — the container names in each host's `compose.yaml` have exactly
|
||||||
|
one quadlet owner (the [[Quadlet Lifecycle]] two-owners trap);
|
||||||
|
- **drift** — `addn-hosts` equals its render, each tracked node-exporter unit
|
||||||
|
equals its per-host render; the Prometheus job set, the DNS names, and the
|
||||||
|
homepage probe ports equal the inventory's fields.
|
||||||
|
|
||||||
|
## 3. Adding, moving, retiring a service
|
||||||
|
|
||||||
|
- **Add**: append an entry (name, hosts, kind, unit_dir, units, paths, tests,
|
||||||
|
…), add the quadlet file, run `check` — `plan`/`allows`/`seams` pick it up
|
||||||
|
with zero script edits.
|
||||||
|
- **Move between hosts**: edit `hosts` (and per-host `unit_dir`/`probe` maps).
|
||||||
|
- **Retire**: delete the entry, move files to `.AGENTTRASH/` (see
|
||||||
|
[[Testing Policy]] for the seam the mapper applies to trash paths).
|
||||||
|
|
||||||
|
Field notes: `alias` carries deployable names that differ from the service
|
||||||
|
name (`calibre-web` deploys the unit `calibre-server`); `probe` for multi-host
|
||||||
|
services is a `{host: port}` map; `dependents` names services whose config
|
||||||
|
embeds a reference to this one (config-only coupling — a `.env` edit selects
|
||||||
|
both, a unit edit selects the graph closure).
|
||||||
|
|
||||||
|
## 4. Known cutover deltas (2026-10-13)
|
||||||
|
|
||||||
|
The 17 deltas against the old planner are all in these classes: anubis paths
|
||||||
|
now select `anubis` (the old planner had no rule at all); cross-host paths
|
||||||
|
select nothing; `scripts/qwen-sharp-v22.4.0.jinja` (retired template) plans
|
||||||
|
nothing while `scripts/qwen38-froggeric-v22.5.jinja` (the live frank-archon
|
||||||
|
template, previously unmapped) plans frank-archon;
|
||||||
|
`dankserver/romm/romm.container` (top-level quadlet dir the old unit-file rule
|
||||||
|
missed) now plans the romm trio.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
Related: [[Deployment Flow]], [[Testing Policy]], [[Quadlet Lifecycle]],
|
||||||
|
[[Container Network Namespaces]], [[Deploy State Triage]]
|
||||||
@@ -15,10 +15,9 @@ This repository tests its configuration at defined seams. The tests run with `ba
|
|||||||
|
|
||||||
| Seam | File | What it asserts |
|
| Seam | File | What it asserts |
|
||||||
|------|------|-----------------|
|
|------|------|-----------------|
|
||||||
| Compose validity | `tests/compose_validity.bats` | YAML parses, services have images, no named volumes |
|
|
||||||
| Env consistency | `tests/env_consistency.bats` | every `${VAR}` is defined in `.env.example`; no real secrets; the live gitea password never appears |
|
| Env consistency | `tests/env_consistency.bats` | every `${VAR}` is defined in `.env.example`; no real secrets; the live gitea password never appears |
|
||||||
| Caddyfile validity | `tests/caddyfile.bats` | per-host Caddyfile, `.local` convention, balanced braces, basic auth present |
|
| Caddyfile validity | `tests/caddyfile.bats` | per-host Caddyfile, `.local` convention, balanced braces, basic auth present |
|
||||||
| Docs completeness | `tests/docs_completeness.bats` | every compose service has a docs page; index links every page |
|
| Docs completeness | `tests/docs_completeness.bats` | every service has a docs page; index links every page; no unresolved vault links |
|
||||||
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
||||||
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
|
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
|
||||||
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
|
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
|
||||||
|
|||||||
@@ -357,11 +357,15 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
|
|||||||
routing matches on hostname, verified 2026-09-02).
|
routing matches on hostname, verified 2026-09-02).
|
||||||
`internal_host = http://caddy:8017`, all three flows set, and an
|
`internal_host = http://caddy:8017`, all three flows set, and an
|
||||||
Application attached — the two provisioning rules above.
|
Application attached — the two provisioning rules above.
|
||||||
2. **Route flip in ONE commit/deploy**: the vhost becomes
|
2. **Route flip in ONE commit/deploy**: the vhost body becomes
|
||||||
`reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
|
`import guarded` — the named snippet at the top of the Caddyfile is the
|
||||||
inside the `http://:8017` listener add `@<svc> host <vhost>` +
|
whole pre-auth chain (`reverse_proxy anubis:8923` + `header_up X-Real-Ip
|
||||||
|
{remote_host}`; rank 6 retired the nine verbatim copies), and inside the
|
||||||
|
`http://:8017` listener add `@<svc> host <vhost>` +
|
||||||
`handle @<svc> { reverse_proxy <upstream> }`. Anubis guard set, Caddy TLS
|
`handle @<svc> { reverse_proxy <upstream> }`. Anubis guard set, Caddy TLS
|
||||||
edge, `:8017` routing table, and provider list change together.
|
edge, `:8017` routing table, and provider list change together.
|
||||||
|
`tests/caddyfile.bats` pins that the chain exists exactly once as the
|
||||||
|
`(guarded)` snippet and every guarded vhost imports it.
|
||||||
3. Deploy via the repo deploy flow (commit first; deploy pushes main itself).
|
3. Deploy via the repo deploy flow (commit first; deploy pushes main itself).
|
||||||
|
|
||||||
**Verification (all three):**
|
**Verification (all three):**
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ conversation slot and unquantized KV back.
|
|||||||
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
|
| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
|
||||||
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-zealot` name), `/v1` OpenAI surface |
|
| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-zealot` name), `/v1` OpenAI surface |
|
||||||
| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`); `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
|
| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`); `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
|
||||||
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
|
| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=always` (raised from `on-failure` 2026-10-13 for fleet parity — exit 0 is a CLEAN stop to systemd, so the weaker policy leaves the compaction hub dark while the unit reports success; `tests/llm_units.bats` gates it), `LimitMEMLOCK=infinity` |
|
||||||
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
|
| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
|
||||||
| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
|
| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
|
||||||
| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
|
| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
|
||||||
|
|||||||
@@ -28,6 +28,7 @@ covers both workloads).
|
|||||||
- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
|
- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
|
||||||
- [[Storage Policy]] — bind mounts, data locations, ownership
|
- [[Storage Policy]] — bind mounts, data locations, ownership
|
||||||
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
|
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
|
||||||
|
- [[Service Inventory]] — shared/services.yml, the one inventory of deployable services, and service-map.py
|
||||||
- [[Testing Policy]] — the test seams and how to run them
|
- [[Testing Policy]] — the test seams and how to run them
|
||||||
- [[E2E Suite]] — the Playwright suite that drives the live guard chain
|
- [[E2E Suite]] — the Playwright suite that drives the live guard chain
|
||||||
- [[Container Network Namespaces]] — which local services are addressable from each Podman network namespace on Dankclaw
|
- [[Container Network Namespaces]] — which local services are addressable from each Podman network namespace on Dankclaw
|
||||||
|
|||||||
@@ -0,0 +1,598 @@
|
|||||||
|
# =============================================================================
|
||||||
|
# services.yml — THE service inventory.
|
||||||
|
#
|
||||||
|
# One entry per deployable service, in canonical deploy order (dependencies
|
||||||
|
# before consumers, the LLM fleet last). Four hand-maintained lists used to
|
||||||
|
# agree with each other by discipline alone; every one of them now READS this
|
||||||
|
# file:
|
||||||
|
#
|
||||||
|
# scripts/deploy-plan.py which services a changed path selects
|
||||||
|
# scripts/deploy-remote.sh which services a target accepts (allow-list)
|
||||||
|
# scripts/test-changed.sh which Bats seams a changed path must re-run
|
||||||
|
# scripts/service-map.py renderers + drift gates (`check`)
|
||||||
|
#
|
||||||
|
# `service-map.py check` is the referee: every quadlet unit file and every
|
||||||
|
# root unit in the tree is claimed by exactly one entry, every repo file is
|
||||||
|
# owned by a service or listed in `unowned:` with a reason, every seam exists,
|
||||||
|
# the rendered Pi-hole hosts file equals the render, and Prometheus' scrape
|
||||||
|
# targets plus the homepage probes equal the `scrape:` / `probe:` facts.
|
||||||
|
#
|
||||||
|
# --- fields ------------------------------------------------------------------
|
||||||
|
# name the deployer's service word == the systemd unit name for
|
||||||
|
# quadlet / root-unit / vm-quadlet kinds
|
||||||
|
# hosts targets that accept it (the deploy-remote.sh allow-list)
|
||||||
|
# kind quadlet rootless user unit from a .container file
|
||||||
|
# root-unit systemd system unit symlinked out of scripts/
|
||||||
|
# vm-quadlet quadlet inside the FrankBook-Pro podman machine
|
||||||
|
# host a host-level script/timer, no container
|
||||||
|
# deployed_by this service deploys as part of another deployable's handler
|
||||||
|
# (the trio/pair shape). It has no dirs/files of its own; the
|
||||||
|
# owner names the deployable whose handler installs it.
|
||||||
|
# unit_dir repo dir holding <unit>.container — a scalar, or a per-host map
|
||||||
|
# when one service carries its unit on two hosts (node-exporter)
|
||||||
|
# units container names this deployable owns (netns coupling is derived
|
||||||
|
# from the `Network=` lines in those files, never declared here)
|
||||||
|
# dirs repo dirs whose every file selects this service
|
||||||
|
# files exact repo files that select this service (root-unit trios,
|
||||||
|
# vendored templates, host scripts installed by this handler)
|
||||||
|
# retires files whose DELETION selects this service (the unit the
|
||||||
|
# handler must remove before it can claim the ports)
|
||||||
|
# only Dankserver's media stack still has one)
|
||||||
|
# health the deployer's readiness URL; `wait` its 2 s poll budget
|
||||||
|
# scrape Prometheus jobs: {job, target, metrics_path?, labels?}
|
||||||
|
# dns names this service answers at (rendered into the Pi-hole
|
||||||
|
# addn-hosts; every vhost resolves at the Dankclaw Caddy edge)
|
||||||
|
# probe host:port the homepage checker dials (per-host map if on 2 hosts)
|
||||||
|
# tests the Bats seams a change to this service must re-run
|
||||||
|
# =============================================================================
|
||||||
|
|
||||||
|
hosts:
|
||||||
|
dankclaw:
|
||||||
|
address: 192.168.0.143
|
||||||
|
# Caddy is consolidated on Dankclaw: it fronts every service on every
|
||||||
|
# host, so every DNS name in the fleet resolves here.
|
||||||
|
dns_edge: 192.168.0.143
|
||||||
|
dankserver:
|
||||||
|
address: 192.168.0.235
|
||||||
|
frankbook-pro:
|
||||||
|
address: 192.168.0.112
|
||||||
|
|
||||||
|
# Repo files no service owns. Each entry is a fact about the tree, not a
|
||||||
|
# fallback: the census gate fails on anything unowned that is NOT listed, so
|
||||||
|
# retirement work shrinks this list and a new stray file cannot slip in.
|
||||||
|
unowned:
|
||||||
|
shared/services.yml: the inventory this tool reads
|
||||||
|
scripts: repo tooling (build, fetch, bench, deploy, test); the deployers'
|
||||||
|
files are claimed by name in the service entries
|
||||||
|
autoresearch.sh: bench scratch at the repo root (retirement pending)
|
||||||
|
echo_prompt.txt: bench fixture at the repo root (retirement pending)
|
||||||
|
echo_request.json: bench fixture at the repo root (retirement pending)
|
||||||
|
prose_prompt.txt: bench fixture at the repo root (retirement pending)
|
||||||
|
prose_request.json: bench fixture at the repo root (retirement pending)
|
||||||
|
dankclaw/apply_qwen21_patch.py: retired patch script (its target runtime is gone)
|
||||||
|
dankclaw/comfyui-start.sh: superseded by the comfyui quadlet + Containerfile
|
||||||
|
dankclaw/udev: host udev rules, applied by manual sudo install + udevadm trigger
|
||||||
|
tests: the seams themselves
|
||||||
|
report: frozen point-in-time audit reports
|
||||||
|
old: retired scripts, reference only
|
||||||
|
.githooks: the commit hooks that run the seams
|
||||||
|
|
||||||
|
services:
|
||||||
|
# --- Search stack: one netns family, gluetun owns it ---------------------
|
||||||
|
- name: gluetun
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/searxng/quadlet
|
||||||
|
units: [gluetun]
|
||||||
|
tests: [tests/searxng_quadlet.bats]
|
||||||
|
# The tunnel's WireGuard bind — the homepage card for the tunnel itself.
|
||||||
|
probe: 127.0.0.1:51820
|
||||||
|
|
||||||
|
- name: searxng-valkey
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/searxng/quadlet
|
||||||
|
units: [searxng-valkey]
|
||||||
|
# An app-level dependency, not a network one: the deploy handler
|
||||||
|
# restarts SearXNG after its cache, so the cache's files select both.
|
||||||
|
dependents: [searxng]
|
||||||
|
tests: [tests/searxng_quadlet.bats]
|
||||||
|
probe: 127.0.0.1:6379
|
||||||
|
|
||||||
|
- name: searxng
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/searxng/quadlet
|
||||||
|
units: [searxng]
|
||||||
|
dirs: [shared/searxng]
|
||||||
|
tests: [tests/searxng_limiter.bats, tests/searxng_quadlet.bats]
|
||||||
|
scrape:
|
||||||
|
- job: services
|
||||||
|
target: 192.168.0.143:8081
|
||||||
|
metrics_path: /
|
||||||
|
dns: [search.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:8081
|
||||||
|
|
||||||
|
- name: marreta
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/marreta/quadlet
|
||||||
|
units: [marreta, marreta-browser]
|
||||||
|
tests: [tests/marreta_quadlet.bats]
|
||||||
|
dns: [marreta.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:8082
|
||||||
|
|
||||||
|
# --- DNS ------------------------------------------------------------------
|
||||||
|
- name: pihole
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/pihole/quadlet
|
||||||
|
units: [pihole]
|
||||||
|
dirs: [dankclaw/pihole]
|
||||||
|
tests: [tests/pihole.bats, tests/docs_completeness.bats]
|
||||||
|
dns: [pihole.franksdank.cloud]
|
||||||
|
probe: 192.168.0.143:8083
|
||||||
|
|
||||||
|
# --- Chat front end -------------------------------------------------------
|
||||||
|
- name: open-webui
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/open-webui/quadlet
|
||||||
|
units: [open-webui]
|
||||||
|
tests: [tests/open_webui_quadlet.bats, tests/caddyfile.bats]
|
||||||
|
scrape:
|
||||||
|
- job: services
|
||||||
|
target: 192.168.0.143:3005
|
||||||
|
metrics_path: /
|
||||||
|
dns: [ai.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:3005
|
||||||
|
|
||||||
|
# --- Git hosting ----------------------------------------------------------
|
||||||
|
- name: gitea-postgres
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/gitea/quadlet
|
||||||
|
units: [gitea-postgres]
|
||||||
|
deployed_by: gitea-server
|
||||||
|
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
|
||||||
|
probe: 127.0.0.1:5432
|
||||||
|
|
||||||
|
- name: gitea-server
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/gitea/quadlet
|
||||||
|
units: [gitea-server]
|
||||||
|
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: gitea
|
||||||
|
target: 192.168.0.143:3000
|
||||||
|
metrics_path: /metrics
|
||||||
|
dns: [gitea.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:3000
|
||||||
|
|
||||||
|
# --- Image generation -----------------------------------------------------
|
||||||
|
- name: comfyui
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/comfyui/quadlet
|
||||||
|
units: [comfyui]
|
||||||
|
dirs: [dankclaw/comfyui]
|
||||||
|
files: [dankclaw/Containerfile.comfyui]
|
||||||
|
health: http://127.0.0.1:8188/
|
||||||
|
wait: 300
|
||||||
|
tests: [tests/comfyui_rocm.bats, tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: comfyui
|
||||||
|
target: comfyui:8188
|
||||||
|
metrics_path: /
|
||||||
|
dns: [comfyui.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:8188
|
||||||
|
|
||||||
|
# --- Requests -------------------------------------------------------------
|
||||||
|
- name: seerr
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/seerr/quadlet
|
||||||
|
units: [seerr]
|
||||||
|
tests: [tests/seerr_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [seerr.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:5055
|
||||||
|
|
||||||
|
# --- Agent stack: one handler, four containers ----------------------------
|
||||||
|
- name: hermes
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/hermes/quadlet
|
||||||
|
units: [hermes, hermes-dashboard, hermes-webui, chrome-devtools-protocol]
|
||||||
|
dirs: [dankclaw/hermes]
|
||||||
|
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||||
|
dns: [hermes.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:8787
|
||||||
|
|
||||||
|
- name: hermes-dashboard
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
deployed_by: hermes
|
||||||
|
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||||
|
dns: [hermesadmin.franksdank.cloud]
|
||||||
|
probe: 192.168.0.143:9119
|
||||||
|
|
||||||
|
- name: hermes-webui
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
deployed_by: hermes
|
||||||
|
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||||
|
|
||||||
|
# --- Observability --------------------------------------------------------
|
||||||
|
- name: prometheus
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/observability/quadlet
|
||||||
|
units: [prometheus]
|
||||||
|
dirs: [shared/prometheus]
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: prometheus
|
||||||
|
target: localhost:9090
|
||||||
|
dns: [prometheus.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:9090
|
||||||
|
|
||||||
|
- name: grafana
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/observability/quadlet
|
||||||
|
units: [grafana]
|
||||||
|
dirs: [shared/grafana]
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: grafana
|
||||||
|
target: grafana:3000
|
||||||
|
metrics_path: /metrics
|
||||||
|
dns: [grafana.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:3002
|
||||||
|
|
||||||
|
- name: loki
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/observability/quadlet
|
||||||
|
units: [loki]
|
||||||
|
dirs: [dankclaw/loki, shared/loki]
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: loki
|
||||||
|
target: loki:3100
|
||||||
|
metrics_path: /metrics
|
||||||
|
dns: [loki.franksdank.cloud]
|
||||||
|
probe: 127.0.0.1:3100
|
||||||
|
|
||||||
|
- name: alloy
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/observability/quadlet
|
||||||
|
units: [alloy]
|
||||||
|
dirs: [dankclaw/alloy, shared/alloy]
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
probe: 127.0.0.1:12345
|
||||||
|
|
||||||
|
- name: node-exporter
|
||||||
|
# One service on two hosts: the same exporter, its own unit per host.
|
||||||
|
hosts: [dankclaw, dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir:
|
||||||
|
dankclaw: dankclaw/observability/quadlet
|
||||||
|
dankserver: dankserver/observability/quadlet
|
||||||
|
units: [node-exporter]
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: node-dankclaw
|
||||||
|
target: host.containers.internal:9100
|
||||||
|
- job: node-dankserver
|
||||||
|
target: 192.168.0.235:9100
|
||||||
|
probe: 127.0.0.1:9100
|
||||||
|
|
||||||
|
- name: alertmanager
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/observability/quadlet
|
||||||
|
units: [alertmanager]
|
||||||
|
# The Alertmanager config ships from Prometheus' directory, and the Signal
|
||||||
|
# relay is a HOST unit this handler installs: a change to either reaches
|
||||||
|
# the machine only through an alertmanager deploy.
|
||||||
|
files:
|
||||||
|
- shared/prometheus/alertmanager.yml
|
||||||
|
- scripts/signal-relay.py
|
||||||
|
- scripts/signal-relay.service
|
||||||
|
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/docs_completeness.bats]
|
||||||
|
probe: 127.0.0.1:9093
|
||||||
|
|
||||||
|
# --- Identity -------------------------------------------------------------
|
||||||
|
- name: authentik-postgres
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/authentik/quadlet
|
||||||
|
units: [authentik-postgres]
|
||||||
|
deployed_by: authentik-server
|
||||||
|
tests: [tests/authentik.bats]
|
||||||
|
|
||||||
|
- name: authentik-server
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/authentik/quadlet
|
||||||
|
units: [authentik-server]
|
||||||
|
tests: [tests/authentik.bats]
|
||||||
|
dns: [auth.franksdank.cloud]
|
||||||
|
|
||||||
|
- name: authentik-worker
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/authentik/quadlet
|
||||||
|
units: [authentik-worker]
|
||||||
|
deployed_by: authentik-server
|
||||||
|
tests: [tests/authentik.bats]
|
||||||
|
|
||||||
|
- name: authentik-outpost
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/authentik/quadlet
|
||||||
|
units: [authentik-outpost]
|
||||||
|
tests: [tests/authentik.bats]
|
||||||
|
|
||||||
|
- name: authentik-outpost-ldap
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/authentik/quadlet
|
||||||
|
units: [authentik-outpost-ldap]
|
||||||
|
tests: [tests/authentik.bats]
|
||||||
|
|
||||||
|
# --- The auth chain's edge, then the edge itself --------------------------
|
||||||
|
- name: anubis
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/anubis/quadlet
|
||||||
|
units: [anubis]
|
||||||
|
dirs: [dankclaw/anubis]
|
||||||
|
tests: [tests/anubis.bats, tests/caddyfile.bats, tests/homepage_caddy.bats]
|
||||||
|
|
||||||
|
- name: caddy
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankclaw/caddy/quadlet
|
||||||
|
units: [caddy]
|
||||||
|
dirs: [dankclaw/caddy]
|
||||||
|
files: [dankclaw/Caddyfile]
|
||||||
|
# Retiring the old split proxy selects Caddy: it must remove the installed
|
||||||
|
# unit and container before it can claim 80/443.
|
||||||
|
retires: [dankclaw/gitea/quadlet/gitea-caddy.container]
|
||||||
|
tests: [tests/caddy_quadlet.bats, tests/caddyfile.bats,
|
||||||
|
tests/homepage_caddy.bats, tests/docs_completeness.bats]
|
||||||
|
dns:
|
||||||
|
- franksdank.cloud
|
||||||
|
- www.franksdank.cloud
|
||||||
|
- dankclaw.franksdank.cloud
|
||||||
|
- dankserver.franksdank.cloud
|
||||||
|
probe: 127.0.0.1:80
|
||||||
|
|
||||||
|
- name: homepage
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: host
|
||||||
|
dirs: [shared/homepage]
|
||||||
|
files:
|
||||||
|
- scripts/homepage-status.py
|
||||||
|
- scripts/homepage-status.service
|
||||||
|
- scripts/homepage-status.timer
|
||||||
|
tests: [tests/homepage_caddy.bats, tests/homepage_config.bats,
|
||||||
|
tests/homepage_deploy.bats, tests/homepage_pages.bats,
|
||||||
|
tests/homepage_checker.bats]
|
||||||
|
|
||||||
|
# --- Dankserver media stack ----------------------------------------------
|
||||||
|
# The romm trio is one handler and one directory: every file in it selects
|
||||||
|
# all three, exactly as the deployer restarts all three.
|
||||||
|
- name: romm-mariadb
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/romm/quadlet
|
||||||
|
units: [romm-mariadb]
|
||||||
|
deployed_by: romm
|
||||||
|
dirs: [dankserver/romm]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
probe: 192.168.0.235:3306
|
||||||
|
|
||||||
|
- name: romm-redis
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/romm/quadlet
|
||||||
|
units: [romm-redis]
|
||||||
|
deployed_by: romm
|
||||||
|
dirs: [dankserver/romm]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
|
||||||
|
- name: romm
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/romm/quadlet
|
||||||
|
units: [romm]
|
||||||
|
dirs: [dankserver/romm]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [romm.franksdank.cloud]
|
||||||
|
probe: 192.168.0.235:8085
|
||||||
|
|
||||||
|
- name: copyparty
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/copyparty/quadlet
|
||||||
|
units: [copyparty]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [files.franksdank.cloud]
|
||||||
|
probe: 192.168.0.235:3923
|
||||||
|
|
||||||
|
- name: jellyfin
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/jellyfin/quadlet
|
||||||
|
units: [jellyfin]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [jelly.franksdank.cloud]
|
||||||
|
probe: 192.168.0.235:8096
|
||||||
|
|
||||||
|
- name: calibre-web
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/calibre-web/quadlet
|
||||||
|
units: [calibre-web]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [books.franksdank.cloud]
|
||||||
|
probe: 192.168.0.235:8083
|
||||||
|
|
||||||
|
- name: calibre-server
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/calibre-server/quadlet
|
||||||
|
units: [calibre-server]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
|
||||||
|
- name: plex
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/plex/quadlet
|
||||||
|
units: [plex]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [plex.franksdank.cloud, plex.franksdank.casa]
|
||||||
|
probe: 192.168.0.235:32400
|
||||||
|
|
||||||
|
- name: tautulli
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: quadlet
|
||||||
|
unit_dir: dankserver/tautulli/quadlet
|
||||||
|
units: [tautulli]
|
||||||
|
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
probe: 192.168.0.235:8181
|
||||||
|
|
||||||
|
# --- FrankBook-Pro: the *arr stack inside the podman machine --------------
|
||||||
|
# Every container rides gluetun's netns, and that coupling is DERIVED: each
|
||||||
|
# arr's unit file says `Network=gluetun.container`, so a gluetun unit change
|
||||||
|
# selects the whole stack with no declaration here.
|
||||||
|
- name: gluetun
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/gluetun/quadlet
|
||||||
|
units: [gluetun]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
|
||||||
|
- name: prowlarr
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/prowlarr/quadlet
|
||||||
|
units: [prowlarr]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [prowlarr.franksdank.cloud]
|
||||||
|
probe: 192.168.0.112:9696
|
||||||
|
|
||||||
|
- name: qbittorrent
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/qbittorrent/quadlet
|
||||||
|
units: [qbittorrent]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [qbittorrent.franksdank.cloud]
|
||||||
|
probe: 192.168.0.112:8080
|
||||||
|
|
||||||
|
- name: radarr
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/radarr/quadlet
|
||||||
|
units: [radarr]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [radarr.franksdank.cloud]
|
||||||
|
probe: 192.168.0.112:7878
|
||||||
|
|
||||||
|
- name: sonarr
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/sonarr/quadlet
|
||||||
|
units: [sonarr]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
probe: 192.168.0.112:8989
|
||||||
|
|
||||||
|
- name: lidarr
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/lidarr/quadlet
|
||||||
|
units: [lidarr]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [lidarr.franksdank.cloud]
|
||||||
|
probe: 192.168.0.112:8686
|
||||||
|
|
||||||
|
- name: readarr
|
||||||
|
hosts: [frankbook-pro]
|
||||||
|
kind: vm-quadlet
|
||||||
|
unit_dir: frankbook-pro/readarr/quadlet
|
||||||
|
units: [readarr]
|
||||||
|
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||||
|
dns: [readarr.franksdank.cloud]
|
||||||
|
probe: 192.168.0.112:8787
|
||||||
|
|
||||||
|
# --- The LLM fleet (last: never implied by a generic change) --------------
|
||||||
|
- name: frank-dragoon
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: root-unit
|
||||||
|
files:
|
||||||
|
- scripts/frank-dragoon.env
|
||||||
|
- scripts/frank-dragoon.service
|
||||||
|
- scripts/serve-frank-dragoon.sh
|
||||||
|
health: http://127.0.0.1:7770/health
|
||||||
|
wait: 1200
|
||||||
|
tests: [tests/frank_dragoon_unit.bats, tests/llm_units.bats,
|
||||||
|
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||||
|
tests/scripts_syntax.bats, tests/docs_completeness.bats]
|
||||||
|
scrape:
|
||||||
|
- job: frank-dragoon
|
||||||
|
target: host.containers.internal:7770
|
||||||
|
metrics_path: /metrics
|
||||||
|
labels: {service: frank-dragoon}
|
||||||
|
probe: 127.0.0.1:7770
|
||||||
|
|
||||||
|
- name: frank-zealot
|
||||||
|
hosts: [dankclaw]
|
||||||
|
kind: root-unit
|
||||||
|
files:
|
||||||
|
- scripts/frank-zealot.env
|
||||||
|
- scripts/frank-zealot.service
|
||||||
|
- scripts/serve-frank-zealot.sh
|
||||||
|
health: http://127.0.0.1:7771/health
|
||||||
|
wait: 900
|
||||||
|
tests: [tests/frank_zealot_unit.bats, tests/llm_units.bats,
|
||||||
|
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||||
|
tests/scripts_syntax.bats]
|
||||||
|
scrape:
|
||||||
|
- job: frank-zealot
|
||||||
|
target: host.containers.internal:7771
|
||||||
|
metrics_path: /metrics
|
||||||
|
labels: {service: frank-zealot}
|
||||||
|
probe: 127.0.0.1:7771
|
||||||
|
|
||||||
|
- name: frank-archon
|
||||||
|
hosts: [dankserver]
|
||||||
|
kind: root-unit
|
||||||
|
files:
|
||||||
|
- scripts/frank-archon.env
|
||||||
|
- scripts/frank-archon.service
|
||||||
|
- scripts/serve-frank-archon.sh
|
||||||
|
# The vendored chat template the engine renders with.
|
||||||
|
- scripts/qwen38-froggeric-v22.5.jinja
|
||||||
|
health: http://127.0.0.1:7770/health
|
||||||
|
wait: 900
|
||||||
|
tests: [tests/frank_archon_unit.bats, tests/llm_units.bats,
|
||||||
|
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||||
|
tests/scripts_syntax.bats]
|
||||||
|
scrape:
|
||||||
|
- job: frank-archon
|
||||||
|
target: 192.168.0.235:7770
|
||||||
|
metrics_path: /metrics
|
||||||
|
labels: {service: frank-archon}
|
||||||
|
probe: 192.168.0.235:7770
|
||||||
@@ -122,7 +122,7 @@ UNIT_DIR="$REPO_ROOT/dankclaw/authentik/quadlet"
|
|||||||
@test "the homepage vhost is www, guarded by Anubis" {
|
@test "the homepage vhost is www, guarded by Anubis" {
|
||||||
local block
|
local block
|
||||||
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDYFILE")
|
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDYFILE")
|
||||||
echo "$block" | grep -q 'reverse_proxy anubis:8923'
|
echo "$block" | grep -q 'import guarded'
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the naked domain redirects to www at the TLS edge" {
|
@test "the naked domain redirects to www at the TLS edge" {
|
||||||
|
|||||||
@@ -251,3 +251,14 @@ CADDYFILE="$REPO_ROOT/dankclaw/Caddyfile"
|
|||||||
grep -q -- '--server.http.listen-addr=0.0.0.0:12345' "$unit"
|
grep -q -- '--server.http.listen-addr=0.0.0.0:12345' "$unit"
|
||||||
grep -q '^PublishPort=127.0.0.1:12345:12345$' "$unit"
|
grep -q '^PublishPort=127.0.0.1:12345:12345$' "$unit"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "the guard chain is a named snippet the guarded vhosts import" {
|
||||||
|
# Rank 6: the Anubis pre-auth chain (reverse_proxy anubis:8923 with the
|
||||||
|
# X-Real-Ip header_up) existed as nine verbatim copies. One snippet,
|
||||||
|
# per-vhost imports; a chain edit is a one-line change.
|
||||||
|
grep -q '^(guarded) {' "$CADDYFILE"
|
||||||
|
local imports
|
||||||
|
imports=$(grep -c $'^\timport guarded$' "$CADDYFILE")
|
||||||
|
[ "$imports" -ge 9 ]
|
||||||
|
[ "$(grep -c 'reverse_proxy anubis:8923' "$CADDYFILE")" -eq 1 ]
|
||||||
|
}
|
||||||
|
|||||||
@@ -13,11 +13,15 @@ load helpers
|
|||||||
[ "$status" -eq 0 ]
|
[ "$status" -eq 0 ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "changed-test wrapper maps a compose change to compose and environment tests" {
|
@test "changed-test wrapper maps a shared config change to its service seams" {
|
||||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankserver/compose.yaml
|
# The compose era is retired (rank 7): the former compose fixture here
|
||||||
|
# mapped dankserver/compose.yaml, a file that no longer exists. The
|
||||||
|
# equivalent contract now: a service config change maps to the
|
||||||
|
# service's seams and nothing unrelated.
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/searxng/settings.yml
|
||||||
[ "$status" -eq 0 ]
|
[ "$status" -eq 0 ]
|
||||||
[[ "$output" == *"tests/compose_validity.bats"* ]]
|
[[ "$output" == *"tests/searxng_limiter.bats"* ]]
|
||||||
[[ "$output" == *"tests/env_consistency.bats"* ]]
|
[[ "$output" == *"tests/searxng_quadlet.bats"* ]]
|
||||||
[[ "$output" != *"tests/homepage_pages.bats"* ]]
|
[[ "$output" != *"tests/homepage_pages.bats"* ]]
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -47,3 +51,27 @@ load helpers
|
|||||||
[[ "$output" != *"tests/frank_medium_unit.bats"* ]]
|
[[ "$output" != *"tests/frank_medium_unit.bats"* ]]
|
||||||
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
|
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "changed-test wrapper maps the frankbook unit to the frankbook and deploy-flow seams" {
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths frankbook-pro/sonarr/quadlet/sonarr.container
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/frankbook_quadlet.bats"* ]]
|
||||||
|
[[ "$output" == *"tests/deployment_flow.bats"* ]]
|
||||||
|
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
|
||||||
|
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "changed-test wrapper maps the inventory itself to its seam" {
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/services.yml
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/service_inventory.bats"* ]]
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/service-map.py
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/service_inventory.bats"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "changed-test wrapper refuses an unmapped path" {
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths newdir/file.txt
|
||||||
|
[ "$status" -eq 2 ]
|
||||||
|
[[ "$output" == *"no focused-test mapping"* ]]
|
||||||
|
}
|
||||||
|
|||||||
Executable
+129
@@ -0,0 +1,129 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# deploy_sandbox.bash — the deploy-handler seam's fixture builder.
|
||||||
|
#
|
||||||
|
# The seam (tests/llm_deploy_gates.bats) EXECUTES scripts/deploy-remote.sh's
|
||||||
|
# LLM handlers with every host touchpoint stubbed (tests/stubs/) and every
|
||||||
|
# artifact pointed at a fixture, so the assertions are the handler's exit
|
||||||
|
# status, its refusal text, and what it ASKED systemd/podman to do.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# source tests/deploy_sandbox.bash
|
||||||
|
# sandbox_setup
|
||||||
|
# run_handler dankserver frank-archon
|
||||||
|
# sandbox_teardown
|
||||||
|
#
|
||||||
|
# After sandbox_setup the caller owns:
|
||||||
|
# $STAGE the temp root (payload/, bin/, systemd/, state/)
|
||||||
|
# $REMOTE_DIR the sandbox checkout (a git clone; the handler
|
||||||
|
# cds into it, so relative paths resolve there)
|
||||||
|
# $STAGE/systemctl.log every systemctl invocation
|
||||||
|
# $STAGE/podman.log every podman invocation
|
||||||
|
# $STAGE/systemd/<unit>.service the unit link a green deploy leaves behind
|
||||||
|
# Env the caller sets per case: STUB_IMAGE ("" = empty storage),
|
||||||
|
# STUB_MAIN_PID (0 = a stray process owns the port), the LLM_* overrides.
|
||||||
|
|
||||||
|
sandbox_setup() {
|
||||||
|
STAGE="$(mktemp -d)"
|
||||||
|
export HOME="$STAGE/home"
|
||||||
|
mkdir -p "$HOME" "$STAGE/state" "$STAGE/systemd" "$STAGE/bin"
|
||||||
|
|
||||||
|
# Host touchpoints: the seam's stubs, on PATH under their real names.
|
||||||
|
local stub repo_root
|
||||||
|
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
for stub in "$repo_root"/tests/stubs/*; do
|
||||||
|
ln -s "$stub" "$STAGE/bin/$(basename "$stub")"
|
||||||
|
done
|
||||||
|
export PATH="$STAGE/bin:$PATH"
|
||||||
|
export STUB_SYSTEMCTL_LOG="$STAGE/systemctl.log"
|
||||||
|
export STUB_PODMAN_LOG="$STAGE/podman.log"
|
||||||
|
# A deploy marks state; a sandbox with no git identity would make the
|
||||||
|
# identity, not the handler, the thing under test.
|
||||||
|
export GIT_AUTHOR_NAME=seam GIT_AUTHOR_EMAIL=seam@localhost
|
||||||
|
export GIT_COMMITTER_NAME=seam GIT_COMMITTER_EMAIL=seam@localhost
|
||||||
|
export SYSTEMD_SYSTEM_DIR="$STAGE/systemd"
|
||||||
|
export DEPLOY_STATE_DIR="$STAGE/state"
|
||||||
|
# run_privileged pipes the password from FRANK_SUDO_PW into `sudo -S --`;
|
||||||
|
# the stub drops it and execs, so no privilege gate hides a handler bug.
|
||||||
|
export FRANK_SUDO_PW=test
|
||||||
|
|
||||||
|
# Fixture payload: 3 MiB where a size floor applies, 10 bytes elsewhere.
|
||||||
|
mkdir -p "$STAGE/models/tokenizer"
|
||||||
|
truncate -s 3145728 "$STAGE/models/payload.rad"
|
||||||
|
truncate -s 10 "$STAGE/models/ck.hgn"
|
||||||
|
truncate -s 10 "$STAGE/models/ngram.hgn"
|
||||||
|
truncate -s 10 "$STAGE/models/vision.hgn"
|
||||||
|
printf 'x\n' > "$STAGE/models/llama-server"
|
||||||
|
chmod +x "$STAGE/models/llama-server"
|
||||||
|
truncate -s 10 "$STAGE/models/model.gguf"
|
||||||
|
truncate -s 10 "$STAGE/models/mmproj.gguf"
|
||||||
|
local tf
|
||||||
|
for tf in tokenizer.json chat_template.jinja tokenizer_config.json \
|
||||||
|
merges.txt vocab.json generation_config.json; do
|
||||||
|
printf 'x\n' > "$STAGE/models/tokenizer/$tf"
|
||||||
|
done
|
||||||
|
|
||||||
|
# $HOME/serverAppConfigs is the checkout the handler works in: a git
|
||||||
|
# WORKTREE of the caller's checkout, so the handler runs the EXACT files
|
||||||
|
# under test, and it is a real git repo: the deployer refuses a revision
|
||||||
|
# it cannot resolve. The handler cds into REMOTE_DIR, so every relative
|
||||||
|
# path it uses resolves there.
|
||||||
|
#
|
||||||
|
# HEAD is committed state, so the seam asserts on what is in history —
|
||||||
|
# the same rule as the whole suite, and the reason an uncommitted
|
||||||
|
# deployer change is invisible here until it is committed.
|
||||||
|
git -C "$repo_root" worktree add -q --detach "$HOME/serverAppConfigs" HEAD
|
||||||
|
export REMOTE_DIR="$HOME/serverAppConfigs"
|
||||||
|
|
||||||
|
# Every endpoint variable is env-overridable, so the handlers see the
|
||||||
|
# fixtures and never the real /home/dank paths.
|
||||||
|
export STUB_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:deadbeef
|
||||||
|
export LLM_DRAGOON_IMAGE="$STUB_IMAGE"
|
||||||
|
export LLM_DRAGOON_CKPT="$STAGE/models/ck.hgn" LLM_DRAGOON_CKPT_BYTES=10
|
||||||
|
export LLM_DRAGOON_NGRAM="$STAGE/models/ngram.hgn" LLM_DRAGOON_NGRAM_BYTES=10
|
||||||
|
export LLM_DRAGOON_VISION="$STAGE/models/vision.hgn" LLM_DRAGOON_VISION_BYTES=10
|
||||||
|
export LLM_DRAGOON_TOK_DIR="$STAGE/models/tokenizer"
|
||||||
|
export LLM_ZEALOT_RUNTIME="$STAGE/models/llama-server"
|
||||||
|
export LLM_ZEALOT_MODEL_FILE="$STAGE/models/model.gguf"
|
||||||
|
export LLM_ZEALOT_MMPROJ="$STAGE/models/mmproj.gguf"
|
||||||
|
export LLM_ARCHON_ENV_FILE="$REMOTE_DIR/scripts/frank-archon.env"
|
||||||
|
# The archon handler reads the served image out of the unit's own env
|
||||||
|
# file, so the stub's storage must hold exactly that ref.
|
||||||
|
ARCHON_IMAGE="$(sed -nE 's/^FRANK_ARCHON_IMAGE=(.+)$/\1/p' "$LLM_ARCHON_ENV_FILE")"
|
||||||
|
}
|
||||||
|
|
||||||
|
sandbox_teardown() {
|
||||||
|
[[ -z "${STAGE:-}" ]] && return 0
|
||||||
|
# A worktree leaves a registration in the caller's .git/worktrees, so
|
||||||
|
# remove it through git, not just the directory.
|
||||||
|
[[ -d "$STAGE/home/serverAppConfigs" ]] &&
|
||||||
|
git -C "$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" \
|
||||||
|
worktree remove --force "$STAGE/home/serverAppConfigs"
|
||||||
|
rm -rf "$STAGE"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_handler() {
|
||||||
|
local target="$1" service="$2"
|
||||||
|
run bash "$REMOTE_DIR/scripts/deploy-remote.sh" \
|
||||||
|
"$target" "$(git -C "$REMOTE_DIR" rev-parse HEAD)" "$service"
|
||||||
|
}
|
||||||
|
|
||||||
|
# archon_payload_env — the unit's env file with the payload coordinates
|
||||||
|
# redirected into the sandbox (and the size floor optionally lowered, which
|
||||||
|
# is how a test builds the truncated-.rad case).
|
||||||
|
archon_payload_env() {
|
||||||
|
local re="s#^FRANK_ARCHON_MODEL_DIR=.*#FRANK_ARCHON_MODEL_DIR=$STAGE/models#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_MODEL_FILE=.*#FRANK_ARCHON_MODEL_FILE=payload.rad#"
|
||||||
|
if [[ -n "${1:-}" ]]; then
|
||||||
|
re+="; s#^FRANK_ARCHON_RAD_MIN_MIB=.*#FRANK_ARCHON_RAD_MIN_MIB=$1#"
|
||||||
|
fi
|
||||||
|
sed -E "$re" "$LLM_ARCHON_ENV_FILE" > "$STAGE/archon.env"
|
||||||
|
export LLM_ARCHON_ENV_FILE="$STAGE/archon.env"
|
||||||
|
}
|
||||||
|
|
||||||
|
restarts_requested() {
|
||||||
|
# grep -c prints 0 and exits 1 on no match; the count, not the status,
|
||||||
|
# is what the assertions read.
|
||||||
|
local n=0
|
||||||
|
[[ -f "$STAGE/systemctl.log" ]] && n="$(grep -c 'restart ' "$STAGE/systemctl.log")"
|
||||||
|
printf '%s\n' "$n"
|
||||||
|
}
|
||||||
@@ -78,7 +78,10 @@ load helpers
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "remote runner health-checks an inference endpoint" {
|
@test "remote runner health-checks an inference endpoint" {
|
||||||
grep -q '127.0.0.1:7770/health' "$REPO_ROOT/scripts/deploy-remote.sh"
|
# start_root_unit carries the health gate for all three LLM units; the
|
||||||
|
# contract is that each endpoint's start names its port, and 7770 is
|
||||||
|
# frank-dragoon's.
|
||||||
|
grep -q 'start_root_unit frank-dragoon 7770' "$REPO_ROOT/scripts/deploy-remote.sh"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "deploy.sh pulls the merged checkout on the target" {
|
@test "deploy.sh pulls the merged checkout on the target" {
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
#!/usr/bin/env bats
|
#!/usr/bin/env bats
|
||||||
# Seam: documentation completeness.
|
# Seam: documentation completeness.
|
||||||
|
# (compose era retired 2026-10-13; the sweep below now skips trash so a
|
||||||
|
# revived compose file stays covered without resurrecting retired services)
|
||||||
# Every service defined in a compose.yaml must have a matching documentation
|
# Every service defined in a compose.yaml must have a matching documentation
|
||||||
# page in serverAppConfigs-docs/ and must be reachable from the index. This is the "docs first"
|
# page in serverAppConfigs-docs/ and must be reachable from the index. This is the "docs first"
|
||||||
# contract of this repo: no service ships without documentation.
|
# contract of this repo: no service ships without documentation.
|
||||||
@@ -18,7 +20,7 @@ load helpers
|
|||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
done < <(services_in "$f")
|
done < <(services_in "$f")
|
||||||
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
|
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' -not -path '*/.AGENTTRASH/*' | sort)
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "docs index links every docs page" {
|
@test "docs index links every docs page" {
|
||||||
|
|||||||
@@ -1,33 +1,12 @@
|
|||||||
#!/usr/bin/env bats
|
#!/usr/bin/env bats
|
||||||
# Seam: environment variable consistency and secret hygiene.
|
# Seam: environment variable consistency and secret hygiene.
|
||||||
# Every ${VAR} referenced in a compose.yaml or Caddyfile must be defined in the
|
# Every ${VAR} referenced in a Caddyfile must be defined in the matching
|
||||||
# matching .env.example. .env.example files must contain only placeholders, and
|
# .env.example. .env.example files must contain only placeholders, and
|
||||||
# the live gitea database password must never appear in tracked files.
|
# the live gitea database password must never appear in tracked files.
|
||||||
|
# (The compose.yaml sweeps retired with the last compose file on 2026-10-13.)
|
||||||
|
|
||||||
load helpers
|
load helpers
|
||||||
|
|
||||||
@test "every compose.yaml has a sibling .env.example" {
|
|
||||||
while IFS= read -r f; do
|
|
||||||
local envfile
|
|
||||||
envfile="$(dirname "$f")/.env.example"
|
|
||||||
[ -f "$envfile" ] || { echo "missing $envfile for $f"; return 1; }
|
|
||||||
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
|
|
||||||
}
|
|
||||||
|
|
||||||
@test "every \${VAR} in compose.yaml is defined in the sibling .env.example" {
|
|
||||||
while IFS= read -r f; do
|
|
||||||
local envfile
|
|
||||||
envfile="$(dirname "$f")/.env.example"
|
|
||||||
while IFS= read -r var; do
|
|
||||||
[ -z "$var" ] && continue
|
|
||||||
if ! env_vars_defined "$envfile" | grep -qx "$var"; then
|
|
||||||
echo "$f references \${$var} but $envfile does not define it"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
done < <(env_vars_in "$f")
|
|
||||||
done < <(find "$REPO_ROOT" -name 'compose.yaml' -not -path '*/.git/*' | sort)
|
|
||||||
}
|
|
||||||
|
|
||||||
@test ".env.example files contain only placeholder values" {
|
@test ".env.example files contain only placeholder values" {
|
||||||
while IFS= read -r f; do
|
while IFS= read -r f; do
|
||||||
has_placeholder_secret "$f"
|
has_placeholder_secret "$f"
|
||||||
|
|||||||
+1773
File diff suppressed because it is too large
Load Diff
+143
-61
@@ -13,13 +13,17 @@
|
|||||||
# the serve script ENFORCES it, the served window is what models.yml must
|
# the serve script ENFORCES it, the served window is what models.yml must
|
||||||
# quote, the host-networking contract (no --publish), the image gate on a
|
# quote, the host-networking contract (no --publish), the image gate on a
|
||||||
# name@sha256 ref, the .rad payload gate with its size floor, the
|
# name@sha256 ref, the .rad payload gate with its size floor, the
|
||||||
# chat-template override mount, the self-healing law (the run returns before
|
# chat-template override mount, the self-healing contract (`podman run`
|
||||||
# the engine is up, so the serve script must wait on the container for
|
# returns before the engine is up, so the serve script must stay attached
|
||||||
# Restart=always to mean anything), the deployer's false-green MainPID check,
|
# for Restart=always to mean anything), the deployer's false-green MainPID
|
||||||
# the vendored hfget fetch path (no token for this ungated repo), and the
|
# check, the vendored hfget fetch path (no token for this ungated repo), and the
|
||||||
# client contract (reasoning auto -> reasoning_content; models.yml maps the
|
# client contract (reasoning auto -> reasoning_content; models.yml maps the
|
||||||
# field on the harness side, not here).
|
# field on the harness side, not here).
|
||||||
|
# The serve script is EXECUTED against fixtures (stub podman, stub device
|
||||||
|
# nodes, stub ttm sysfs knobs, a temp payload dir) so the gates are observed
|
||||||
|
# as the operator observes them: exit status and the words on stderr. The
|
||||||
|
# knobs that make that possible (FRANK_ARCHON_DEV_KFD, _DEV_DRI,
|
||||||
|
# _TTM_SYSFS_DIR, _LOG) default to the production paths.
|
||||||
load helpers
|
load helpers
|
||||||
|
|
||||||
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
||||||
@@ -30,6 +34,133 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
|
|||||||
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
||||||
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
||||||
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
||||||
|
#
|
||||||
|
# Fixtures: a full env derived from the SHIPPED frank-archon.env with only
|
||||||
|
# the host paths redirected, so a payload rename moves the env, not this file.
|
||||||
|
setup() {
|
||||||
|
STAGE="$(mktemp -d)"
|
||||||
|
MODELS="$STAGE/models"
|
||||||
|
STATE="$STAGE/state"
|
||||||
|
TTM="$STAGE/ttm"
|
||||||
|
mkdir -p "$MODELS" "$STATE" "$TTM" "$STAGE/bin" "$STAGE/dri"
|
||||||
|
touch "$STAGE/kfd" "$STAGE/template.jinja"
|
||||||
|
# A 3 MiB .rad clears the 2 MiB floor the fixture env pins.
|
||||||
|
dd if=/dev/zero of="$MODELS/payload.rad" bs=1048576 count=3 status=none
|
||||||
|
# 200 GiB of pages, the shipped cap's value, in 4 KiB pages.
|
||||||
|
printf '52428800\n' > "$TTM/pages_limit"
|
||||||
|
cat > "$STAGE/bin/podman" <<'STUB'
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
case "$1" in
|
||||||
|
image)
|
||||||
|
case " ${STUB_IMAGE:-} " in
|
||||||
|
*" $3 "*) exit 0 ;;
|
||||||
|
*) exit 1 ;;
|
||||||
|
esac ;;
|
||||||
|
rm) exit 0 ;;
|
||||||
|
run) echo "PODMAN $*" >> "${STUB_ARGV:-/dev/null}"; exit 0 ;;
|
||||||
|
*) exit 0 ;;
|
||||||
|
esac
|
||||||
|
STUB
|
||||||
|
chmod +x "$STAGE/bin/podman"
|
||||||
|
IMAGE="$(sed -nE 's/^FRANK_ARCHON_IMAGE=(.+)$/\1/p' "$ENV_FILE")"
|
||||||
|
# The fixture env IS the shipped env with only the host paths and the
|
||||||
|
# size floor redirected, so a payload rename moves frank-archon.env and
|
||||||
|
# this file follows with it.
|
||||||
|
local re="s#^FRANK_ARCHON_MODEL_DIR=.*#FRANK_ARCHON_MODEL_DIR=$MODELS#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_MODEL_FILE=.*#FRANK_ARCHON_MODEL_FILE=payload.rad#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_RAD_MIN_MIB=.*#FRANK_ARCHON_RAD_MIN_MIB=2#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_TEMPLATE=.*#FRANK_ARCHON_TEMPLATE=$STAGE/template.jinja#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_STATE_DIR=.*#FRANK_ARCHON_STATE_DIR=$STATE#;"
|
||||||
|
re+=" s#^FRANK_ARCHON_TTM_MIN_MIB=.*#FRANK_ARCHON_TTM_MIN_MIB=150000#"
|
||||||
|
sed -E "$re" "$ENV_FILE" > "$STAGE/serve.env"
|
||||||
|
{
|
||||||
|
echo "FRANK_ARCHON_TTM_SYSFS=$TTM"
|
||||||
|
echo "FRANK_ARCHON_DEV_KFD=$STAGE/kfd"
|
||||||
|
echo "FRANK_ARCHON_DEV_DRI=$STAGE/dri"
|
||||||
|
echo "FRANK_ARCHON_LOG=$STAGE/serve.log"
|
||||||
|
echo "STUB_IMAGE=$IMAGE"
|
||||||
|
echo "STUB_ARGV=$STAGE/argv.txt"
|
||||||
|
} >> "$STAGE/serve.env"
|
||||||
|
}
|
||||||
|
|
||||||
|
teardown() {
|
||||||
|
rm -rf "$STAGE"
|
||||||
|
}
|
||||||
|
|
||||||
|
# serve_run — execute the serve script against the fixtures.
|
||||||
|
serve_run() {
|
||||||
|
run env PATH="$STAGE/bin:$PATH" FRANK_ARCHON_ENV_FILE="$STAGE/serve.env" \
|
||||||
|
bash "$SERVE"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses when the image is not in storage" {
|
||||||
|
# The env file wins over the ambient environment (the serve script
|
||||||
|
# sources it with `set -a`), so the stub's storage is emptied in the file.
|
||||||
|
sed -i 's#^STUB_IMAGE=.*#STUB_IMAGE=#' "$STAGE/serve.env"
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"not in podman storage"* ]]
|
||||||
|
[[ "$output" == *"podman pull"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses a missing .rad and names its builder" {
|
||||||
|
rm -f "$MODELS/payload.rad"
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"container: missing"*"$MODELS/payload.rad"* ]]
|
||||||
|
[[ "$output" == *"scripts/convert-swift15-w4dn8.sh"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses a .rad below the size floor, naming both sizes" {
|
||||||
|
# A truncated rad-convert is a guaranteed long-timeout boot; the floor is
|
||||||
|
# the fast failure (FRANK_ARCHON_RAD_MIN_MIB).
|
||||||
|
dd if=/dev/zero of="$MODELS/payload.rad" bs=1048576 count=1 status=none
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"1 MiB"* ]]
|
||||||
|
[[ "$output" == *"2 MiB"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses an absent chat template" {
|
||||||
|
# The engine is started with --override-chat-template; a missing file is
|
||||||
|
# a container that dies at argv parse, not a fast refusal.
|
||||||
|
rm -f "$STAGE/template.jinja"
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"template: missing"*"$STAGE/template.jinja"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses when the ttm pages cap is below the pinned ledger" {
|
||||||
|
printf '1048576\n' > "$TTM/pages_limit" # 4 GiB, under the ~147 GiB ledger
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"ttm pages cap is 4096 MiB"* ]]
|
||||||
|
[[ "$output" == *"150000"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script refuses when the GPU device nodes are absent" {
|
||||||
|
rm -f "$STAGE/kfd"
|
||||||
|
serve_run
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"GPU device nodes absent"*"$STAGE/kfd"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the serve script runs the engine with the .rad, the template and TP=2" {
|
||||||
|
serve_run
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
argv="$(cat "$STAGE/argv.txt")"
|
||||||
|
grep -q -- "--model /models/payload.rad" <<<"$argv"
|
||||||
|
grep -q -- "-v $MODELS:/models:ro" <<<"$argv"
|
||||||
|
grep -q -- "--override-chat-template /templates/template.jinja" <<<"$argv"
|
||||||
|
grep -q -- "--tp $(sed -nE 's/^FRANK_ARCHON_TP=([0-9]+)$/\1/p' "$ENV_FILE")" <<<"$argv"
|
||||||
|
grep -q -- "--tp-wire $(sed -nE 's/^FRANK_ARCHON_TP_WIRE=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
|
||||||
|
grep -q -- "--kv-cache-dtype $(sed -nE 's/^FRANK_ARCHON_KV_DTYPE=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
|
||||||
|
# The container name is the unit's ExecStop target, and the host-network
|
||||||
|
# contract publishes no port.
|
||||||
|
grep -q -- "--name $(sed -nE 's/^FRANK_ARCHON_CONTAINER=(.+)$/\1/p' "$ENV_FILE")" <<<"$argv"
|
||||||
|
grep -q -- "--network host" <<<"$argv"
|
||||||
|
! grep -q -- "--publish" <<<"$argv"
|
||||||
|
}
|
||||||
|
|
||||||
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
||||||
[ -f "$UNIT" ]
|
[ -f "$UNIT" ]
|
||||||
@@ -122,11 +253,11 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
[ "$ml" -ge 177356 ]
|
[ "$ml" -ge 177356 ]
|
||||||
# Concurrency: 2 slots x the served window is the capacity contract, and
|
# Concurrency: 2 slots x the served window is the capacity contract, and
|
||||||
# the fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
|
# the fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
|
||||||
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
# under the 25-row bound at which the wht6 wire stays bit-exact. A
|
||||||
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
# "this slot count cannot boot" refusal is per-runtime (compiler shape
|
||||||
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
# specialization rejects counts the engine running here accepts), so it
|
||||||
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
# must be re-tested, not inherited; measurements here put seqs 2 and 3
|
||||||
# tied inside the +-3% noise floor.
|
# inside each other's noise floor.
|
||||||
grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
||||||
@@ -134,8 +265,8 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
||||||
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
||||||
# KV pool moves the pool with it. KV is UNQUANTIZED bf16 (the engine takes
|
# KV pool moves the pool with it. KV is UNQUANTIZED bf16 (the engine takes
|
||||||
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table rides
|
||||||
# in host RAM per Frank's law.
|
# pinned host RAM per the placement knob.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
||||||
@@ -255,22 +386,6 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
! grep -q 'FRANK_ARCHON_TOOL_PARSER' "$ENV_FILE"
|
! grep -q 'FRANK_ARCHON_TOOL_PARSER' "$ENV_FILE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the serve script gates the image, the .rad payload, the template and the GPU nodes" {
|
|
||||||
# The image gate is the digest ref itself; the payload gate checks the
|
|
||||||
# .rad file and its size floor — a truncated convert would otherwise be
|
|
||||||
# a guaranteed long-timeout boot instead of a fast failure. The ttm
|
|
||||||
# preflight refuses to start under a pages cap below the pinned ledger.
|
|
||||||
grep -q 'FRANK_ARCHON_IMAGE' "$SERVE"
|
|
||||||
grep -q 'podman image exists' "$SERVE"
|
|
||||||
grep -q 'FRANK_ARCHON_MODEL_FILE' "$SERVE"
|
|
||||||
grep -q 'FRANK_ARCHON_RAD_MIN_MIB' "$SERVE"
|
|
||||||
grep -q 'FRANK_ARCHON_TEMPLATE' "$SERVE"
|
|
||||||
grep -q 'FRANK_ARCHON_TTM_MIN_MIB' "$SERVE"
|
|
||||||
grep -q 'pages_limit' "$SERVE"
|
|
||||||
grep -q 'pages_max' "$SERVE"
|
|
||||||
grep -q '/dev/kfd' "$SERVE"
|
|
||||||
grep -q -- '--device /dev/dri' "$SERVE"
|
|
||||||
}
|
|
||||||
|
|
||||||
@test "the serve script clears stale containers and self-heals via podman wait" {
|
@test "the serve script clears stale containers and self-heals via podman wait" {
|
||||||
# A crashed unit can leave the container name registered forever after;
|
# A crashed unit can leave the container name registered forever after;
|
||||||
@@ -304,36 +419,3 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
! grep -qE '^(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$ENV_FILE"
|
! grep -qE '^(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$ENV_FILE"
|
||||||
! grep -qE '^(export )?(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$SERVE"
|
! grep -qE '^(export )?(TEMP|HOST|GPU_MAX_HEAP_SIZE)=' "$SERVE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the deployer gates the image, payload and containers, then health-checks" {
|
|
||||||
# The deployer refuses when the image is absent from dank's rootless
|
|
||||||
# storage or the .rad is missing/too small — then clears stale
|
|
||||||
# containers (including the legacy identities), waits out the port,
|
|
||||||
# relinks/enables/restarts the root unit and health-gates it.
|
|
||||||
grep -q 'LLM_ARCHON_CONTAINER=frank-archon$' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'FRANK_ARCHON_IMAGE' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'podman image exists' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'FRANK_ARCHON_MODEL_FILE' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'FRANK_ARCHON_RAD_MIN_MIB' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'frank-archon-qwen' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'qwen-fp8' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'glm-probe' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'relink_unit frank-archon' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'systemctl enable frank-archon.service' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'systemctl restart frank-archon.service' "$REMOTE_DEPLOY"
|
|
||||||
# The health gate is a bare curl: the HTTP listener binds AFTER the
|
|
||||||
# full load (main.cpp — engine.start() then engine.serve()), so during
|
|
||||||
# load the port refuses connections and a bare curl is honest. The
|
|
||||||
# FreeToken-era body read ('status ok') is gone with that engine.
|
|
||||||
grep -q 'curl -fsS http://127.0.0.1:7770/health' "$REMOTE_DEPLOY"
|
|
||||||
! grep -q '"status":"ok"' "$REMOTE_DEPLOY"
|
|
||||||
}
|
|
||||||
|
|
||||||
@test "the deployer will not report green on a stray port-holder" {
|
|
||||||
# The 2026-09-26 false green: a leftover probe held :7770 and satisfied
|
|
||||||
# the health curl while the real unit was stopped. The deploy is only
|
|
||||||
# green when the unit's own MainPID is live behind the health answer.
|
|
||||||
grep -q 'systemctl show -p MainPID --value frank-archon.service' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'stray process holds the port' "$REMOTE_DEPLOY"
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -10,11 +10,11 @@
|
|||||||
# exact strings the engine reads: the image DIGEST, the checkpoint/n-gram/
|
# exact strings the engine reads: the image DIGEST, the checkpoint/n-gram/
|
||||||
# vision file names AND byte counts inside the /models mount, and the
|
# vision file names AND byte counts inside the /models mount, and the
|
||||||
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
|
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
|
||||||
# the container). A serve gate and a deployer gate that disagree is the
|
# the container). A serve gate and a deployer gate that disagree let a
|
||||||
# 2026-10-01 crash-loop; both sides are pinned here, plus the one engine
|
# deploy pass what the unit would refuse, so both sides are pinned here —
|
||||||
# law the card is emphatic about: HALOGEN_SPEC_ADAPT=0 — the default
|
# plus the one engine behavior the card is emphatic about:
|
||||||
# adaptive policy silently disables the MTP head per request and HALVES
|
# HALOGEN_SPEC_ADAPT=0 — the default adaptive policy silently disables the
|
||||||
# decode.
|
# MTP head per request and HALVES decode.
|
||||||
#
|
#
|
||||||
# Everything is tested from what an operator or supervisor can observe —
|
# Everything is tested from what an operator or supervisor can observe —
|
||||||
# the command line podman would run, the exit code, the deployer's error
|
# the command line podman would run, the exit code, the deployer's error
|
||||||
@@ -45,6 +45,13 @@ podman_stub() {
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
case "$1" in
|
case "$1" in
|
||||||
images) printf '%s\n' "${STUB_DIGESTS:-}" ;;
|
images) printf '%s\n' "${STUB_DIGESTS:-}" ;;
|
||||||
|
image)
|
||||||
|
# `podman image exists REF` — exit 1 when storage lacks the ref,
|
||||||
|
# which is exactly the primitive the shared gate library calls.
|
||||||
|
case " ${STUB_DIGESTS:-} " in
|
||||||
|
*" $3 "*) exit 0 ;;
|
||||||
|
*) exit 1 ;;
|
||||||
|
esac ;;
|
||||||
rm) exit 0 ;;
|
rm) exit 0 ;;
|
||||||
run|wait) echo "PODMAN $*"; exit "${STUB_WAIT_RC:-0}" ;;
|
run|wait) echo "PODMAN $*"; exit "${STUB_WAIT_RC:-0}" ;;
|
||||||
*) exit 0 ;;
|
*) exit 0 ;;
|
||||||
@@ -110,10 +117,11 @@ good_env() {
|
|||||||
|
|
||||||
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
|
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
|
||||||
# The card is emphatic: the default adaptive policy turns the MTP head
|
# The card is emphatic: the default adaptive policy turns the MTP head
|
||||||
# off for whole requests after low acceptance and HALVES decode. The
|
# off for whole requests after low acceptance and HALVES decode. A
|
||||||
# two-slot law does NOT carry over from the 27B engine — on this line
|
# "slots > 1 disables drafting" restriction belongs to the engine
|
||||||
# batching and speculation coexist (the card's quickstart ships 2
|
# generation that imposed it — on this line batching and speculation
|
||||||
# slots WITH the head), so 2 is concurrent capacity, not a tax.
|
# coexist (the card's quickstart ships 2 slots WITH the head), so 2 is
|
||||||
|
# concurrent capacity, not a tax.
|
||||||
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
|
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
|
||||||
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
|
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
|
||||||
# The cap/timeout pair is coupled (FLAGS.md table): a request at the
|
# The cap/timeout pair is coupled (FLAGS.md table): a request at the
|
||||||
@@ -179,7 +187,7 @@ good_env() {
|
|||||||
|
|
||||||
@test "the serve script refuses a checkpoint of the wrong byte count" {
|
@test "the serve script refuses a checkpoint of the wrong byte count" {
|
||||||
# A truncated curl exit-0 artifact must not boot; the Hub's byte
|
# A truncated curl exit-0 artifact must not boot; the Hub's byte
|
||||||
# count is the contract (fleet fetch law).
|
# count is the contract, any other size is a broken artifact.
|
||||||
podman_stub
|
podman_stub
|
||||||
good_env
|
good_env
|
||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
@@ -197,7 +205,7 @@ good_env() {
|
|||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
FRANK_DRAGOON_NGRAM_TABLE=absent.hgn "$SERVE"
|
FRANK_DRAGOON_NGRAM_TABLE=absent.hgn "$SERVE"
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
[[ "$output" == *"n-gram-table"*"missing"* ]]
|
[[ "$output" == *"frank-dragoon: n-gram: missing at $TEST_TMP/absent.hgn"* ]]
|
||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
FRANK_DRAGOON_NGRAM_BYTES=42 "$SERVE"
|
FRANK_DRAGOON_NGRAM_BYTES=42 "$SERVE"
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
@@ -213,7 +221,7 @@ good_env() {
|
|||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
FRANK_DRAGOON_VISION_TOWER=absent.hgn "$SERVE"
|
FRANK_DRAGOON_VISION_TOWER=absent.hgn "$SERVE"
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
[[ "$output" == *"vision-tower"*"missing"* ]]
|
[[ "$output" == *"frank-dragoon: vision: missing at $TEST_TMP/absent.hgn"* ]]
|
||||||
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
|
||||||
FRANK_DRAGOON_VISION_BYTES=42 "$SERVE"
|
FRANK_DRAGOON_VISION_BYTES=42 "$SERVE"
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
@@ -280,16 +288,16 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
|
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
@test "the shipped env keeps the harness fit margin inside the ceiling" {
|
||||||
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
||||||
# (a longer prompt is a 400 naming the limit, never a silent
|
# (a longer prompt is a 400 naming the limit, never a silent
|
||||||
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
||||||
# − 4096), compacts at 85 % of its declared 217088 window, and its
|
# − 4096), compacts at 85 % of its declared 217088 window, and its
|
||||||
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
|
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
|
||||||
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with
|
# traffic — so a harness window equal to the engine ceiling would make
|
||||||
# window == ceiling, every request over ~146k est was a guaranteed
|
# every long request a guaranteed 400 before compaction could fire.
|
||||||
# 400 before compaction could fire at 85 %). The 45k gap between the
|
# The 45k gap between the harness window and the engine ceiling must
|
||||||
# harness window and the engine ceiling must absorb that error at
|
# absorb that error at
|
||||||
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
|
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
|
||||||
# full budget at the compaction threshold: 1.03*184524 + 45056
|
# full budget at the compaction threshold: 1.03*184524 + 45056
|
||||||
# clamped budget at the clamp's zero point: 1.03*212992 + 0
|
# clamped budget at the clamp's zero point: 1.03*212992 + 0
|
||||||
@@ -328,11 +336,6 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER"
|
grep -qF "LLM_DRAGOON_CKPT_BYTES=\"\${LLM_DRAGOON_CKPT_BYTES:-$bytes}\"" "$DEPLOYER"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the deployer points a missing artifact at the flashnext fetch script" {
|
|
||||||
grep -qE 'download-frank-dragoon-flashnext\.sh' "$DEPLOYER"
|
|
||||||
grep -qE 'podman pull' "$DEPLOYER"
|
|
||||||
}
|
|
||||||
|
|
||||||
@test "the stray sweep cannot take frank-zealot or the deployer down" {
|
@test "the stray sweep cannot take frank-zealot or the deployer down" {
|
||||||
# frank-zealot also listens on this host (7771, llama-server) and the
|
# frank-zealot also listens on this host (7771, llama-server) and the
|
||||||
# deployer's own command line contains "frank-dragoon"; only the FULL
|
# deployer's own command line contains "frank-dragoon"; only the FULL
|
||||||
@@ -344,8 +347,9 @@ $(cat "$TEST_TMP/serve.log")"
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "the deployed unit keeps the fleet's restart and boot-persistence shape" {
|
@test "the deployed unit keeps the fleet's restart and boot-persistence shape" {
|
||||||
grep -qE '^Restart=(always|on-failure)$' "$UNIT"
|
# Restart=always and the WantedBy value are the FLEET parity seam
|
||||||
grep -qE '^WantedBy=' "$UNIT"
|
# (tests/llm_units.bats); what is endpoint-specific is which script the
|
||||||
|
# unit runs, under whom, and what a stop must reap.
|
||||||
grep -qE '^ExecStart=.*/serve-frank-dragoon\.sh$' "$UNIT"
|
grep -qE '^ExecStart=.*/serve-frank-dragoon\.sh$' "$UNIT"
|
||||||
grep -qE '^User=dank$' "$UNIT"
|
grep -qE '^User=dank$' "$UNIT"
|
||||||
grep -qE '^LimitMEMLOCK=infinity$' "$UNIT"
|
grep -qE '^LimitMEMLOCK=infinity$' "$UNIT"
|
||||||
|
|||||||
@@ -85,10 +85,12 @@ run_stage() {
|
|||||||
[ -f "$ENV_FILE" ]
|
[ -f "$ENV_FILE" ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "frank-zealot is boot-persistent, self-healing and unprivileged" {
|
@test "frank-zealot runs unprivileged with the memlock budget and its env file" {
|
||||||
grep -q '^WantedBy=multi-user.target$' "$UNIT"
|
# Restart policy, RestartSec and WantedBy are the FLEET parity seam
|
||||||
grep -q '^Restart=' "$UNIT"
|
# (tests/llm_units.bats). What is endpoint-specific: CUDA pins its
|
||||||
grep -q '^RestartSec=' "$UNIT"
|
# staging buffers, so the memlock budget is load-bearing even though the
|
||||||
|
# weights are copied into VRAM, and the unit must name the serve script
|
||||||
|
# and the env file that sizes it.
|
||||||
grep -q '^User=dank$' "$UNIT"
|
grep -q '^User=dank$' "$UNIT"
|
||||||
grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
|
grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
|
||||||
grep -q '^ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh$' "$UNIT"
|
grep -q '^ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-zealot.sh$' "$UNIT"
|
||||||
@@ -253,26 +255,20 @@ run_stage() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "the serve script refuses a missing runtime, GGUF or projector" {
|
@test "the serve script refuses a missing runtime, GGUF or projector" {
|
||||||
|
# The refusal is the shared gate library's: it names the artifact class,
|
||||||
|
# the exact path, and the script that produces it.
|
||||||
rm -f "$MODEL_DIR/$MM_STUB"
|
rm -f "$MODEL_DIR/$MM_STUB"
|
||||||
serve_argv
|
serve_argv
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
[[ "$output" == *"frank-zealot: missing"* ]]
|
[[ "$output" == *"frank-zealot:"* ]]
|
||||||
|
[[ "$output" == *"mmproj"*"missing"*"$MM_STUB"* ]]
|
||||||
|
[[ "$output" == *"download-frank-zealot.sh"* ]]
|
||||||
|
|
||||||
rm -f "$RUNTIME_DIR/bin/llama-server"
|
rm -f "$RUNTIME_DIR/bin/llama-server"
|
||||||
serve_argv
|
serve_argv
|
||||||
[ "$status" -ne 0 ]
|
[ "$status" -ne 0 ]
|
||||||
[[ "$output" == *"frank-zealot: missing"* ]]
|
[[ "$output" == *"runtime"*"missing"*"llama-server"* ]]
|
||||||
}
|
[[ "$output" == *"build-llama-master-cuda.sh"* ]]
|
||||||
|
|
||||||
@test "the deploy handler gates on binary, GGUF and projector, then MainPID-hardens health" {
|
|
||||||
grep -q 'deploy_llm_zealot' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'LLM_ZEALOT_RUNTIME' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'LLM_ZEALOT_MODEL_FILE' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'LLM_ZEALOT_MMPROJ' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'relink_unit frank-zealot' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'systemctl enable frank-zealot.service' "$REMOTE_DEPLOY"
|
|
||||||
grep -q '127.0.0.1:7771/health' "$REMOTE_DEPLOY"
|
|
||||||
grep -q 'frank-zealot.service has no MainPID' "$REMOTE_DEPLOY"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the two endpoints keep separate ports and the large handler never pkills llama-server" {
|
@test "the two endpoints keep separate ports and the large handler never pkills llama-server" {
|
||||||
|
|||||||
@@ -4,6 +4,12 @@
|
|||||||
|
|
||||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
|
||||||
|
# skip_if_no_podman — skip the test when podman is absent (the image gate
|
||||||
|
# asks real storage a real question)
|
||||||
|
skip_if_no_podman() {
|
||||||
|
command -v podman >/dev/null 2>&1 || skip "podman is not installed"
|
||||||
|
}
|
||||||
|
|
||||||
# yaml_parse_ok <file> — assert a compose file parses as valid YAML
|
# yaml_parse_ok <file> — assert a compose file parses as valid YAML
|
||||||
yaml_parse_ok() {
|
yaml_parse_ok() {
|
||||||
python3 - "$1" <<'PYEOF'
|
python3 - "$1" <<'PYEOF'
|
||||||
|
|||||||
@@ -101,7 +101,10 @@ webui_service_yaml() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "remote deployer accepts and dispatches hermes-webui" {
|
@test "remote deployer accepts and dispatches hermes-webui" {
|
||||||
grep -q 'dankclaw:hermes-webui' "$REMOTE_DEPLOY"
|
# Acceptance is the inventory (rank 3): the old literal allow-list in
|
||||||
|
# the remote runner is retired.
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
# Dispatched to the unit handler, never to compose: deploy_hermes installs
|
# Dispatched to the unit handler, never to compose: deploy_hermes installs
|
||||||
# the quadlets and retires compose-era containers.
|
# the quadlets and retires compose-era containers.
|
||||||
grep -q 'hermes|hermes-dashboard|hermes-webui) deploy_hermes' "$REMOTE_DEPLOY"
|
grep -q 'hermes|hermes-dashboard|hermes-webui) deploy_hermes' "$REMOTE_DEPLOY"
|
||||||
@@ -111,5 +114,7 @@ webui_service_yaml() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "top-level deployer accepts hermes-webui as a Dankclaw service" {
|
@test "top-level deployer accepts hermes-webui as a Dankclaw service" {
|
||||||
grep -q 'dankclaw:hermes-webui' "$DEPLOY"
|
# deploy.sh validates --service through the inventory's allows.
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ setup() {
|
|||||||
# domain never enters the chain (it redirects at the TLS edge first).
|
# domain never enters the chain (it redirects at the TLS edge first).
|
||||||
local block
|
local block
|
||||||
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDY")
|
block=$(awk '/^www\.franksdank\.cloud[[:space:]]*\{/,/^}/' "$CADDY")
|
||||||
echo "$block" | grep -q 'reverse_proxy anubis:8923'
|
echo "$block" | grep -q 'import guarded'
|
||||||
# forwards accepted traffic to; Host survives the hop, so per-vhost
|
# forwards accepted traffic to; Host survives the hop, so per-vhost
|
||||||
# routing resumes there — guarded vhosts proxy the whole request
|
# routing resumes there — guarded vhosts proxy the whole request
|
||||||
# through Authentik, which forwards authenticated traffic back on the
|
# through Authentik, which forwards authenticated traffic back on the
|
||||||
|
|||||||
Executable
+136
@@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: the deployer's LLM handlers are the twin of the serve scripts'
|
||||||
|
# artifact gates (the 2026-10-01 crash-loop: a green deploy against a
|
||||||
|
# missing payload left a restart-looping unit and a dark endpoint).
|
||||||
|
#
|
||||||
|
# These tests EXECUTE scripts/deploy-remote.sh's LLM handlers in a sandbox
|
||||||
|
# (tests/deploy_sandbox.bash): every host touchpoint is a stub, the artifacts
|
||||||
|
# are fixtures, and the assertions are exit status, the refusal on stderr,
|
||||||
|
# and what the handler asked systemd to do. A handler that started the unit
|
||||||
|
# before checking its artifacts would pass a text-grep seam and fail here.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
load deploy_sandbox
|
||||||
|
|
||||||
|
setup() { sandbox_setup; }
|
||||||
|
teardown() { sandbox_teardown; }
|
||||||
|
|
||||||
|
@test "the dragoon handler refuses when the engine image is not in storage" {
|
||||||
|
export STUB_IMAGE=
|
||||||
|
run_handler dankclaw frank-dragoon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-dragoon: image:"*"not in podman storage"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the dragoon handler refuses a truncated checkpoint, pointing at the fetch" {
|
||||||
|
truncate -s 9 "$STAGE/models/ck.hgn"
|
||||||
|
run_handler dankclaw frank-dragoon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"checkpoint: is 9 bytes, expected 10"* ]]
|
||||||
|
# The refusal is only useful if it names the remedy: the fetch script
|
||||||
|
# re-verifies byte counts and hashes.
|
||||||
|
[[ "$output" == *"download-frank-dragoon-flashnext.sh"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the dragoon handler refuses a symlinked tokenizer file" {
|
||||||
|
# Snapshot layouts link their members into place, and the link dangles
|
||||||
|
# inside the container mount — only regular files are servable.
|
||||||
|
rm "$STAGE/models/tokenizer/vocab.json"
|
||||||
|
ln -s /nowhere/vocab.json "$STAGE/models/tokenizer/vocab.json"
|
||||||
|
run_handler dankclaw frank-dragoon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-dragoon: tokenizer: $STAGE/models/tokenizer/vocab.json is missing or a symlink"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the dragoon handler starts the unit once every artifact gate passes" {
|
||||||
|
run_handler dankclaw frank-dragoon
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
grep -q 'restart frank-dragoon.service' "$STAGE/systemctl.log"
|
||||||
|
grep -qx 'rm -f frank-dragoon frank-dragoon-halogen' "$STAGE/podman.log"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the zealot handler refuses when the llama.cpp runtime is not executable" {
|
||||||
|
# The runtime is an operator-built binary; a tree that compiled but lost
|
||||||
|
# its exec bit must refuse exactly like a missing one.
|
||||||
|
rm "$STAGE/models/llama-server"
|
||||||
|
run_handler dankclaw frank-zealot
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-zealot: runtime: missing at $STAGE/models/llama-server"* ]]
|
||||||
|
[[ "$output" == *"cmake"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the zealot handler refuses a missing projector — a text-only endpoint is not deployable" {
|
||||||
|
# The projector IS the vision capability: deploying without it silently
|
||||||
|
# ships a text-only endpoint.
|
||||||
|
rm "$STAGE/models/mmproj.gguf"
|
||||||
|
run_handler dankclaw frank-zealot
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-zealot: mmproj: missing at $STAGE/models/mmproj.gguf"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the zealot handler starts the unit once every artifact gate passes" {
|
||||||
|
run_handler dankclaw frank-zealot
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
grep -q 'restart frank-zealot.service' "$STAGE/systemctl.log"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the archon handler refuses when the radiance image is not in storage" {
|
||||||
|
export STUB_IMAGE=
|
||||||
|
run_handler dankserver frank-archon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-archon: image:"*"not in podman storage"* ]]
|
||||||
|
[[ "$output" == *"podman pull"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the archon handler refuses a .rad below the env's size floor" {
|
||||||
|
truncate -s 1048576 "$STAGE/models/payload.rad" # 1 MiB, far under the floor
|
||||||
|
archon_payload_env
|
||||||
|
export STUB_IMAGE="$ARCHON_IMAGE"
|
||||||
|
run_handler dankserver frank-archon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-archon: .rad: is 1 MiB, expected >= "* ]]
|
||||||
|
[[ "$output" == *"convert-swift15-w4dn8.sh"* ]]
|
||||||
|
[ "$(restarts_requested)" = "0" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the archon handler clears the legacy container identities and relinks the unit" {
|
||||||
|
# The retired identities (the R9V container, the ad hoc probe, the glm
|
||||||
|
# experiment) hold the same GPUs, so a deploy that leaves one running
|
||||||
|
# leaves the new engine unable to bind. The unit link is the root-unit
|
||||||
|
# shape's whole install.
|
||||||
|
archon_payload_env 2
|
||||||
|
export STUB_IMAGE="$ARCHON_IMAGE"
|
||||||
|
run_handler dankserver frank-archon
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
grep -q 'restart frank-archon.service' "$STAGE/systemctl.log"
|
||||||
|
[ -L "$STAGE/systemd/frank-archon.service" ]
|
||||||
|
local ctr
|
||||||
|
for ctr in frank-archon frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe; do
|
||||||
|
grep -qx "rm -f $ctr" "$STAGE/podman.log"
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "no LLM handler reports green when its unit has no MainPID behind the port" {
|
||||||
|
# The 2026-09-26 false green: a leftover probe held :7770 and satisfied
|
||||||
|
# the health curl while the real unit was stopped.
|
||||||
|
archon_payload_env 2
|
||||||
|
export STUB_IMAGE="$ARCHON_IMAGE"
|
||||||
|
export STUB_MAIN_PID=0
|
||||||
|
run_handler dankserver frank-archon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"frank-archon.service has no MainPID"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "a refused LLM handler leaves its service pending in the deploy state" {
|
||||||
|
export STUB_IMAGE=
|
||||||
|
run_handler dankserver frank-archon
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
grep -q 'status=pending' "$STAGE/state/frank-archon"
|
||||||
|
[ ! -e "$STAGE/state/frank-zealot" ]
|
||||||
|
}
|
||||||
Executable
+172
@@ -0,0 +1,172 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: the shared artifact gates every LLM endpoint runs before it starts.
|
||||||
|
#
|
||||||
|
# The rule these encode: a serve script and the deployer must refuse the
|
||||||
|
# SAME missing/truncated artifact, and they must refuse it the same way.
|
||||||
|
# Two hand-written copies of the same gates drift apart until a deploy
|
||||||
|
# passes what the unit would refuse; the library is the one place the
|
||||||
|
# refusal lives, and both callers source it.
|
||||||
|
#
|
||||||
|
# Everything here is observable: arguments in, exit status and stderr out.
|
||||||
|
# Nothing inspects the library's internals.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
LIB="$REPO_ROOT/scripts/lib/llm-gates.sh"
|
||||||
|
|
||||||
|
setup() {
|
||||||
|
TMP="$(mktemp -d)"
|
||||||
|
# A fixture with a known, independent size: 12 bytes.
|
||||||
|
printf '0123456789ab' > "$TMP/artifact.bin"
|
||||||
|
mkdir -p "$TMP/flat"
|
||||||
|
touch "$TMP/flat/tokenizer.json" "$TMP/flat/config.json"
|
||||||
|
}
|
||||||
|
|
||||||
|
teardown() {
|
||||||
|
rm -rf "$TMP"
|
||||||
|
}
|
||||||
|
|
||||||
|
# call <invocation> — run one gate in a fresh shell under `set -e`, exactly
|
||||||
|
# as a serve script sources the library, and capture status + both streams.
|
||||||
|
call() {
|
||||||
|
run env GATE_PREFIX="${GATE_PREFIX:-testgate}" PATH="$PATH" \
|
||||||
|
bash -c "set -euo pipefail; source '$LIB'; $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "a present artifact passes the file gate with no output" {
|
||||||
|
GATE_PREFIX=testgate call "require_file '$TMP/artifact.bin' checkpoint 'fetch it'"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "an absent artifact fails the file gate naming the path and the remedy" {
|
||||||
|
call "require_file '$TMP/absent.bin' checkpoint 'run scripts/download-frank-archon-swift15.sh'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"checkpoint: missing"* ]]
|
||||||
|
[[ "$output" == *"$TMP/absent.bin"* ]]
|
||||||
|
[[ "$output" == *"run scripts/download-frank-archon-swift15.sh"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the file gate refuses a directory where an artifact is required" {
|
||||||
|
# The serve gate is `-f`, not `-e`: a directory named like the payload
|
||||||
|
# (an unpacked staging dir) must not boot the engine.
|
||||||
|
call "require_file '$TMP/flat' checkpoint 'fetch it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"checkpoint"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "a symlinked artifact is refused: a HF snapshot link dangles in the container mount" {
|
||||||
|
ln -s "$TMP/artifact.bin" "$TMP/linked.bin"
|
||||||
|
call "require_file '$TMP/linked.bin' checkpoint 'fetch it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"checkpoint"*"missing"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the byte gate accepts the exact contracted size" {
|
||||||
|
call "require_bytes '$TMP/artifact.bin' checkpoint 12 'fetch it'"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the byte gate refuses a size mismatch quoting both the real and the expected size" {
|
||||||
|
# The Hub's declared byte count is the contract; curl writes the error
|
||||||
|
# body and exits 0, so a truncation is invisible without this gate.
|
||||||
|
call "require_bytes '$TMP/artifact.bin' checkpoint 42 'run the fetch script'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"12 bytes"* ]]
|
||||||
|
[[ "$output" == *"expected 42"* ]]
|
||||||
|
[[ "$output" == *"run the fetch script"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the byte gate is an equality contract, not a floor" {
|
||||||
|
# A LARGER file is not a better artifact: it is a different artifact
|
||||||
|
# (the compacted r2 of a checkpoint is byte-exact, not "at least").
|
||||||
|
call "require_bytes '$TMP/artifact.bin' checkpoint 6 'fetch it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"expected 6"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the minimum-size gate accepts the threshold itself" {
|
||||||
|
dd if=/dev/zero of="$TMP/two-mib.rad" bs=1048576 count=2 status=none
|
||||||
|
call "require_min_mib '$TMP/two-mib.rad' container 2 'rebuild it'"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the minimum-size gate refuses below the threshold naming both sizes" {
|
||||||
|
dd if=/dev/zero of="$TMP/one-mib.rad" bs=1048576 count=1 status=none
|
||||||
|
call "require_min_mib '$TMP/one-mib.rad' container 2 'rebuild it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"1 MiB"* ]]
|
||||||
|
[[ "$output" == *"2 MiB"* ]]
|
||||||
|
[[ "$output" == *"rebuild it"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the flat-directory gate accepts a directory of regular files" {
|
||||||
|
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json config.json"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the flat-directory gate names the member that is absent" {
|
||||||
|
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json vocab.json"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"tokenizer"*"missing"* ]]
|
||||||
|
[[ "$output" == *"vocab.json"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the flat-directory gate refuses a symlinked member" {
|
||||||
|
# A HuggingFace cache snapshot's entries are symlinks into blobs/ and
|
||||||
|
# dangle inside the container mount even when the directory is real.
|
||||||
|
ln -s /nowhere/x "$TMP/flat/vocab.json"
|
||||||
|
call "require_dir_files '$TMP/flat' tokenizer 'fetch the tokenizer' tokenizer.json vocab.json"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"vocab.json"*"missing or a symlink"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the flat-directory gate refuses a missing directory" {
|
||||||
|
call "require_dir_files '$TMP/nope' tokenizer 'fetch the tokenizer' tokenizer.json"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"tokenizer: missing"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the flat-directory gate refuses a symlinked directory" {
|
||||||
|
# The snapshot directory itself can be the link (a HF cache blobs/
|
||||||
|
# sibling), and a link into the container mount dangles there too.
|
||||||
|
ln -s "$TMP/flat" "$TMP/linked-dir"
|
||||||
|
call "require_dir_files '$TMP/linked-dir' tokenizer 'fetch the tokenizer' tokenizer.json"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"tokenizer: missing"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the image gate refuses a ref the storage does not hold" {
|
||||||
|
# A name@sha256 ref exists only if THAT manifest was pulled, so the
|
||||||
|
# gate is the ref itself — one command, no grep race.
|
||||||
|
skip_if_no_podman
|
||||||
|
call "require_image 'example.invalid/nothing@sha256:0000000000000000000000000000000000000000000000000000000000000000' 'podman pull it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == *"not in podman storage"* ]]
|
||||||
|
[[ "$output" == *"podman pull it"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "every failure names the endpoint the operator is standing in front of" {
|
||||||
|
GATE_PREFIX=frank-dragoon call "require_file '$TMP/absent.bin' checkpoint 'fetch it'"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" == frank-dragoon:* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "a failing gate stops the caller from reaching the runtime" {
|
||||||
|
# The composition property that matters: the gate is not advisory. A
|
||||||
|
# serve script that calls it and then execs the engine must die first.
|
||||||
|
cat > "$TMP/serve-like.sh" <<EOF
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
source '$LIB'
|
||||||
|
require_file '$TMP/absent.bin' checkpoint 'fetch it'
|
||||||
|
echo RUNTIME-EXECUTED
|
||||||
|
EOF
|
||||||
|
run bash "$TMP/serve-like.sh"
|
||||||
|
[ "$status" -ne 0 ]
|
||||||
|
[[ "$output" != *"RUNTIME-EXECUTED"* ]]
|
||||||
|
[[ "$output" == *"checkpoint: missing"* ]]
|
||||||
|
}
|
||||||
@@ -80,3 +80,38 @@ unit_repo_path() {
|
|||||||
}
|
}
|
||||||
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
|
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "every LLM root unit is self-healing against a voluntary exit" {
|
||||||
|
# Restart=always, not on-failure. Every endpoint here is a container
|
||||||
|
# runner (podman run) or a server whose front end can exit 0 on a
|
||||||
|
# control-plane event; systemd treats exit 0 as a CLEAN stop and
|
||||||
|
# on-failure leaves the endpoint dark until the next boot, with the
|
||||||
|
# unit reporting success. No LLM unit may ship the weaker policy: a
|
||||||
|
# single endpoint on on-failure is a daemon that goes dark silently.
|
||||||
|
local unit
|
||||||
|
while IFS= read -r unit; do
|
||||||
|
grep -qx 'Restart=always' "$unit" || {
|
||||||
|
echo "$(basename "$unit"): $(grep '^Restart=' "$unit") (must be always)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "every LLM root unit restarts promptly and starts at boot" {
|
||||||
|
# RestartSec=10 is the fleet's cadence: fast enough that a transient
|
||||||
|
# fault is invisible to traffic, slow enough that a hard fault does not
|
||||||
|
# spin the engine (each radiance boot re-pins ~147 GiB).
|
||||||
|
# WantedBy=multi-user.target is boot persistence for a ROOT unit —
|
||||||
|
# default.target is the USER-manager's target and means nothing here.
|
||||||
|
local unit
|
||||||
|
while IFS= read -r unit; do
|
||||||
|
grep -qE '^RestartSec=([1-9]|1[0-9]|20)$' "$unit" || {
|
||||||
|
echo "$(basename "$unit"): no RestartSec in 1..20"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
grep -qx 'WantedBy=multi-user.target' "$unit" || {
|
||||||
|
echo "$(basename "$unit"): $(grep '^WantedBy=' "$unit")"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
done < <(find "$REPO_ROOT/scripts" -name "frank-*.service" -type f | grep -v AGENTTRASH | sort)
|
||||||
|
}
|
||||||
|
|||||||
@@ -112,11 +112,11 @@ GLUETUN="$REPO_ROOT/dankclaw/searxng/quadlet/gluetun.container"
|
|||||||
# Public edge: the vhost must proxy to the challenge proxy, not straight to the app.
|
# Public edge: the vhost must proxy to the challenge proxy, not straight to the app.
|
||||||
awk '
|
awk '
|
||||||
/^marreta\.franksdank\.cloud \{/ { inblk=1 }
|
/^marreta\.franksdank\.cloud \{/ { inblk=1 }
|
||||||
inblk && /reverse_proxy anubis:8923/ { challenge=1 }
|
inblk && /import guarded/ { challenge=1 }
|
||||||
inblk && /^\}/ { inblk=0 }
|
inblk && /^\}/ { inblk=0 }
|
||||||
END { exit challenge ? 0 : 1 }
|
END { exit challenge ? 0 : 1 }
|
||||||
' "$REPO_ROOT/dankclaw/Caddyfile" \
|
' "$REPO_ROOT/dankclaw/Caddyfile" \
|
||||||
|| { echo "marreta vhost must reverse_proxy anubis:8923"; return 1; }
|
|| { echo "marreta vhost must import the guarded chain (the one named Anubis snippet)"; return 1; }
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the Caddyfile post-auth listener routes marreta to the published 8082" {
|
@test "the Caddyfile post-auth listener routes marreta to the published 8082" {
|
||||||
|
|||||||
@@ -433,11 +433,14 @@ assert any('ALERTS' in e for e in exprs), exprs
|
|||||||
grep -q -- '--web.listen-address=127.0.0.1:9100' "$unit"
|
grep -q -- '--web.listen-address=127.0.0.1:9100' "$unit"
|
||||||
grep -q -- '--web.listen-address=192.168.0.235:9100' "$unit"
|
grep -q -- '--web.listen-address=192.168.0.235:9100' "$unit"
|
||||||
! grep -qE -- '--web\.listen-address=(0\.0\.0\.0|:9100|\[::\])' "$unit"
|
! grep -qE -- '--web\.listen-address=(0\.0\.0\.0|:9100|\[::\])' "$unit"
|
||||||
# One owner per container name: the compose service must be gone, or two
|
# One owner per container name: Dankserver has no compose file at all
|
||||||
# definitions fight over the same container.
|
# since the 2026-10-13 retirement (it lived beside diverged quadlet
|
||||||
! grep -qE '^\s+node-exporter:' "$REPO_ROOT/dankserver/compose.yaml"
|
# mounts for weeks — the two-owners trap by construction).
|
||||||
# The deploy path has to know about it on this target.
|
[ ! -e "$REPO_ROOT/dankserver/compose.yaml" ]
|
||||||
grep -q 'dankserver:node-exporter' "$REPO_ROOT/scripts/deploy-remote.sh"
|
# The deploy path has to know about it on this target — acceptance
|
||||||
|
# is the inventory (rank 3), not a literal list in the runner.
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service node-exporter
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
|
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
|
||||||
--paths dankserver/observability/quadlet/node-exporter.container
|
--paths dankserver/observability/quadlet/node-exporter.container
|
||||||
[ "$status" -eq 0 ]
|
[ "$status" -eq 0 ]
|
||||||
|
|||||||
Executable
+138
@@ -0,0 +1,138 @@
|
|||||||
|
#!/usr/bin/env bats
|
||||||
|
# Seam: the service inventory (rank 3). shared/services.yml is the one
|
||||||
|
# inventory of deployable services; scripts/service-map.py is its only
|
||||||
|
# reader, and every consumer (deploy-plan.py, deploy-remote.sh's
|
||||||
|
# allow-list, deploy.sh's --service gate, test-changed.sh's seam mapper,
|
||||||
|
# the Pi-hole hosts render, the Prometheus scrape facts, the homepage
|
||||||
|
# probes) goes through it. These tests pin the inventory's integrity,
|
||||||
|
# the planner's selection against a captured fixture, and the rendered
|
||||||
|
# artifacts' drift against the tracked copies.
|
||||||
|
|
||||||
|
load helpers
|
||||||
|
|
||||||
|
@test "inventory: schema, unit claims, path claims, compose ownership, and render drift are sound" {
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" check
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: the deployers' allow-list is the inventory" {
|
||||||
|
# Every service the inventory places on a host must pass "allows" for
|
||||||
|
# that host, and a name the inventory never places must be refused.
|
||||||
|
for host in dankclaw dankserver frankbook-pro; do
|
||||||
|
while IFS= read -r svc; do
|
||||||
|
[ -n "$svc" ] || continue
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$host" --service "$svc"
|
||||||
|
[ "$status" -eq 0 ] || { echo "allows refused $host:$svc"; return 1; }
|
||||||
|
done < <(python3 "$REPO_ROOT/scripts/service-map.py" services --host "$host")
|
||||||
|
done
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service sonarr
|
||||||
|
[ "$status" -ne 0 ] # the *arr stack lives on frankbook-pro, not dankserver
|
||||||
|
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service frank-archon
|
||||||
|
[ "$status" -ne 0 ] # frank-archon lives on dankserver
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: the planner reproduces the pinned selection fixture" {
|
||||||
|
# tests/fixtures/service-plan-golden.tsv is the characterization of
|
||||||
|
# path -> service selection across every tracked path, every target,
|
||||||
|
# and the multi-path combos the old hand-written planner was fed.
|
||||||
|
python3 - "$REPO_ROOT" <<'PYEOF'
|
||||||
|
import importlib.util
|
||||||
|
import sys
|
||||||
|
|
||||||
|
root = sys.argv[1]
|
||||||
|
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
|
||||||
|
sm = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(sm)
|
||||||
|
inv = sm.Inventory.load(root + "/shared/services.yml")
|
||||||
|
bad = []
|
||||||
|
with open(root + "/tests/fixtures/service-plan-golden.tsv") as fh:
|
||||||
|
for line in fh:
|
||||||
|
host, paths, want = line.rstrip("\n").split("\t")
|
||||||
|
got = ",".join(sm.select(inv, host, paths.split("+")))
|
||||||
|
if got != want:
|
||||||
|
bad.append(f"{host} {paths}\n want: {want}\n got: {got}")
|
||||||
|
if bad:
|
||||||
|
print(f"{len(bad)} selection mismatches:", *bad[:20], sep="\n")
|
||||||
|
sys.exit(1)
|
||||||
|
PYEOF
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: deploy-plan.py keeps its CLI contract on top of service-map" {
|
||||||
|
# stdin paths, positional paths, and the --repo/--base/--head diff mode
|
||||||
|
# all route to the same selection.
|
||||||
|
run bash -c "cd '$REPO_ROOT' && printf '%s\n' dankclaw/searxng/quadlet/gluetun.container | \
|
||||||
|
scripts/deploy-plan.py --target dankclaw"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "${lines[0]}" = "gluetun" ]
|
||||||
|
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver scripts/frank-archon.env
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "${lines[0]}" = "frank-archon" ]
|
||||||
|
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankclaw --repo "$REPO_ROOT" --base HEAD --head HEAD
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ -z "$output" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: the Pi-hole hosts file is a rendered artifact" {
|
||||||
|
# The tracked addn-hosts must be byte-identical to the render; editing
|
||||||
|
# it by hand is drift the inventory check also catches.
|
||||||
|
run bash -c "cd '$REPO_ROOT' && python3 scripts/service-map.py render pihole-hosts | \
|
||||||
|
diff -u dankclaw/pihole/addn-hosts -"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: changed-test seams for a service unit are the service's own" {
|
||||||
|
# The rank-3 contract: a unit change selects the OWNING service's
|
||||||
|
# declared seams, not the old shotgun of unrelated quadlet seams.
|
||||||
|
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankclaw/gitea/quadlet/gitea.network
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[[ "$output" == *"tests/caddyfile.bats"* ]]
|
||||||
|
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
|
||||||
|
[[ "$output" != *"tests/open_webui_quadlet.bats"* ]]
|
||||||
|
[[ "$output" != *"tests/searxng_quadlet.bats"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: every tracked quadlet unit is claimed by exactly one service" {
|
||||||
|
# check units covers claim conflicts; this pins the census size so a
|
||||||
|
# unit added WITHOUT an inventory entry (unclaimed) fails the seam.
|
||||||
|
python3 - "$REPO_ROOT" <<'PYEOF'
|
||||||
|
import importlib.util
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import PurePosixPath
|
||||||
|
|
||||||
|
root = sys.argv[1]
|
||||||
|
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
|
||||||
|
sm = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(sm)
|
||||||
|
inv = sm.Inventory.load(root + "/shared/services.yml")
|
||||||
|
graph = sm.graph_for(tuple(inv.hosts))
|
||||||
|
owned = {u for u in graph.units}
|
||||||
|
problems = []
|
||||||
|
tracked_units = [
|
||||||
|
p for p in sm.tracked_files()
|
||||||
|
if PurePosixPath(p).name.endswith((".container", ".network")) and not sm.is_trash(p)
|
||||||
|
]
|
||||||
|
for path in tracked_units:
|
||||||
|
parts = PurePosixPath(path).parts
|
||||||
|
name = parts[-1]
|
||||||
|
key = (parts[0], name.rsplit(".", 1)[0])
|
||||||
|
if key not in graph.units and key not in graph.networks:
|
||||||
|
problems.append(f"unclaimed unit-file path: {path}")
|
||||||
|
if problems:
|
||||||
|
print("\n".join(problems))
|
||||||
|
sys.exit(1)
|
||||||
|
PYEOF
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "inventory: the node-exporter unit is a per-host rendered template" {
|
||||||
|
# Both quadlets are rendered artifacts: the render — not a hand-edited
|
||||||
|
# mirror kept in step by a comment — defines the unit on every host.
|
||||||
|
for host in dankclaw dankserver; do
|
||||||
|
run bash -c "cd '$REPO_ROOT' && python3 scripts/service-map.py render node-exporter-unit --host $host | \
|
||||||
|
diff -u $host/observability/quadlet/node-exporter.container -"
|
||||||
|
[ "$status" -eq 0 ] || { echo "$host unit drift"; return 1; }
|
||||||
|
done
|
||||||
|
# Hand-edited copies fail the drift gate:
|
||||||
|
grep -q "Keep this file in step" dankserver/observability/quadlet/node-exporter.container && return 1
|
||||||
|
true
|
||||||
|
}
|
||||||
Executable
+2
@@ -0,0 +1,2 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
exit 0
|
||||||
Executable
+11
@@ -0,0 +1,11 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Stub podman for the deploy-handler seam: `image exists` succeeds only for
|
||||||
|
# $STUB_IMAGE (empty = nothing in storage); every other verb is a no-op.
|
||||||
|
# The invocations are logged so a test can read what the handler executed.
|
||||||
|
printf '%s\n' "$*" >> "${STUB_PODMAN_LOG:-/dev/null}"
|
||||||
|
case "$1 $2" in
|
||||||
|
"image exists")
|
||||||
|
[[ -n "${STUB_IMAGE:-}" && "$3" == "$STUB_IMAGE" ]] || exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
exit 0
|
||||||
Executable
+2
@@ -0,0 +1,2 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
exit 0
|
||||||
Executable
+2
@@ -0,0 +1,2 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
exit 0
|
||||||
Executable
+13
@@ -0,0 +1,13 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Stub sudo for the deploy-handler seam: run_privileged pipes the password
|
||||||
|
# on stdin and calls `sudo -S -- <cmd>`. The stub drops the password and the
|
||||||
|
# flags and execs the command, so no privilege gate hides a handler bug.
|
||||||
|
cat >/dev/null 2>&1 || true
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
-S|--prompt) shift ;;
|
||||||
|
--) shift; break ;;
|
||||||
|
*) break ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
exec "$@"
|
||||||
Executable
+12
@@ -0,0 +1,12 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Stub systemctl for the deploy-handler seam: every invocation is logged, so
|
||||||
|
# a test asserts on what the handler ASKED systemd to do. `is-active` answers
|
||||||
|
# "inactive" (the state a cutover finds the unit in) and MainPID answers
|
||||||
|
# $STUB_MAIN_PID — 0 is the stray-port-holder shape, the false-green gate
|
||||||
|
# every LLM handler reads after a health answer.
|
||||||
|
printf '%s\n' "$*" >> "${STUB_SYSTEMCTL_LOG:-/dev/null}"
|
||||||
|
case "$*" in
|
||||||
|
"is-active "*) echo inactive ;;
|
||||||
|
*"MainPID --value "*) echo "${STUB_MAIN_PID:-4242}" ;;
|
||||||
|
esac
|
||||||
|
exit 0
|
||||||
Reference in New Issue
Block a user