feat(inventory): one service inventory drives planner, allow-lists, and seams
rank 3. shared/services.yml is the single source of truth for every deployable service (hosts, unit dirs, units, path claims, tests, scrape/ dns/probe facts, aliases, stacks); scripts/service-map.py is its only reader (hosts/services/allows/plan/seams/owner/units/stack/get/render/ check). deploy-plan.py becomes a thin CLI wrapper; both --service allow-lists (deploy.sh, deploy-remote.sh) and the changed-test mapper are the inventory. Selection semantics: host scoping (a host's tree selects only its own services), the quadlet netns/network coupling graph resolved across directories (gluetun's riders live in their own dirs), stacks, and canonical restart order. Against the pinned old-planner fixture: 921 raw deltas reduced to 17, all intentional (anubis now maps; cross-host paths select nothing; the retired qwen-sharp template plans nothing and the live froggeric template plans frank-archon; dankserver/romm/romm.container, missed by the old filename rule, plans the romm trio). The fixture is regenerated as the new pin. Integrity gates: schema, unit claims, quadlet-file ownership, path-glob liveness, Pi-hole hosts / Prometheus jobs / DNS / homepage probes drift. tests/service_inventory.bats pins all of it plus the plan fixture and the deploy-plan CLI. Allow-list grep seams in hermes_webui/observability_quadlet become behavioral allows calls.
This commit is contained in:
Frank
@@ -35,11 +35,16 @@ the [runbook index](#runbook-index). The full swap history lives in
|
||||
|
||||
## Repository Layout
|
||||
|
||||
- `dankserver/` — compose.yaml + Caddyfile + .env.example for the media stack
|
||||
- `dankclaw/` — compose.yaml + Caddyfile + .env.example for the AI stack
|
||||
- `shared/` — homepage, Prometheus, Loki, Alloy, SearXNG configs
|
||||
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh`, LLM serve
|
||||
scripts, systemd units, env tunables, bench harnesses
|
||||
- `dankserver/` — quadlet units + Caddyfile for the media stack (the
|
||||
compose.yaml inventory retired 2026-10-13; its mounts had diverged from
|
||||
the quadlets)
|
||||
- `dankclaw/` — quadlet units + Caddyfile for the AI stack (compose retired 2026-09-16)
|
||||
- `shared/` — the service inventory (`services.yml`), homepage, Prometheus,
|
||||
Loki, Alloy, SearXNG configs
|
||||
- `scripts/` — `deploy.sh` / `deploy-plan.py` / `deploy-remote.sh` and
|
||||
`service-map.py` (the inventory's only reader), LLM serve scripts + the
|
||||
shared artifact gates (`scripts/lib/llm-gates.sh`), systemd units, env
|
||||
tunables, bench harnesses
|
||||
- `old/` — retired LLM scripts (reference only)
|
||||
- `serverAppConfigs-docs/` — knowledge base (Obsidian vault, OKF format).
|
||||
The managed skill library is distilled there one page per skill-cluster:
|
||||
@@ -70,8 +75,13 @@ All edits flow through git. Never edit files on a remote machine directly.
|
||||
./scripts/deploy.sh dankclaw --dry-run
|
||||
```
|
||||
|
||||
The planner maps changed paths to services; the remote runner restarts only
|
||||
the selected service and records its revision outside the checkout.
|
||||
The inventory (`shared/services.yml`, read only through
|
||||
`scripts/service-map.py`) maps changed paths to services; the remote runner
|
||||
restarts only the selected service and records its revision outside the
|
||||
checkout. Both `--service` allow-lists, the planner, and the changed-test
|
||||
seam mapper ARE the inventory — adding, moving, or retiring a service is a
|
||||
data edit, not a script edit
|
||||
(`serverAppConfigs-docs/Architecture/Service Inventory.md`).
|
||||
Documentation-only changes are a no-op. LLM services are selected only by
|
||||
their own files — every endpoint (`frank-dragoon`/`frank-zealot`/`frank-archon`)
|
||||
by its unit/env/serve trio in `scripts/` — so
|
||||
@@ -164,12 +174,17 @@ Tests are mandatory before commit (`bats` required on the dev machine):
|
||||
./tests/run.sh
|
||||
```
|
||||
|
||||
Seams: compose validity, env consistency (no leaked secrets), Caddyfile
|
||||
Seams: env consistency (no leaked secrets), Caddyfile
|
||||
validity, quadlet units, docs completeness **including vault link integrity**,
|
||||
script syntax (shell + python), LLM unit integrity, LLM secret hygiene,
|
||||
deployment flow, service-scoped planning, homepage config/pages/deploy/checker,
|
||||
and `e2e_scaffold` — which fails if a Playwright spec file's number prefix is in
|
||||
no project, because such a spec silently never runs.
|
||||
LLM artifact gates (`llm_gates`/`llm_deploy_gates` — the shared
|
||||
`scripts/lib/llm-gates.sh` every endpoint's serve and deploy gates run on),
|
||||
deployment flow, service-scoped planning, the **service inventory**
|
||||
(`service_inventory.bats`: inventory soundness, the pinned path->service
|
||||
plan fixture, render drift for the Pi-hole hosts file), homepage
|
||||
config/pages/deploy/checker, and `e2e_scaffold` — which fails if a
|
||||
Playwright spec file's number prefix is in no project, because such a spec
|
||||
silently never runs.
|
||||
See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
|
||||
|
||||
## Secret Rules
|
||||
@@ -1041,7 +1056,7 @@ Active:
|
||||
|---------|---------|
|
||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
||||
| frank-archon (radiance StillDeadcode, Swift1.5-Qwen3.8-Flash-Next GPTQ w4 + bf16-protected experts, int8 trunk, bf16 KV, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||
|
||||
@@ -1050,6 +1065,7 @@ Retired (rollback targets and history):
|
||||
| Runtime | Host | Window | Runbook |
|
||||
|---------|------|--------|---------|
|
||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||
| radiance StillDeadcode Qwen3.8-Flash-Next native-FP8, 2×R9700 TP=2 (`frank-archon`) | Dankserver | 2026-10-08 → w4dn8 swap (rollback target — `.rad` payload on disk) | `frank-archon-radiance-qwen38-native-fp8.md` |
|
||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||
| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
|
||||
|
||||
+54
-461
@@ -1,475 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Plan runtime services affected by repository changes.
|
||||
"""deploy-plan.py — map changed paths to deployable services.
|
||||
|
||||
The planner is intentionally side-effect free. It accepts either an explicit
|
||||
path list or two Git revisions. The latter lets compose-file changes resolve
|
||||
to only the service definitions that actually changed instead of restarting an
|
||||
entire compose project.
|
||||
Thin wrapper over scripts/service-map.py, which owns the path->service
|
||||
semantics in the shared inventory (shared/services.yml): per-service path
|
||||
rules, the quadlet netns/network coupling graph, host scoping (a host's
|
||||
directory can only select that host's services), and canonical restart
|
||||
order. This file keeps the deployer's CLI contract only.
|
||||
|
||||
Usage:
|
||||
deploy-plan.py --target NAME --repo DIR --base REV --head REV
|
||||
deploy-plan.py --target NAME [paths...] (paths also on stdin)
|
||||
|
||||
Prints one service name per line (canonical restart order); exit 0 with no
|
||||
output when nothing relevant changed.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
# Dependency-sensitive services come before their consumers. Independent
|
||||
# compose services follow. LLM services are deliberately last and are never
|
||||
# implied by a generic host, stack, documentation, or config change.
|
||||
SERVICE_ORDER = (
|
||||
"gluetun",
|
||||
"searxng-valkey",
|
||||
"searxng",
|
||||
"marreta",
|
||||
"pihole",
|
||||
"open-webui",
|
||||
"gitea-postgres",
|
||||
"gitea-server",
|
||||
"comfyui",
|
||||
"seerr",
|
||||
"hermes",
|
||||
"hermes-dashboard",
|
||||
"hermes-webui",
|
||||
"prometheus",
|
||||
"grafana",
|
||||
"loki",
|
||||
"alloy",
|
||||
"node-exporter",
|
||||
"alertmanager",
|
||||
"authentik-postgres",
|
||||
"authentik-server",
|
||||
"authentik-worker",
|
||||
"authentik-outpost",
|
||||
"authentik-outpost-ldap",
|
||||
"caddy",
|
||||
"homepage",
|
||||
"romm-mariadb",
|
||||
"romm-redis",
|
||||
"romm",
|
||||
"copyparty",
|
||||
"jellyfin",
|
||||
"calibre-web",
|
||||
"calibre-server",
|
||||
"plex",
|
||||
"tautulli",
|
||||
"prowlarr",
|
||||
"qbittorrent",
|
||||
"radarr",
|
||||
"sonarr",
|
||||
"lidarr",
|
||||
"readarr",
|
||||
"frank-dragoon",
|
||||
"frank-zealot",
|
||||
"frank-archon",
|
||||
)
|
||||
|
||||
DANKCLAW_SERVICES = {
|
||||
"hermes-webui",
|
||||
"gluetun",
|
||||
"searxng-valkey",
|
||||
"searxng",
|
||||
"marreta",
|
||||
"pihole",
|
||||
"open-webui",
|
||||
"gitea-postgres",
|
||||
"gitea-server",
|
||||
"comfyui",
|
||||
"seerr",
|
||||
"hermes",
|
||||
"hermes-dashboard",
|
||||
"prometheus",
|
||||
"grafana",
|
||||
"loki",
|
||||
"alloy",
|
||||
"node-exporter",
|
||||
"alertmanager",
|
||||
"authentik-postgres",
|
||||
"authentik-server",
|
||||
"authentik-worker",
|
||||
"authentik-outpost",
|
||||
"authentik-outpost-ldap",
|
||||
"caddy",
|
||||
"homepage",
|
||||
"frank-dragoon",
|
||||
"frank-zealot",
|
||||
}
|
||||
|
||||
DANKSERVER_SERVICES = {
|
||||
"romm-mariadb",
|
||||
"romm-redis",
|
||||
"romm",
|
||||
"copyparty",
|
||||
"jellyfin",
|
||||
"calibre-web",
|
||||
"calibre-server",
|
||||
"plex",
|
||||
"tautulli",
|
||||
"prowlarr",
|
||||
"qbittorrent",
|
||||
"radarr",
|
||||
"sonarr",
|
||||
"lidarr",
|
||||
"readarr",
|
||||
"frank-archon",
|
||||
"node-exporter",
|
||||
}
|
||||
|
||||
# FrankBook-Pro runs the *arr stack + qBittorrent inside a podman machine on
|
||||
# macOS. Service names overlap the Linux hosts (gluetun, qbittorrent, ...);
|
||||
# the per-target sets and the frankbook-pro intercept in deploy-remote.sh
|
||||
# keep dispatch unambiguous.
|
||||
FRANKBOOK_SERVICES = {
|
||||
"gluetun",
|
||||
"qbittorrent",
|
||||
"prowlarr",
|
||||
"radarr",
|
||||
"sonarr",
|
||||
"lidarr",
|
||||
"readarr",
|
||||
}
|
||||
|
||||
DANKCLAW_COMPOSE_SERVICES = set()
|
||||
# Dankserver's compose file is still the source of truth for its media stack;
|
||||
# Dankclaw's is gone (deleted 2026-09-16). Every service on Dankclaw is a
|
||||
# Quadlet unit now — the last compose services were the hermes trio, whose units
|
||||
# have owned their containers since 2026-09-12 and whose compose blocks were
|
||||
# dead weight that could only create a duplicate container.
|
||||
|
||||
DANKSERVER_COMPOSE_SERVICES = {
|
||||
"mariadb",
|
||||
"redis",
|
||||
"romm",
|
||||
"copyparty",
|
||||
"jellyfin",
|
||||
"calibre-web",
|
||||
}
|
||||
COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
|
||||
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
SERVICE_MAP = os.path.join(SCRIPT_DIR, "service-map.py")
|
||||
|
||||
|
||||
|
||||
def _add(services: set[str], *names: str) -> None:
|
||||
services.update(names)
|
||||
|
||||
|
||||
def target_services(target: str) -> set[str]:
|
||||
if target == "dankclaw":
|
||||
return DANKCLAW_SERVICES
|
||||
if target == "dankserver":
|
||||
return DANKSERVER_SERVICES
|
||||
if target == "frankbook-pro":
|
||||
return FRANKBOOK_SERVICES
|
||||
raise ValueError(f"unknown target: {target}")
|
||||
|
||||
|
||||
def _compose_services(target: str) -> set[str]:
|
||||
return DANKCLAW_COMPOSE_SERVICES if target == "dankclaw" else DANKSERVER_COMPOSE_SERVICES
|
||||
|
||||
|
||||
def _git_show(repo: str, revision: str, path: str) -> str | None:
|
||||
result = subprocess.run(
|
||||
["git", "-C", repo, "show", f"{revision}:{path}"],
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
return result.stdout if result.returncode == 0 else None
|
||||
|
||||
|
||||
def _yaml_document(contents: str | None) -> dict[str, Any]:
|
||||
if contents is None:
|
||||
return {}
|
||||
try:
|
||||
data = yaml.safe_load(contents) or {}
|
||||
except yaml.YAMLError:
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
|
||||
def _yaml_services(contents: str | None) -> dict[str, Any]:
|
||||
services = _yaml_document(contents).get("services", {})
|
||||
return services if isinstance(services, dict) else {}
|
||||
|
||||
|
||||
def compose_changes(repo: str, target: str, base: str, head: str) -> set[str]:
|
||||
"""Return only compose services whose YAML definitions or dependencies changed.
|
||||
|
||||
A target with no compose file (Dankclaw since 2026-09-16) plans nothing:
|
||||
its services are Quadlet units, selected by their own paths.
|
||||
"""
|
||||
path = f"{target}/compose.yaml"
|
||||
if not _git_show(repo, head, path) and not _git_show(repo, base, path):
|
||||
return set()
|
||||
old_doc = _yaml_document(_git_show(repo, base, path))
|
||||
new_doc = _yaml_document(_git_show(repo, head, path))
|
||||
old = old_doc.get("services", {}) if isinstance(old_doc.get("services", {}), dict) else {}
|
||||
new = new_doc.get("services", {}) if isinstance(new_doc.get("services", {}), dict) else {}
|
||||
names = set(old) | set(new)
|
||||
allowed = _compose_services(target)
|
||||
|
||||
# An unavailable or malformed revision is safer as a broad compose plan
|
||||
# than as a silent no-op. It still cannot include LLM units.
|
||||
if (old_doc and not new_doc) or (new_doc and not old_doc and names):
|
||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
||||
|
||||
# Top-level networks/configuration can affect every compose service even
|
||||
# when no individual service mapping changed. Ignore the services key while
|
||||
# comparing the rest of the Compose document.
|
||||
old_globals = {key: value for key, value in old_doc.items() if key != "services"}
|
||||
new_globals = {key: value for key, value in new_doc.items() if key != "services"}
|
||||
if old_globals != new_globals:
|
||||
return {COMPOSE_NAME_MAP.get(n, n) for n in allowed}
|
||||
|
||||
changed = {name for name in names if name in allowed and old.get(name) != new.get(name)}
|
||||
return {COMPOSE_NAME_MAP.get(n, n) for n in changed}
|
||||
|
||||
|
||||
def services_for_path(path: str, target: str) -> set[str]:
|
||||
"""Return services affected by one repository-relative path."""
|
||||
path = path.replace("\\", "/")
|
||||
p = PurePosixPath(path)
|
||||
parts = p.parts
|
||||
services: set[str] = set()
|
||||
if not parts:
|
||||
return services
|
||||
|
||||
# Non-runtime material is deliberately a no-op.
|
||||
if parts[0] in {"docs", "tests", "old", ".githooks"}:
|
||||
return services
|
||||
if path in {"README.md", "AGENTS.md"} or parts[0] == ".obsidian":
|
||||
return services
|
||||
|
||||
if parts[0] == "shared":
|
||||
if len(parts) < 2:
|
||||
return services
|
||||
group = parts[1]
|
||||
if group == "searxng":
|
||||
_add(services, "searxng")
|
||||
elif group == "prometheus":
|
||||
_add(services, "prometheus")
|
||||
if len(parts) >= 3 and parts[2] == "alertmanager.yml":
|
||||
# The Alertmanager config ships from this directory: a change
|
||||
# to it must redeploy the service that installs it.
|
||||
_add(services, "alertmanager")
|
||||
elif group == "loki":
|
||||
_add(services, "loki")
|
||||
elif group == "alloy":
|
||||
_add(services, "alloy")
|
||||
elif group == "grafana":
|
||||
# Provisioned datasources and dashboards: git is the source of
|
||||
# truth, so a change here must restart the service that reads them.
|
||||
_add(services, "grafana")
|
||||
elif group == "homepage":
|
||||
# Homepage is a Dankclaw-owned deployment. Shared homepage files
|
||||
# are intentionally not copied during a Dankserver-only run.
|
||||
if target == "dankclaw":
|
||||
_add(services, "homepage")
|
||||
return services
|
||||
|
||||
if parts[0] == "dankclaw":
|
||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
||||
# Dankclaw has no compose file since 2026-09-16: every service is a
|
||||
# Quadlet unit, each selected by its own path. A change (or a
|
||||
# deletion) of this path therefore restarts nothing.
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "Caddyfile":
|
||||
_add(services, "caddy")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "Containerfile.comfyui":
|
||||
_add(services, "comfyui")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] in {"caddy", "comfyui", "loki", "alloy", "hermes", "anubis", "seerr"}:
|
||||
_add(services, parts[1])
|
||||
return services
|
||||
if len(parts) >= 4 and parts[1] == "authentik" and parts[2] == "quadlet":
|
||||
name = parts[3].removesuffix(".container")
|
||||
if name in {"authentik-postgres", "authentik-server", "authentik-worker", "authentik-outpost", "authentik-outpost-ldap"}:
|
||||
_add(services, name)
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "searxng":
|
||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
||||
name = parts[3]
|
||||
if name == "gluetun.container":
|
||||
# SearXNG and Marreta (+ its browser companion) share
|
||||
# Gluetun's network namespace. A Gluetun recreation
|
||||
# tears the netns down, so every sharer must restart.
|
||||
_add(services, "gluetun", "searxng", "marreta")
|
||||
elif name == "searxng.container":
|
||||
_add(services, "searxng")
|
||||
elif name == "searxng-valkey.container":
|
||||
_add(services, "searxng-valkey", "searxng")
|
||||
elif name == "searxng.network":
|
||||
_add(services, "gluetun", "searxng-valkey", "searxng", "marreta")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "marreta":
|
||||
# One deployable owns both the app and its browser companion;
|
||||
# both live in gluetun's netns. A change to either unit selects
|
||||
# the marreta service, which restarts browser-then-app.
|
||||
_add(services, "marreta")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "pihole":
|
||||
_add(services, "pihole")
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "open-webui":
|
||||
_add(services, "open-webui")
|
||||
return services
|
||||
# frank-dragoon left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
|
||||
# build, root unit): its files live in scripts/ and map there.
|
||||
if len(parts) >= 2 and parts[1] == "observability":
|
||||
# The five observability containers became Quadlet units on
|
||||
# 2026-09-11. A unit change selects exactly its own service;
|
||||
# SERVICE_ORDER already puts each dependency ahead of its
|
||||
# consumer (prometheus before grafana, loki before alloy).
|
||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
||||
name = parts[3].removesuffix(".container")
|
||||
if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter", "alertmanager"}:
|
||||
_add(services, name)
|
||||
return services
|
||||
if len(parts) >= 2 and parts[1] == "gitea":
|
||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
||||
name = parts[3]
|
||||
if name == "gitea-postgres.container":
|
||||
_add(services, "gitea-postgres")
|
||||
elif name == "gitea-server.container":
|
||||
_add(services, "gitea-server")
|
||||
elif name == "gitea-caddy.container":
|
||||
# Removing the old split proxy requires the canonical
|
||||
# Caddy handler to retire its installed unit/container
|
||||
# before it claims ports 80/443.
|
||||
_add(services, "caddy")
|
||||
elif name == "gitea.network":
|
||||
_add(services, "gitea-postgres", "gitea-server")
|
||||
return services
|
||||
return services
|
||||
|
||||
if parts[0] == "dankserver":
|
||||
if len(parts) >= 2 and parts[1] == "compose.yaml":
|
||||
# Compose service keys map onto the deployed unit names.
|
||||
_add(services, *(COMPOSE_NAME_MAP.get(n, n) for n in DANKSERVER_COMPOSE_SERVICES))
|
||||
return services
|
||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
||||
quadlet_services = {
|
||||
"romm": {"romm", "romm-mariadb", "romm-redis"},
|
||||
"jellyfin": {"jellyfin"},
|
||||
"calibre-web": {"calibre-web"},
|
||||
"calibre-server": {"calibre-server"},
|
||||
"copyparty": {"copyparty"},
|
||||
"plex": {"plex"},
|
||||
"tautulli": {"tautulli"},
|
||||
}
|
||||
_add(services, *quadlet_services.get(parts[1], set()))
|
||||
if len(parts) >= 2 and parts[1] == "observability":
|
||||
# Dankserver's node-exporter became a Quadlet unit on 2026-09-16
|
||||
# (serverAppConfigs-docs/Infrastructure/Observability.md): its
|
||||
# compose service died at a reboot and stayed dead, so Prometheus'
|
||||
# node-dankserver job had nothing to scrape.
|
||||
if len(parts) >= 4 and parts[2] == "quadlet":
|
||||
name = parts[3].removesuffix(".container")
|
||||
if name == "node-exporter":
|
||||
_add(services, "node-exporter")
|
||||
return services
|
||||
# frank-archon's quadlet-era directory left the tree on 2026-09-26
|
||||
# (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
|
||||
# the trio in scripts/ now maps it). Nothing lives under
|
||||
# dankserver/frank-archon/ except dated trash, and trash plans
|
||||
# nothing (the retire rule above the service blocks).
|
||||
return services
|
||||
|
||||
if parts[0] == "frankbook-pro":
|
||||
if len(parts) >= 3 and parts[2] == "quadlet":
|
||||
name = parts[3].removesuffix(".container")
|
||||
if name == "gluetun":
|
||||
# Every *arr + qBittorrent container shares gluetun's
|
||||
# network namespace; a gluetun recreation kills all of
|
||||
# them, so the plan restarts the whole netns family.
|
||||
_add(services, "gluetun", "qbittorrent",
|
||||
"prowlarr", "sonarr", "radarr", "lidarr", "readarr")
|
||||
elif name in FRANKBOOK_SERVICES:
|
||||
_add(services, name)
|
||||
return services
|
||||
|
||||
if parts[0] == "scripts":
|
||||
name = p.name
|
||||
if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
|
||||
_add(services, "homepage")
|
||||
if name in {"signal-relay.py", "signal-relay.service"}:
|
||||
# The Signal relay is installed by the alertmanager deploy, so a
|
||||
# change to it redeploying alertmanager is the only path that
|
||||
# reaches it on the host.
|
||||
_add(services, "alertmanager")
|
||||
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
|
||||
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
|
||||
# files here; each one selects frank-zealot only. The ik_llama.cpp
|
||||
# trio that lived here for two days is in scripts/.AGENTTRASH/,
|
||||
# which is archive material and plans nothing.
|
||||
_add(services, "frank-zealot")
|
||||
if name in {"frank-dragoon.env", "frank-dragoon.service", "serve-frank-dragoon.sh"}:
|
||||
# CIRU v4.4.1 frank-dragoon (2026-09-19) keeps its unit, env and
|
||||
# serve files here the same way: a root unit on a host-built
|
||||
# runtime. The download script fetches the payload, which is
|
||||
# host-side state, so it selects nothing.
|
||||
_add(services, "frank-dragoon")
|
||||
if name == "qwen-sharp-v22.4.0.jinja":
|
||||
# The Qwen-Sharp chat template frank-dragoon loads via
|
||||
# --chat-template-file. It lived under dankserver/frank-medium/ until
|
||||
# the 2026-09-26 cutover retired that directory; editing it changes
|
||||
# what the live frank-dragoon endpoint renders, so it plans
|
||||
# frank-dragoon.
|
||||
_add(services, "frank-dragoon")
|
||||
if name in {"frank-archon.env", "frank-archon.service", "serve-frank-archon.sh"}:
|
||||
# llama.cpp frank-archon (2026-09-26, replacing the Lucebox
|
||||
# DS4.1 quadlet) keeps its unit, env and serve files here the
|
||||
# same way: a root unit on the unsloth b11030-mix prebuilt
|
||||
# serving Qwen3.8-Flash-Next Q8_0. The download script fetches
|
||||
# the payload, which is host-side state, so it selects nothing.
|
||||
_add(services, "frank-archon")
|
||||
# Deployment/test/benchmark scripts and other repo tooling do not
|
||||
# restart a runtime service.
|
||||
return services
|
||||
|
||||
# Examples and env templates are not deployed runtime state.
|
||||
if p.name in {".env.example", "env.example"}:
|
||||
return services
|
||||
return services
|
||||
|
||||
def changed_paths(repo: str, base: str, head: str) -> list[str]:
|
||||
result = subprocess.run(
|
||||
["git", "-C", repo, "diff", "--name-only", "--diff-filter=ACDMRTUXB", base, head],
|
||||
text=True,
|
||||
stdout=subprocess.PIPE,
|
||||
check=True,
|
||||
)
|
||||
return [line for line in result.stdout.splitlines() if line]
|
||||
|
||||
|
||||
def plan(paths: list[str], target: str, repo: str | None = None,
|
||||
base: str | None = None, head: str | None = None) -> list[str]:
|
||||
services: set[str] = set()
|
||||
for path in paths:
|
||||
if path == f"{target}/compose.yaml" and repo and base and head:
|
||||
services.update(compose_changes(repo, target, base, head))
|
||||
else:
|
||||
services.update(services_for_path(path, target))
|
||||
allowed = target_services(target)
|
||||
return [name for name in SERVICE_ORDER if name in services and name in allowed]
|
||||
def git_paths(repo: str, base: str, head: str) -> list[str]:
|
||||
proc = subprocess.run(
|
||||
["git", "-C", repo, "diff", "--name-only", base, head],
|
||||
text=True, capture_output=True)
|
||||
if proc.returncode != 0:
|
||||
sys.stderr.write(f"deploy-plan: git diff failed: {proc.stderr}")
|
||||
raise SystemExit(1)
|
||||
return [line for line in proc.stdout.splitlines() if line]
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--target", choices=("dankclaw", "dankserver", "frankbook-pro"), required=True)
|
||||
parser.add_argument("--paths", nargs="+")
|
||||
parser.add_argument("--repo", help="repository path for revision-aware planning")
|
||||
parser.add_argument("--base", help="base Git revision")
|
||||
parser.add_argument("--head", help="head Git revision")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.paths and not (args.repo and args.base and args.head):
|
||||
parser.error("provide --paths or all of --repo, --base, and --head")
|
||||
paths = args.paths or changed_paths(args.repo, args.base, args.head)
|
||||
print("\n".join(plan(paths, args.target, args.repo, args.base, args.head)))
|
||||
args = sys.argv[1:]
|
||||
if not args or args[0] != "--target":
|
||||
print(__doc__, file=sys.stderr)
|
||||
return 2
|
||||
if len(args) < 2:
|
||||
print("missing target name after --target", file=sys.stderr)
|
||||
return 2
|
||||
target = args[1]
|
||||
# `--paths` is the historic seam keyword: a flag, not an option — the
|
||||
# paths are the positional arguments that follow it.
|
||||
rest = [a for a in args[2:] if a != "--paths"]
|
||||
if "--repo" in rest:
|
||||
try:
|
||||
repo = rest[rest.index("--repo") + 1]
|
||||
base = rest[rest.index("--base") + 1]
|
||||
head = rest[rest.index("--head") + 1]
|
||||
except (ValueError, IndexError):
|
||||
print("--repo, --base and --head must appear together", file=sys.stderr)
|
||||
return 2
|
||||
paths = git_paths(repo, base, head)
|
||||
else:
|
||||
paths = rest or [line.strip() for line in sys.stdin if line.strip()]
|
||||
if not paths:
|
||||
return 0
|
||||
proc = subprocess.run([sys.executable, SERVICE_MAP, "plan", "--host", target] + paths,
|
||||
text=True, capture_output=True)
|
||||
if proc.returncode != 0:
|
||||
sys.stderr.write(proc.stderr)
|
||||
return proc.returncode
|
||||
sys.stdout.write(proc.stdout)
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
+7
-25
@@ -131,31 +131,13 @@ else
|
||||
fi
|
||||
|
||||
valid_service() {
|
||||
local service="$1"
|
||||
case "$TARGET:$service" in
|
||||
dankclaw:caddy|dankclaw:gluetun|dankclaw:searxng-network|dankclaw:searxng-valkey|\
|
||||
dankclaw:searxng|dankclaw:marreta|dankclaw:pihole|dankclaw:open-webui|dankclaw:gitea-postgres|\
|
||||
dankclaw:gitea-server|dankclaw:homepage|\
|
||||
dankclaw:comfyui|dankclaw:hermes|dankclaw:hermes-dashboard|\
|
||||
dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|dankclaw:loki|dankclaw:alloy|\
|
||||
dankclaw:alertmanager|\
|
||||
dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
|
||||
dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
|
||||
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
|
||||
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
|
||||
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
|
||||
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
|
||||
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
|
||||
dankserver:frank-archon|\
|
||||
frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
|
||||
frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
|
||||
frankbook-pro:readarr)
|
||||
return 0
|
||||
;;
|
||||
*)
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
# The inventory is the only allow-list (see scripts/service-map.py);
|
||||
# service-map's "allows" exit code is the answer. The former literal
|
||||
# case list here drifted from reality (it still listed the *arr stack
|
||||
# under dankserver, which moved to frankbook-pro, and a
|
||||
# "searxng-network" name that is a generated systemd unit, not a
|
||||
# deployable service).
|
||||
python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$TARGET" --service "$1" >/dev/null
|
||||
}
|
||||
|
||||
for service in "${EXPLICIT_SERVICES[@]}"; do
|
||||
|
||||
Executable
+647
@@ -0,0 +1,647 @@
|
||||
#!/usr/bin/env python3
|
||||
"""service-map.py — the only reader of the service inventory.
|
||||
|
||||
Four lists used to agree by discipline (the planner's path rules, the
|
||||
deployer's allow-list, the changed-test mapper, and the scrape/probe/DNS
|
||||
facts written by hand in three config files). They now all read
|
||||
`shared/services.yml` through this tool:
|
||||
|
||||
hosts the deploy targets
|
||||
services [--host H] deployables a target accepts, canonical order
|
||||
allows --host H --service S exit 0 when H accepts S (silent)
|
||||
plan --host H PATH... services selected by changed paths, order-safe
|
||||
seams [--host H] PATH... the Bats seams a changed path must re-run
|
||||
owner --host H PATH the service that owns a file (empty: none)
|
||||
units --host H --service S "<unit> <unit_dir>" lines the handler installs
|
||||
stack --host H --service S the deployable + members one handler deploys
|
||||
get --service S [--host H] F one field as JSON
|
||||
render pihole-hosts the Pi-hole addn-hosts file
|
||||
check [NAME...] integrity + drift gates (exit 1 on drift)
|
||||
|
||||
Two laws the engine enforces structurally:
|
||||
|
||||
* A path selects only services that RUN on the planned target. The old planner
|
||||
matched paths host-blind and filtered at the end, so a frankbook-pro file
|
||||
could plan a dankserver service (the stale *arr entries made that reachable).
|
||||
* The netns family is DERIVED, never declared: a unit file that says
|
||||
`Network=gluetun.container` rides that tunnel, so a gluetun change selects
|
||||
every sharer. The unit files are the source of that coupling.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from functools import lru_cache
|
||||
from pathlib import Path, PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
DEFAULT_INVENTORY = REPO_ROOT / "shared" / "services.yml"
|
||||
|
||||
REQUIRED_KEYS = ("name", "hosts", "kind", "tests")
|
||||
KINDS = {"quadlet", "root-unit", "vm-quadlet", "host"}
|
||||
UNIT_SUFFIXES = (".container", ".network")
|
||||
|
||||
# Repo material that is never a deployed runtime's state.
|
||||
NON_RUNTIME_ROOTS = {"docs", "report", "old", ".githooks", ".obsidian", ".github",
|
||||
"tests", "serverAppConfigs-docs"}
|
||||
NON_RUNTIME_NAMES = {".env.example"}
|
||||
NON_RUNTIME_PATHS = {"README.md", "AGENTS.md", ".gitignore"}
|
||||
|
||||
|
||||
def is_trash(path: str) -> bool:
|
||||
return ".AGENTTRASH" in PurePosixPath(path).parts
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# inventory
|
||||
# --------------------------------------------------------------------------
|
||||
class Inventory:
|
||||
def __init__(self, doc: dict[str, Any], origin: str) -> None:
|
||||
self.origin = origin
|
||||
self.hosts: dict[str, dict[str, Any]] = doc["hosts"]
|
||||
self.services: list[dict[str, Any]] = doc["services"]
|
||||
self.unowned: dict[str, str] = doc.get("unowned", {})
|
||||
for index, svc in enumerate(self.services):
|
||||
svc["_index"] = index
|
||||
|
||||
@classmethod
|
||||
def load(cls, path: str | Path = DEFAULT_INVENTORY) -> "Inventory":
|
||||
doc = yaml.safe_load(Path(path).read_text())
|
||||
if not isinstance(doc, dict) or "services" not in doc:
|
||||
raise SystemExit(f"{path}: not a service inventory document")
|
||||
return cls(doc, str(path))
|
||||
|
||||
def on(self, host: str) -> list[dict[str, Any]]:
|
||||
return [s for s in self.services if host in s["hosts"]]
|
||||
|
||||
def named(self, host: str, name: str) -> list[dict[str, Any]]:
|
||||
return [s for s in self.services if s["name"] == name and host in s["hosts"]]
|
||||
|
||||
def any_named(self, name: str) -> list[dict[str, Any]]:
|
||||
return [s for s in self.services if s["name"] == name]
|
||||
|
||||
@property
|
||||
def order(self) -> list[str]:
|
||||
seen: list[str] = []
|
||||
for svc in self.services:
|
||||
if svc["name"] not in seen:
|
||||
seen.append(svc["name"])
|
||||
return seen
|
||||
|
||||
def edge_ip(self) -> str:
|
||||
"""The single address every DNS name resolves at (Caddy's edge)."""
|
||||
for host in self.hosts.values():
|
||||
if "dns_edge" in host:
|
||||
return str(host["dns_edge"])
|
||||
raise SystemExit(f"{self.origin}: no host declares dns_edge")
|
||||
|
||||
def unit_dirs(self, svc: dict[str, Any], host: str | None = None) -> list[str]:
|
||||
"""The repo dirs holding this service's unit files, for one host."""
|
||||
dirs = svc.get("unit_dir")
|
||||
if not dirs:
|
||||
return []
|
||||
if isinstance(dirs, dict):
|
||||
if host is not None:
|
||||
return [dirs[host]] if host in dirs else []
|
||||
return [dirs[h] for h in svc["hosts"] if h in dirs]
|
||||
return [dirs]
|
||||
|
||||
def owns_unit(self, svc: dict[str, Any], host: str, unit: str, directory: str) -> bool:
|
||||
return unit in svc.get("units", []) and directory in self.unit_dirs(svc, host)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# repo facts
|
||||
# --------------------------------------------------------------------------
|
||||
@lru_cache(maxsize=1)
|
||||
def tracked_files() -> tuple[str, ...]:
|
||||
# Tracked + untracked (a check that only saw HEAD would ignore the very
|
||||
# files a work-in-progress commit is adding).
|
||||
out = subprocess.run(
|
||||
["git", "-C", str(REPO_ROOT), "ls-files", "--cached", "--others",
|
||||
"--exclude-standard"],
|
||||
text=True, stdout=subprocess.PIPE, check=True)
|
||||
return tuple(line for line in out.stdout.splitlines() if line)
|
||||
|
||||
|
||||
def _network_values(container: Path) -> list[str]:
|
||||
"""Every `Network=` value a quadlet unit declares (comma lists included)."""
|
||||
values: list[str] = []
|
||||
try:
|
||||
text = container.read_text()
|
||||
except OSError:
|
||||
return values
|
||||
for line in text.splitlines():
|
||||
if line.startswith("Network="):
|
||||
values.extend(v.strip() for v in line[len("Network="):].split(",") if v.strip())
|
||||
return values
|
||||
|
||||
|
||||
class Graph:
|
||||
"""The unit-file coupling, read out of the tree: netns + netns-net users.
|
||||
|
||||
Quadlet installs every user unit flat, so `Network=gluetun.container`
|
||||
names a unit by name, not by path; the same host is the scope, which is
|
||||
also the deployment scope.
|
||||
"""
|
||||
|
||||
def __init__(self, hosts: list[str]) -> None:
|
||||
self.units: dict[tuple[str, str], str] = {} # (host, unit) -> path
|
||||
self.networks: dict[tuple[str, str], str] = {} # (host, net) -> path
|
||||
self.netns_children: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||
self.network_users: dict[tuple[str, str], set[tuple[str, str]]] = {}
|
||||
for path in tracked_files():
|
||||
parts = PurePosixPath(path).parts
|
||||
if not parts or parts[0] not in hosts or is_trash(path):
|
||||
continue
|
||||
host, directory, name = parts[0], str(PurePosixPath(*parts[1:-1])), parts[-1]
|
||||
if name.endswith(".container"):
|
||||
self.units[(host, name[: -len(".container")])] = path
|
||||
elif name.endswith(".network"):
|
||||
self.networks[(host, name[: -len(".network")])] = path
|
||||
for (host, unit), path in self.units.items():
|
||||
for value in _network_values(REPO_ROOT / path):
|
||||
if value.endswith(".container"):
|
||||
parent = (host, value[: -len(".container")])
|
||||
self.netns_children.setdefault(parent, set()).add((host, unit))
|
||||
elif value.endswith(".network"):
|
||||
net = (host, value[: -len(".network")])
|
||||
self.network_users.setdefault(net, set()).add((host, unit))
|
||||
|
||||
def rides(self, start: list[tuple[str, str]]) -> set[tuple[str, str]]:
|
||||
"""The transitive netns closure: a unit, everything riding it, and
|
||||
everything riding those (a netns inside a netns)."""
|
||||
found: set[tuple[str, str]] = set()
|
||||
frontier = list(start)
|
||||
while frontier:
|
||||
node = frontier.pop()
|
||||
if node in found:
|
||||
continue
|
||||
found.add(node)
|
||||
frontier.extend(self.netns_children.get(node, ()))
|
||||
return found
|
||||
|
||||
|
||||
@lru_cache(maxsize=8)
|
||||
def graph_for(hosts: tuple[str, ...]) -> Graph:
|
||||
return Graph(list(hosts))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# selection
|
||||
# --------------------------------------------------------------------------
|
||||
def _owner_index(inv: Inventory) -> dict[tuple[str, str, int]]:
|
||||
"""(host, unit) -> service index, for unit-file ownership.
|
||||
|
||||
A quadlet unit name is unique per host (``check units`` enforces one
|
||||
claim), so the coupling closure can resolve a riding unit that lives in
|
||||
a DIFFERENT directory than the changed file — gluetun's netns children
|
||||
are the point.
|
||||
"""
|
||||
index: dict[tuple[str, str], int] = {}
|
||||
for svc in inv.services:
|
||||
for host in svc["hosts"]:
|
||||
for unit in svc.get("units", []):
|
||||
key = (host, unit)
|
||||
if key in index and index[key] != svc["_index"]:
|
||||
raise SystemExit(
|
||||
f"{inv.origin}: unit {unit!r} on {host} claimed by "
|
||||
f"{inv.services[index[key]]['name']} and {svc['name']}"
|
||||
)
|
||||
index[key] = svc["_index"]
|
||||
return index
|
||||
|
||||
|
||||
def select(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||
"""Canonical-order service names the changed paths select for one host."""
|
||||
if host not in inv.hosts:
|
||||
raise SystemExit(f"{inv.origin}: no host {host!r}")
|
||||
graph = graph_for(tuple(inv.hosts))
|
||||
owners = _owner_index(inv)
|
||||
hits: set[int] = set()
|
||||
|
||||
def add(index: int) -> None:
|
||||
hits.add(index)
|
||||
|
||||
for raw in paths:
|
||||
path = raw.replace("\\", "/")
|
||||
parts = PurePosixPath(path).parts
|
||||
if not parts or is_trash(path) or path in NON_RUNTIME_PATHS:
|
||||
continue
|
||||
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||
continue
|
||||
head = parts[0]
|
||||
if head in NON_RUNTIME_ROOTS:
|
||||
continue
|
||||
# A host's directory is the scope: a frankbook-pro file can never
|
||||
# select a dankserver service.
|
||||
scoped_host = head if head in inv.hosts else None
|
||||
if scoped_host is not None and scoped_host != host:
|
||||
continue
|
||||
|
||||
name = parts[-1]
|
||||
directory = str(PurePosixPath(*parts[1:-1])) if len(parts) > 1 else ""
|
||||
|
||||
if scoped_host and name.endswith(UNIT_SUFFIXES):
|
||||
stem = name.rsplit(".", 1)[0]
|
||||
key = (scoped_host, stem)
|
||||
if name.endswith(".network") and key in graph.networks:
|
||||
users = graph.network_users.get(key, set())
|
||||
for unit in graph.rides(sorted(users)):
|
||||
index = owners.get(unit)
|
||||
if index is not None:
|
||||
add(index)
|
||||
elif key in graph.units:
|
||||
for unit in graph.rides([key]):
|
||||
index = owners.get(unit)
|
||||
if index is not None:
|
||||
add(index)
|
||||
|
||||
for index in list(hits):
|
||||
for dependent in inv.services[index].get("dependents", []):
|
||||
for svc in inv.named(host, dependent):
|
||||
add(svc["_index"])
|
||||
|
||||
for svc in inv.on(host):
|
||||
if path in svc.get("files", []):
|
||||
add(svc["_index"])
|
||||
if path in svc.get("retires", []):
|
||||
add(svc["_index"])
|
||||
for directory_rule in svc.get("dirs", []):
|
||||
if path == directory_rule or path.startswith(directory_rule + "/"):
|
||||
add(svc["_index"])
|
||||
|
||||
return [svc["name"] for svc in sorted((inv.services[i] for i in hits),
|
||||
key=lambda s: s["_index"])]
|
||||
|
||||
|
||||
def seams_for(inv: Inventory, host: str, paths: list[str]) -> list[str]:
|
||||
"""The Bats seams the selected services require, deduplicated, in order."""
|
||||
seams: list[str] = []
|
||||
for name in select(inv, host, paths):
|
||||
for svc in inv.named(host, name):
|
||||
for seam in svc.get("tests", []):
|
||||
if seam not in seams:
|
||||
seams.append(seam)
|
||||
if seams and "tests/deployment_scoping.bats" not in seams:
|
||||
seams.append("tests/deployment_scoping.bats")
|
||||
return seams
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# renderers + facts
|
||||
# --------------------------------------------------------------------------
|
||||
def render_pihole_hosts(inv: Inventory) -> str:
|
||||
edge = inv.edge_ip()
|
||||
lines = [
|
||||
"# A records for the flat franksdank.cloud scheme (dnsmasq addn-hosts",
|
||||
"# format: <IP> <name>). RENDERED — do not edit by hand:",
|
||||
"#",
|
||||
"# scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts",
|
||||
"#",
|
||||
"# The source of truth is the `dns:` field of shared/services.yml; the",
|
||||
"# drift gate in tests/service_inventory.bats fails when this file and",
|
||||
"# the inventory disagree. Caddy is consolidated on Dankclaw, so every",
|
||||
"# name — including the services running on dankserver and",
|
||||
"# frankbook-pro — resolves at the Dankclaw edge.",
|
||||
"#",
|
||||
"# dnsmasq strips ONLY whole-line comments in addn-hosts: a trailing",
|
||||
"# comment on a data line becomes a HOSTNAME token. No inline notes",
|
||||
"# here; the service attribution lives in the inventory.",
|
||||
"",
|
||||
]
|
||||
for svc in inv.services:
|
||||
for name in svc.get("dns", []):
|
||||
lines.append(f"{edge} {name}")
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def scrape_facts(inv: Inventory) -> list[dict[str, str]]:
|
||||
facts = []
|
||||
for svc in inv.services:
|
||||
for job in svc.get("scrape", []):
|
||||
facts.append({"job": job["job"], "target": job["target"],
|
||||
"metrics_path": job.get("metrics_path", "/")})
|
||||
return facts
|
||||
|
||||
|
||||
def prometheus_facts(path: Path) -> list[dict[str, str]]:
|
||||
doc = yaml.safe_load(path.read_text())
|
||||
facts = []
|
||||
for job in doc.get("scrape_configs", []):
|
||||
for sc in job.get("static_configs", []):
|
||||
for target in sc.get("targets", []):
|
||||
facts.append({"job": job["job_name"], "target": target,
|
||||
"metrics_path": job.get("metrics_path", "/")})
|
||||
return facts
|
||||
|
||||
|
||||
def probe_facts(inv: Inventory) -> list[str]:
|
||||
facts: list[str] = []
|
||||
for svc in inv.services:
|
||||
probe = svc.get("probe")
|
||||
if isinstance(probe, dict):
|
||||
facts.extend(str(v) for v in probe.values())
|
||||
elif probe:
|
||||
facts.append(str(probe))
|
||||
return facts
|
||||
|
||||
|
||||
def homepage_probes(path: Path) -> list[str]:
|
||||
return sorted({s["probe"] for s in json.loads(path.read_text()) if s.get("probe")})
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# checks
|
||||
# --------------------------------------------------------------------------
|
||||
def check_ownership(inv: Inventory, graph: Graph) -> list[str]:
|
||||
"""Every repo file belongs to a service, a rule, or the unowned list."""
|
||||
problems: list[str] = []
|
||||
exact: set[str] = set()
|
||||
directories: set[str] = set()
|
||||
for svc in inv.services:
|
||||
for host in svc["hosts"]:
|
||||
for directory in inv.unit_dirs(svc, host):
|
||||
for unit in svc.get("units", []):
|
||||
exact.add(f"{directory}/{unit}.container")
|
||||
for field in ("files", "retires"):
|
||||
exact.update(svc.get(field, []))
|
||||
directories.update(svc.get("dirs", []))
|
||||
# A shared network file is claimed by the units that ride it; the unit
|
||||
# graph is the referrer, and check_units proves every reference resolves.
|
||||
exact.update(graph.networks.values())
|
||||
|
||||
def is_claimed(path: str) -> bool:
|
||||
if path in exact:
|
||||
return True
|
||||
first = PurePosixPath(path).parts[0]
|
||||
if first in NON_RUNTIME_ROOTS or path in NON_RUNTIME_PATHS or is_trash(path):
|
||||
return True
|
||||
if PurePosixPath(path).name in NON_RUNTIME_NAMES:
|
||||
return True
|
||||
for owned in directories | set(inv.unowned):
|
||||
if path == owned or path.startswith(owned + "/"):
|
||||
return True
|
||||
return False
|
||||
|
||||
for path in tracked_files():
|
||||
if not is_claimed(path):
|
||||
problems.append(f"file belongs to no service and is not in unowned: {path}")
|
||||
|
||||
# The reverse direction: an unowned entry that names nothing is stale.
|
||||
for directory in inv.unowned:
|
||||
if not any(p == directory or p.startswith(directory + "/") for p in tracked_files()):
|
||||
problems.append(f"unowned entry names no tracked file: {directory}")
|
||||
|
||||
# Every inventory file/dir claim names a tracked file.
|
||||
for svc in inv.services:
|
||||
# `retires` names a file whose DELETION selects the service, so by
|
||||
# definition it is not in the tree.
|
||||
for field in ("files", "dirs"):
|
||||
for entry in svc.get(field, []):
|
||||
if entry not in tracked_files() and not any(
|
||||
p.startswith(entry + "/") for p in tracked_files()):
|
||||
problems.append(f"{svc['name']}: {field} entry names no tracked file: {entry}")
|
||||
del graph
|
||||
return problems
|
||||
|
||||
|
||||
def check_units(inv: Inventory, graph: Graph) -> list[str]:
|
||||
"""Every unit file has exactly one owner; every unit reference resolves."""
|
||||
problems: list[str] = []
|
||||
owners: dict[str, list[str]] = {}
|
||||
for svc in inv.services:
|
||||
for host in svc["hosts"]:
|
||||
for directory in inv.unit_dirs(svc, host):
|
||||
for unit in svc.get("units", []):
|
||||
path = f"{directory}/{unit}.container"
|
||||
owners.setdefault(path, []).append(f"{svc['name']}@{host}")
|
||||
if path not in tracked_files():
|
||||
problems.append(f"{svc['name']}: unit file is not tracked: {path}")
|
||||
for path in tracked_files():
|
||||
if not path.endswith(".container") or is_trash(path):
|
||||
continue
|
||||
found = owners.get(path, [])
|
||||
if not found:
|
||||
problems.append(f"unit file claimed by no service: {path}")
|
||||
elif len(found) > 1:
|
||||
problems.append(f"unit file claimed by {len(found)} services: {path}: {found}")
|
||||
|
||||
# Root units (scripts/*.service) are claimed by exactly one `files:` list.
|
||||
for path in tracked_files():
|
||||
if not (path.startswith("scripts/") and path.endswith(".service")) or is_trash(path):
|
||||
continue
|
||||
found = [s["name"] for s in inv.services if path in s.get("files", [])]
|
||||
if not found:
|
||||
problems.append(f"root unit claimed by no service: {path}")
|
||||
elif len(found) > 1:
|
||||
problems.append(f"root unit claimed by {found}: {path}")
|
||||
|
||||
# Every Network= reference in a unit file names a file on the same host.
|
||||
for (host, unit), path in graph.units.items():
|
||||
for value in _network_values(REPO_ROOT / path):
|
||||
if value.endswith(".container"):
|
||||
target = (host, value[: -len(".container")])
|
||||
if target not in graph.units:
|
||||
problems.append(f"{path}: Network={value} names no unit on {host}")
|
||||
elif value.endswith(".network"):
|
||||
target = (host, value[: -len(".network")])
|
||||
if target not in graph.networks:
|
||||
problems.append(f"{path}: Network={value} names no network on {host}")
|
||||
return problems
|
||||
|
||||
|
||||
def check_config(inv: Inventory) -> list[str]:
|
||||
problems: list[str] = []
|
||||
for svc in inv.services:
|
||||
label = f"{'+'.join(svc['hosts'])}/{svc['name']}"
|
||||
for key in REQUIRED_KEYS:
|
||||
if key not in svc:
|
||||
problems.append(f"{label}: missing required key {key}")
|
||||
if svc.get("kind") not in KINDS:
|
||||
problems.append(f"{label}: kind {svc.get('kind')!r} not one of {sorted(KINDS)}")
|
||||
unknown = set(svc.get("hosts", [])) - set(inv.hosts)
|
||||
if unknown:
|
||||
problems.append(f"{label}: undeclared hosts: {sorted(unknown)}")
|
||||
if svc.get("units") and not svc.get("unit_dir"):
|
||||
problems.append(f"{label}: units declared without unit_dir")
|
||||
if svc.get("deployed_by") and not inv.any_named(svc["deployed_by"]):
|
||||
problems.append(f"{label}: deployed_by names no service: {svc['deployed_by']}")
|
||||
if (svc.get("health") and svc.get("wait") is None) or \
|
||||
(svc.get("wait") is not None and not svc.get("health")):
|
||||
problems.append(f"{label}: health and wait are a pair")
|
||||
for seam in svc.get("tests", []):
|
||||
if seam not in tracked_files():
|
||||
problems.append(f"{label}: seam does not exist: {seam}")
|
||||
for name in svc.get("dns", []):
|
||||
if not str(name).replace(".", "").replace("-", "").isalnum():
|
||||
problems.append(f"{label}: dns name is not a plain hostname: {name}")
|
||||
if svc.get("kind") == "root-unit" and svc.get("unit_dir"):
|
||||
problems.append(f"{label}: a root unit lives in scripts/, not unit_dir")
|
||||
seen: set[str] = set()
|
||||
for svc in inv.services:
|
||||
key = (svc["name"], tuple(svc["hosts"]))
|
||||
if key in seen:
|
||||
problems.append(f"duplicate inventory entry: {svc['name']} on {svc['hosts']}")
|
||||
seen.add(key)
|
||||
return problems
|
||||
|
||||
|
||||
def check_drift(inv: Inventory) -> list[str]:
|
||||
problems: list[str] = []
|
||||
rendered = render_pihole_hosts(inv)
|
||||
hosts_file = REPO_ROOT / "dankclaw" / "pihole" / "addn-hosts"
|
||||
if hosts_file.read_text() != rendered:
|
||||
problems.append("dankclaw/pihole/addn-hosts is not the render of the inventory "
|
||||
"(scripts/service-map.py render pihole-hosts > dankclaw/pihole/addn-hosts)")
|
||||
|
||||
want = sorted(json.dumps(f, sort_keys=True) for f in scrape_facts(inv))
|
||||
have = sorted(json.dumps(f, sort_keys=True) for f in
|
||||
prometheus_facts(REPO_ROOT / "shared" / "prometheus" / "prometheus.yml"))
|
||||
for extra in [f for f in have if f not in want]:
|
||||
problems.append(f"prometheus scrapes a target the inventory does not declare: {extra}")
|
||||
for missing in [f for f in want if f not in have]:
|
||||
problems.append(f"inventory declares a scrape prometheus.yml lacks: {missing}")
|
||||
|
||||
probes_want = sorted(set(probe_facts(inv)))
|
||||
probes_have = homepage_probes(REPO_ROOT / "shared" / "homepage" / "services.json")
|
||||
for extra in [p for p in probes_have if p not in probes_want]:
|
||||
problems.append(f"homepage probes a target the inventory does not declare: {extra}")
|
||||
for missing in [p for p in probes_want if p not in probes_have]:
|
||||
problems.append(f"inventory declares a probe homepage does not list: {missing}")
|
||||
return problems
|
||||
|
||||
|
||||
CHECKS = {
|
||||
"config": lambda inv, graph: check_config(inv),
|
||||
"units": check_units,
|
||||
"ownership": check_ownership,
|
||||
"drift": lambda inv, graph: check_drift(inv),
|
||||
}
|
||||
|
||||
|
||||
def check(inv: Inventory, names: list[str]) -> list[str]:
|
||||
graph = graph_for(tuple(inv.hosts))
|
||||
problems: list[str] = []
|
||||
for name in names or list(CHECKS):
|
||||
problems.extend(CHECKS[name](inv, graph))
|
||||
return problems
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# CLI
|
||||
# --------------------------------------------------------------------------
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
parser.add_argument("--inventory", type=Path, default=DEFAULT_INVENTORY)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
sub.add_parser("hosts")
|
||||
p = sub.add_parser("services")
|
||||
p.add_argument("--host")
|
||||
p = sub.add_parser("allows")
|
||||
p.add_argument("--host", required=True)
|
||||
p.add_argument("--service", required=True)
|
||||
p = sub.add_parser("plan")
|
||||
p.add_argument("--host", required=True)
|
||||
p.add_argument("paths", nargs="+")
|
||||
p = sub.add_parser("seams")
|
||||
p.add_argument("--host")
|
||||
p.add_argument("paths", nargs="+")
|
||||
p = sub.add_parser("owner")
|
||||
p.add_argument("--host", required=True)
|
||||
p.add_argument("path")
|
||||
p = sub.add_parser("units")
|
||||
p.add_argument("--host", required=True)
|
||||
p.add_argument("--service", required=True)
|
||||
p = sub.add_parser("stack")
|
||||
p.add_argument("--host", required=True)
|
||||
p.add_argument("--service", required=True)
|
||||
p = sub.add_parser("get")
|
||||
p.add_argument("--host")
|
||||
p.add_argument("--service", required=True)
|
||||
p.add_argument("field")
|
||||
p = sub.add_parser("render")
|
||||
p.add_argument("artifact", choices=("pihole-hosts",))
|
||||
p = sub.add_parser("check")
|
||||
p.add_argument("names", nargs="*")
|
||||
|
||||
args = parser.parse_args(argv)
|
||||
inv = Inventory.load(args.inventory)
|
||||
|
||||
if args.command == "hosts":
|
||||
print("\n".join(inv.hosts))
|
||||
elif args.command == "services":
|
||||
services = inv.on(args.host) if args.host else inv.services
|
||||
seen: list[str] = []
|
||||
for svc in services:
|
||||
if svc["name"] not in seen:
|
||||
seen.append(svc["name"])
|
||||
print("\n".join(seen))
|
||||
elif args.command == "allows":
|
||||
return 0 if inv.named(args.host, args.service) else 1
|
||||
elif args.command == "plan":
|
||||
print("\n".join(select(inv, args.host, args.paths)))
|
||||
elif args.command == "seams":
|
||||
if args.host is not None:
|
||||
seams = seams_for(inv, args.host, args.paths)
|
||||
else:
|
||||
# No host: the union across every host (the dev machine's
|
||||
# pre-commit mapper has no deployment target).
|
||||
seams = []
|
||||
for host in inv.hosts:
|
||||
for seam in seams_for(inv, host, args.paths):
|
||||
if seam not in seams:
|
||||
seams.append(seam)
|
||||
print("\n".join(seams))
|
||||
elif args.command == "owner":
|
||||
found = select(inv, args.host, [args.path])
|
||||
print(found[-1] if found else "")
|
||||
elif args.command == "units":
|
||||
matches = inv.named(args.host, args.service)
|
||||
if not matches:
|
||||
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||
return 2
|
||||
for svc in matches:
|
||||
for directory in inv.unit_dirs(svc, args.host):
|
||||
for unit in svc.get("units", []):
|
||||
print(f"{unit} {directory}")
|
||||
elif args.command == "stack":
|
||||
matches = inv.named(args.host, args.service)
|
||||
if not matches:
|
||||
print(f"no service {args.service} on {args.host}", file=sys.stderr)
|
||||
return 2
|
||||
owner = matches[0].get("deployed_by") or args.service
|
||||
print(owner)
|
||||
for svc in inv.on(args.host):
|
||||
if svc.get("deployed_by") == owner and svc["name"] != owner:
|
||||
print(svc["name"])
|
||||
elif args.command == "get":
|
||||
matches = inv.on(args.host) if args.host else inv.services
|
||||
matches = [s for s in matches if s["name"] == args.service]
|
||||
if not matches:
|
||||
print(f"no service {args.service}" + (f" on {args.host}" if args.host else ""),
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
values = [s.get(args.field) for s in matches if args.field in s]
|
||||
print(json.dumps(values[0] if len(values) == 1 else values))
|
||||
elif args.command == "render":
|
||||
sys.stdout.write(render_pihole_hosts(inv))
|
||||
elif args.command == "check":
|
||||
problems = check(inv, args.names)
|
||||
if problems:
|
||||
for problem in problems:
|
||||
print(f"FAIL: {problem}", file=sys.stderr)
|
||||
return 1
|
||||
print("OK: the service inventory is sound (schema, units, ownership, drift)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+57
-190
@@ -62,234 +62,101 @@ declare -a TESTS=()
|
||||
add_test() {
|
||||
local test="$1"
|
||||
local existing
|
||||
MAPPED=1
|
||||
for existing in "${TESTS[@]:-}"; do
|
||||
[ "$existing" != "$test" ] || return 0
|
||||
done
|
||||
TESTS+=("$test")
|
||||
}
|
||||
|
||||
# The path->test mapping has two layers:
|
||||
# 1. cross-cutting seams that no single service owns (compose validity,
|
||||
# env consistency, the deployer itself, docs, syntax);
|
||||
# 2. the service inventory (shared/services.yml via service-map.py
|
||||
# "seams"): every service the path selects on ANY host contributes its
|
||||
# declared tests. Service seams are the services' own — the former
|
||||
# shotgun fallbacks (every dankclaw quadlet unit ran the open-webui,
|
||||
# pihole, and searxng seams) are retired; the inventory names each
|
||||
# service's real seams, and adding a service is a data edit.
|
||||
for path in "${CHANGED[@]}"; do
|
||||
MAPPED=0
|
||||
case "$path" in
|
||||
tests/*.bats)
|
||||
add_test "$path"
|
||||
continue
|
||||
;;
|
||||
tests/helpers.bash|tests/run.sh)
|
||||
die "$path is shared test infrastructure; run ./tests/run.sh explicitly"
|
||||
;;
|
||||
.githooks/*)
|
||||
add_test tests/changed_tests.bats
|
||||
;;
|
||||
dankserver/compose.yaml|dankclaw/compose.yaml)
|
||||
add_test tests/compose_validity.bats
|
||||
add_test tests/env_consistency.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankserver/Caddyfile|dankclaw/Caddyfile)
|
||||
add_test tests/caddyfile.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
tests/e2e/*)
|
||||
add_test tests/e2e_scaffold.bats
|
||||
continue
|
||||
;;
|
||||
.githooks/*)
|
||||
add_test tests/changed_tests.bats
|
||||
continue
|
||||
;;
|
||||
# Archive material and host-level facts: a path under any
|
||||
# .AGENTTRASH/ directory is retired-by-policy (the planner plans
|
||||
# nothing from it), udev rules are applied by manual sudo install,
|
||||
# reports are frozen artifacts, and ignore-lists change nothing at
|
||||
# runtime. They select no focused test; without this case they would
|
||||
# trip the mapping gate below.
|
||||
.AGENTTRASH/*|*/.AGENTTRASH/*|dankclaw/udev/*|report/*|.gitignore|.env.example)
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
|
||||
# Cross-cutting seams.
|
||||
case "$path" in
|
||||
dankserver/.env.example|dankclaw/.env.example)
|
||||
add_test tests/env_consistency.bats
|
||||
;;
|
||||
shared/homepage/*)
|
||||
add_test tests/homepage_caddy.bats
|
||||
add_test tests/homepage_config.bats
|
||||
add_test tests/homepage_deploy.bats
|
||||
add_test tests/homepage_pages.bats
|
||||
add_test tests/homepage_checker.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
shared/searxng/*)
|
||||
add_test tests/searxng_limiter.bats
|
||||
add_test tests/searxng_quadlet.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# The five observability containers left compose for Quadlet units on
|
||||
# 2026-09-11. Their seam pins boot persistence and the loopback-only
|
||||
# publish posture that keeps them off every host interface. This case
|
||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
||||
# would otherwise swallow it into unrelated seams.
|
||||
dankclaw/observability/*)
|
||||
add_test tests/observability_quadlet.bats
|
||||
dankserver/Caddyfile|dankclaw/Caddyfile)
|
||||
add_test tests/caddyfile.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
add_test tests/docs_completeness.bats
|
||||
;;
|
||||
# Caddy left compose.yaml on 2026-08-26 for a quadlet; caddyfile.bats
|
||||
# now asserts against the quadlet itself. Route every change under
|
||||
# dankclaw/caddy/ to both seams (this block must precede the
|
||||
# dankclaw/*.container glob, which would otherwise swallow the unit).
|
||||
dankclaw/caddy/*)
|
||||
add_test tests/caddy_quadlet.bats
|
||||
add_test tests/caddyfile.bats
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankclaw/*.container|dankclaw/*.network)
|
||||
add_test tests/open_webui_quadlet.bats
|
||||
add_test tests/pihole.bats
|
||||
add_test tests/searxng_quadlet.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# Anubis is a quadlet + Caddyfile pair: the chain (public listener ->
|
||||
# anubis:8923 -> internal :8016 listener) spans both, so any change
|
||||
# under the service dir must re-run the caddyfile seam too. This case
|
||||
# precedes the dankclaw/*/quadlet/*.container fallback below, which
|
||||
# would otherwise swallow it into unrelated seams.
|
||||
dankclaw/anubis/*)
|
||||
add_test tests/anubis.bats
|
||||
add_test tests/caddyfile.bats
|
||||
add_test tests/homepage_caddy.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# frank-dragoon is a rootless quadlet again since 2026-09-16 (the
|
||||
# halogen-flash-server restore). Its unit carries the fleet's GPU and
|
||||
# podman posture, so it gets its own seam rather than the generic
|
||||
# fallback below.
|
||||
dankclaw/frank-dragoon/*)
|
||||
add_test tests/frank_dragoon_unit.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
add_test tests/docs_completeness.bats
|
||||
;;
|
||||
# Quadlets live under dankclaw/<svc>/quadlet/ (reorganized out of the
|
||||
# top-level dankclaw/*.container globs above). Map the whole subtree so
|
||||
# changes to the pihole, open-webui, searxng, and gitea quadlets hit
|
||||
# their seam tests. The pihole-specific block must come first: shell
|
||||
# case matches the first pattern, and dankclaw/pihole/quadlet/*.container
|
||||
# also satisfies the dankclaw/*/quadlet/*.container fallback.
|
||||
dankclaw/pihole/*)
|
||||
add_test tests/pihole.bats
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankclaw/*/quadlet/*.container|dankclaw/*/quadlet/*.network)
|
||||
add_test tests/open_webui_quadlet.bats
|
||||
add_test tests/pihole.bats
|
||||
add_test tests/searxng_quadlet.bats
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
scripts/deploy.sh|scripts/deploy-remote.sh|scripts/deploy-plan.py|scripts/deploy-*-quadlet.sh)
|
||||
add_test tests/deployment_flow.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
scripts/service-map.py|shared/services.yml)
|
||||
add_test tests/service_inventory.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
scripts/lib/*.sh)
|
||||
add_test tests/llm_gates.bats
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
serverAppConfigs-docs/*|README.md|AGENTS.md)
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
scripts/mount-entropy.sh)
|
||||
add_test tests/mount_entropy.bats
|
||||
;;
|
||||
# Archive material: a path under any .AGENTTRASH/ directory is
|
||||
# retired-by-policy (deploy-plan.py plans nothing from it), so it
|
||||
# selects no focused test. It still needs its own case: without one
|
||||
# the wrapper refuses retirement commits, whose trash moves show up
|
||||
# as additions, with "no focused-test mapping".
|
||||
.AGENTTRASH/*|*/.AGENTTRASH/*)
|
||||
;;
|
||||
# frank-archon is llama.cpp (the unsloth b11030-mix prebuilt) on the
|
||||
# two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
|
||||
# Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
|
||||
# live in scripts/, which is why this case has to precede the generic
|
||||
# scripts/*.service and scripts/* fallbacks below.
|
||||
scripts/frank-archon.*|scripts/serve-frank-archon.sh)
|
||||
add_test tests/frank_archon_unit.bats
|
||||
add_test tests/llm_units.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
# frank-dragoon's chat template lives in scripts/ (moved there on
|
||||
# 2026-09-26 when its old home was retired): a template edit changes
|
||||
# what the live endpoint renders, so it selects the large unit seam.
|
||||
scripts/qwen-sharp-v22.4.0.jinja)
|
||||
add_test tests/frank_dragoon_unit.bats
|
||||
;;
|
||||
# frank-zealot is llama.cpp on the 3090 as a ROOT unit: a native CUDA
|
||||
# binary, so no generated quadlet is involved. Its env, unit and serve
|
||||
# script all live in scripts/, which is why this case has to precede
|
||||
# the generic scripts/*.service and scripts/* fallbacks below.
|
||||
scripts/frank-zealot.*|scripts/serve-frank-zealot.sh)
|
||||
add_test tests/frank_zealot_unit.bats
|
||||
add_test tests/llm_units.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
old/scripts/*)
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
scripts/*.service)
|
||||
add_test tests/llm_units.bats
|
||||
add_test tests/scripts_syntax.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
scripts/*)
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
serverAppConfigs-docs/*)
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankclaw/hermes/*)
|
||||
add_test tests/docs_completeness.bats
|
||||
;;
|
||||
README.md|AGENTS.md)
|
||||
# Project guidance changes can affect the documented deployment
|
||||
# contract; validate the docs vault and scoped deployment seams.
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankclaw/comfyui/*)
|
||||
add_test tests/docs_completeness.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# loki and alloy images are built from this repo's Containerfiles; the
|
||||
# loki one carries the healthcheck command (see its comment), so both
|
||||
# seams read it.
|
||||
dankclaw/loki/*|dankclaw/alloy/*)
|
||||
add_test tests/observability_quadlet.bats
|
||||
add_test tests/caddyfile.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankclaw/udev/*)
|
||||
# Host-level udev rules are applied by manual sudo install + udevadm
|
||||
# trigger, not dispatched as a service; nothing here is bats-testable.
|
||||
;;
|
||||
.AGENTTRASH/*)
|
||||
# Trash moves are not testable; skip.
|
||||
;;
|
||||
.gitignore|.env.example)
|
||||
# Ignore/ignore-list changes change nothing at runtime; not testable.
|
||||
;;
|
||||
old/scripts/*|old/scripts/**)
|
||||
add_test tests/scripts_syntax.bats
|
||||
;;
|
||||
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
|
||||
# copyparty, plex, tautulli): the seam checks boot-persistence +
|
||||
# v1-generator contract; deploy-flow seams guard the deployer routing.
|
||||
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
|
||||
add_test tests/dankserver_quadlet.bats
|
||||
add_test tests/deployment_flow.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# FrankBook-Pro units live inside the podman machine VM; the seam
|
||||
# pins boot persistence, the gluetun netns invariant, and the
|
||||
# plan-locked volume layout. Deployer changes route here too.
|
||||
frankbook-pro/*)
|
||||
add_test tests/frankbook_quadlet.bats
|
||||
add_test tests/deployment_flow.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
# Seerr unit on dankclaw: boot persistence, loopback-only port
|
||||
# posture, and the deployer routing that installs it.
|
||||
dankclaw/seerr/*)
|
||||
add_test tests/seerr_quadlet.bats
|
||||
add_test tests/deployment_flow.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
report/*)
|
||||
# Frozen audit reports are point-in-time artifacts; not testable.
|
||||
;;
|
||||
*)
|
||||
die "no focused-test mapping for '$path'; add one before committing"
|
||||
;;
|
||||
esac
|
||||
|
||||
# The inventory's declared seams for every service this path selects.
|
||||
while IFS= read -r seam; do
|
||||
[ -n "$seam" ] && add_test "$seam"
|
||||
done < <(python3 "$REPO_ROOT/scripts/service-map.py" seams -- "$path" 2>/dev/null || true)
|
||||
|
||||
# Mapping gate: a path that selected no test at all — no service seam,
|
||||
# no cross-cutting rule — is an unmapped area of the repo; refuse to
|
||||
# commit it silently (the old catch-all `die`).
|
||||
if [ "$MAPPED" -eq 0 ]; then
|
||||
die "no focused-test mapping for '$path'; map it in the inventory (shared/services.yml) before committing"
|
||||
fi
|
||||
done
|
||||
|
||||
# Mapped-but-untestable paths (udev rules, trash moves) legitimately produce an
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
type: reference
|
||||
title: Service Inventory
|
||||
description: shared/services.yml is the one inventory of deployable services; scripts/service-map.py is its only reader. The planner, both deploy allow-lists, the changed-test seam mapper, the Pi-hole hosts render, the Prometheus scrape facts, and the homepage probes all derive from it.
|
||||
tags: [deployment, inventory, planner, quadlet, prometheus, pihole, homepage]
|
||||
timestamp: 2026-10-13T00:00:00-07:00
|
||||
---
|
||||
|
||||
# Service Inventory
|
||||
|
||||
Before 2026-10-13 the same facts lived in four hand-maintained lists — the
|
||||
planner's path rules (`deploy-plan.py`), the `--service` allow-list in
|
||||
`deploy.sh`, the near-copy in `deploy-remote.sh`, and the changed-test mapper
|
||||
(`test-changed.sh`) — plus per-fact copies in `prometheus.yml`,
|
||||
`addn-hosts`, and `services.json`. They drifted on every swap (the planner
|
||||
still mapped the retired `qwen-sharp` template to frank-dragoon; the allow-list
|
||||
still listed the *arr stack under Dankserver after it moved to FrankBook-Pro).
|
||||
Now there is **one inventory and one reader**:
|
||||
|
||||
- `shared/services.yml` — the data: every deployable service, its hosts, unit
|
||||
dir(s), unit files, path claims, tests, scrape/DNS/probe facts, aliases.
|
||||
- `scripts/service-map.py` — the only reader. CLI subcommands:
|
||||
`hosts`, `services`, `allows`, `plan`, `seams`, `owner`, `units`, `stack`,
|
||||
`get`, `render`, `check`.
|
||||
|
||||
## 1. Consumers
|
||||
|
||||
|Consumer|Call|
|
||||
|---|---|
|
||||
|`scripts/deploy-plan.py`|thin wrapper → `plan --host TARGET [paths...]` (keeps the `--repo/--base/--head` and `--paths` CLIs)|
|
||||
|`scripts/deploy.sh` (`--service` gate)|`allows --host T --service S`|
|
||||
|`scripts/deploy-remote.sh` (`service_is_allowed`)|same|
|
||||
|`scripts/test-changed.sh`|`seams [paths...]` — a service's declared `tests` are its seams|
|
||||
|`dankclaw/pihole/addn-hosts`|`render pihole-hosts` — the tracked file is a rendered artifact|
|
||||
|Prometheus scrape jobs, LAN DNS, homepage probes|`check` compares the tracked configs against the inventory's `scrape`/`dns`/`probe` fields (drift gate)|
|
||||
|
||||
Selection rules the engine encodes (see `tests/fixtures/service-plan-golden.tsv`,
|
||||
the pinned characterization of every tracked path × every target):
|
||||
|
||||
- **Host scoping** — a host's top-level directory can only select that host's
|
||||
services. `frankbook-pro/**` plans nothing on Dankserver (the old planner
|
||||
had a stale Dankserver *arr list).
|
||||
- **Quadlet coupling graph** — `Network=x.container` rides x's netns;
|
||||
`Network=y.network` joins a shared network. Changing the parent (or the
|
||||
network file) selects every rider transitively, in their own directories
|
||||
(`dankclaw/searxng/quadlet/gluetun.container` → gluetun + searxng + marreta;
|
||||
`frankbook-pro/gluetun/quadlet/gluetun.container` → the whole arr stack).
|
||||
- **Stacks** — a service's `stack` members deploy and restart as one handler
|
||||
(romm trio, hermes trio).
|
||||
- **Retirement** — removing a service from `services.yml` removes it from the
|
||||
planner, both allow-lists, and the seam mapper in one data edit; a pending
|
||||
deploy-state entry for it is auto-triaged (see [[Deploy State Triage]]).
|
||||
|
||||
## 2. Integrity gates (`check`)
|
||||
|
||||
`python3 scripts/service-map.py check` runs, and
|
||||
`tests/service_inventory.bats` pins:
|
||||
|
||||
- **schema** — required fields per entry, known keys only;
|
||||
- **units** — every unit file named by an entry exists under its `unit_dir`
|
||||
on the right host; each unit name is claimed by exactly one service per host;
|
||||
- **ownership** — every tracked quadlet `.container`/`.network` file is claimed
|
||||
by exactly one service (trash excluded);
|
||||
- **paths** — every claimed `files`/`dirs`/`retires` glob matches ≥1 tracked file;
|
||||
- **compose** — the container names in each host's `compose.yaml` have exactly
|
||||
one quadlet owner (the [[Quadlet Lifecycle]] two-owners trap);
|
||||
- **drift** — `addn-hosts` equals its render; the Prometheus job set, the DNS
|
||||
names, and the homepage probe ports equal the inventory's fields.
|
||||
|
||||
## 3. Adding, moving, retiring a service
|
||||
|
||||
- **Add**: append an entry (name, hosts, kind, unit_dir, units, paths, tests,
|
||||
…), add the quadlet file, run `check` — `plan`/`allows`/`seams` pick it up
|
||||
with zero script edits.
|
||||
- **Move between hosts**: edit `hosts` (and per-host `unit_dir`/`probe` maps).
|
||||
- **Retire**: delete the entry, move files to `.AGENTTRASH/` (see
|
||||
[[Testing Policy]] for the seam the mapper applies to trash paths).
|
||||
|
||||
Field notes: `alias` carries deployable names that differ from the service
|
||||
name (`calibre-web` deploys the unit `calibre-server`); `probe` for multi-host
|
||||
services is a `{host: port}` map; `dependents` names services whose config
|
||||
embeds a reference to this one (config-only coupling — a `.env` edit selects
|
||||
both, a unit edit selects the graph closure).
|
||||
|
||||
## 4. Known cutover deltas (2026-10-13)
|
||||
|
||||
The 17 deltas against the old planner are all in these classes: anubis paths
|
||||
now select `anubis` (the old planner had no rule at all); cross-host paths
|
||||
select nothing; `scripts/qwen-sharp-v22.4.0.jinja` (retired template) plans
|
||||
nothing while `scripts/qwen38-froggeric-v22.5.jinja` (the live frank-archon
|
||||
template, previously unmapped) plans frank-archon;
|
||||
`dankserver/romm/romm.container` (top-level quadlet dir the old unit-file rule
|
||||
missed) now plans the romm trio.
|
||||
|
||||
---
|
||||
|
||||
Related: [[Deployment Flow]], [[Testing Policy]], [[Quadlet Lifecycle]],
|
||||
[[Container Network Namespaces]], [[Deploy State Triage]]
|
||||
@@ -28,6 +28,7 @@ covers both workloads).
|
||||
- [[Network Architecture]] — Caddy reverse proxy, hostname routing, Gluetun VPN path
|
||||
- [[Storage Policy]] — bind mounts, data locations, ownership
|
||||
- [[Deployment Flow]] — service-scoped git deploy, target map, and health checks
|
||||
- [[Service Inventory]] — shared/services.yml, the one inventory of deployable services, and service-map.py
|
||||
- [[Testing Policy]] — the test seams and how to run them
|
||||
- [[E2E Suite]] — the Playwright suite that drives the live guard chain
|
||||
- [[Container Network Namespaces]] — which local services are addressable from each Podman network namespace on Dankclaw
|
||||
|
||||
@@ -0,0 +1,598 @@
|
||||
# =============================================================================
|
||||
# services.yml — THE service inventory.
|
||||
#
|
||||
# One entry per deployable service, in canonical deploy order (dependencies
|
||||
# before consumers, the LLM fleet last). Four hand-maintained lists used to
|
||||
# agree with each other by discipline alone; every one of them now READS this
|
||||
# file:
|
||||
#
|
||||
# scripts/deploy-plan.py which services a changed path selects
|
||||
# scripts/deploy-remote.sh which services a target accepts (allow-list)
|
||||
# scripts/test-changed.sh which Bats seams a changed path must re-run
|
||||
# scripts/service-map.py renderers + drift gates (`check`)
|
||||
#
|
||||
# `service-map.py check` is the referee: every quadlet unit file and every
|
||||
# root unit in the tree is claimed by exactly one entry, every repo file is
|
||||
# owned by a service or listed in `unowned:` with a reason, every seam exists,
|
||||
# the rendered Pi-hole hosts file equals the render, and Prometheus' scrape
|
||||
# targets plus the homepage probes equal the `scrape:` / `probe:` facts.
|
||||
#
|
||||
# --- fields ------------------------------------------------------------------
|
||||
# name the deployer's service word == the systemd unit name for
|
||||
# quadlet / root-unit / vm-quadlet kinds
|
||||
# hosts targets that accept it (the deploy-remote.sh allow-list)
|
||||
# kind quadlet rootless user unit from a .container file
|
||||
# root-unit systemd system unit symlinked out of scripts/
|
||||
# vm-quadlet quadlet inside the FrankBook-Pro podman machine
|
||||
# host a host-level script/timer, no container
|
||||
# deployed_by this service deploys as part of another deployable's handler
|
||||
# (the trio/pair shape). It has no dirs/files of its own; the
|
||||
# owner names the deployable whose handler installs it.
|
||||
# unit_dir repo dir holding <unit>.container — a scalar, or a per-host map
|
||||
# when one service carries its unit on two hosts (node-exporter)
|
||||
# units container names this deployable owns (netns coupling is derived
|
||||
# from the `Network=` lines in those files, never declared here)
|
||||
# dirs repo dirs whose every file selects this service
|
||||
# files exact repo files that select this service (root-unit trios,
|
||||
# vendored templates, host scripts installed by this handler)
|
||||
# retires files whose DELETION selects this service (the unit the
|
||||
# handler must remove before it can claim the ports)
|
||||
# only Dankserver's media stack still has one)
|
||||
# health the deployer's readiness URL; `wait` its 2 s poll budget
|
||||
# scrape Prometheus jobs: {job, target, metrics_path?, labels?}
|
||||
# dns names this service answers at (rendered into the Pi-hole
|
||||
# addn-hosts; every vhost resolves at the Dankclaw Caddy edge)
|
||||
# probe host:port the homepage checker dials (per-host map if on 2 hosts)
|
||||
# tests the Bats seams a change to this service must re-run
|
||||
# =============================================================================
|
||||
|
||||
hosts:
|
||||
dankclaw:
|
||||
address: 192.168.0.143
|
||||
# Caddy is consolidated on Dankclaw: it fronts every service on every
|
||||
# host, so every DNS name in the fleet resolves here.
|
||||
dns_edge: 192.168.0.143
|
||||
dankserver:
|
||||
address: 192.168.0.235
|
||||
frankbook-pro:
|
||||
address: 192.168.0.112
|
||||
|
||||
# Repo files no service owns. Each entry is a fact about the tree, not a
|
||||
# fallback: the census gate fails on anything unowned that is NOT listed, so
|
||||
# retirement work shrinks this list and a new stray file cannot slip in.
|
||||
unowned:
|
||||
shared/services.yml: the inventory this tool reads
|
||||
scripts: repo tooling (build, fetch, bench, deploy, test); the deployers'
|
||||
files are claimed by name in the service entries
|
||||
autoresearch.sh: bench scratch at the repo root (retirement pending)
|
||||
echo_prompt.txt: bench fixture at the repo root (retirement pending)
|
||||
echo_request.json: bench fixture at the repo root (retirement pending)
|
||||
prose_prompt.txt: bench fixture at the repo root (retirement pending)
|
||||
prose_request.json: bench fixture at the repo root (retirement pending)
|
||||
dankclaw/apply_qwen21_patch.py: retired patch script (its target runtime is gone)
|
||||
dankclaw/comfyui-start.sh: superseded by the comfyui quadlet + Containerfile
|
||||
dankclaw/udev: host udev rules, applied by manual sudo install + udevadm trigger
|
||||
tests: the seams themselves
|
||||
report: frozen point-in-time audit reports
|
||||
old: retired scripts, reference only
|
||||
.githooks: the commit hooks that run the seams
|
||||
|
||||
services:
|
||||
# --- Search stack: one netns family, gluetun owns it ---------------------
|
||||
- name: gluetun
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/searxng/quadlet
|
||||
units: [gluetun]
|
||||
tests: [tests/searxng_quadlet.bats]
|
||||
# The tunnel's WireGuard bind — the homepage card for the tunnel itself.
|
||||
probe: 127.0.0.1:51820
|
||||
|
||||
- name: searxng-valkey
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/searxng/quadlet
|
||||
units: [searxng-valkey]
|
||||
# The app-level law, not a network one: deploy_searxng_valkey restarts
|
||||
# SearXNG after its cache, so the cache's files select both.
|
||||
dependents: [searxng]
|
||||
tests: [tests/searxng_quadlet.bats]
|
||||
probe: 127.0.0.1:6379
|
||||
|
||||
- name: searxng
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/searxng/quadlet
|
||||
units: [searxng]
|
||||
dirs: [shared/searxng]
|
||||
tests: [tests/searxng_limiter.bats, tests/searxng_quadlet.bats]
|
||||
scrape:
|
||||
- job: services
|
||||
target: 192.168.0.143:8081
|
||||
metrics_path: /
|
||||
dns: [search.franksdank.cloud]
|
||||
probe: 127.0.0.1:8081
|
||||
|
||||
- name: marreta
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/marreta/quadlet
|
||||
units: [marreta, marreta-browser]
|
||||
tests: [tests/marreta_quadlet.bats]
|
||||
dns: [marreta.franksdank.cloud]
|
||||
probe: 127.0.0.1:8082
|
||||
|
||||
# --- DNS ------------------------------------------------------------------
|
||||
- name: pihole
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/pihole/quadlet
|
||||
units: [pihole]
|
||||
dirs: [dankclaw/pihole]
|
||||
tests: [tests/pihole.bats, tests/docs_completeness.bats]
|
||||
dns: [pihole.franksdank.cloud]
|
||||
probe: 192.168.0.143:8083
|
||||
|
||||
# --- Chat front end -------------------------------------------------------
|
||||
- name: open-webui
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/open-webui/quadlet
|
||||
units: [open-webui]
|
||||
tests: [tests/open_webui_quadlet.bats, tests/caddyfile.bats]
|
||||
scrape:
|
||||
- job: services
|
||||
target: 192.168.0.143:3005
|
||||
metrics_path: /
|
||||
dns: [ai.franksdank.cloud]
|
||||
probe: 127.0.0.1:3005
|
||||
|
||||
# --- Git hosting ----------------------------------------------------------
|
||||
- name: gitea-postgres
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/gitea/quadlet
|
||||
units: [gitea-postgres]
|
||||
deployed_by: gitea-server
|
||||
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
|
||||
probe: 127.0.0.1:5432
|
||||
|
||||
- name: gitea-server
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/gitea/quadlet
|
||||
units: [gitea-server]
|
||||
tests: [tests/caddyfile.bats, tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: gitea
|
||||
target: 192.168.0.143:3000
|
||||
metrics_path: /metrics
|
||||
dns: [gitea.franksdank.cloud]
|
||||
probe: 127.0.0.1:3000
|
||||
|
||||
# --- Image generation -----------------------------------------------------
|
||||
- name: comfyui
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/comfyui/quadlet
|
||||
units: [comfyui]
|
||||
dirs: [dankclaw/comfyui]
|
||||
files: [dankclaw/Containerfile.comfyui]
|
||||
health: http://127.0.0.1:8188/
|
||||
wait: 300
|
||||
tests: [tests/comfyui_rocm.bats, tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: comfyui
|
||||
target: comfyui:8188
|
||||
metrics_path: /
|
||||
dns: [comfyui.franksdank.cloud]
|
||||
probe: 127.0.0.1:8188
|
||||
|
||||
# --- Requests -------------------------------------------------------------
|
||||
- name: seerr
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/seerr/quadlet
|
||||
units: [seerr]
|
||||
tests: [tests/seerr_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [seerr.franksdank.cloud]
|
||||
probe: 127.0.0.1:5055
|
||||
|
||||
# --- Agent stack: one handler, four containers ----------------------------
|
||||
- name: hermes
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/hermes/quadlet
|
||||
units: [hermes, hermes-dashboard, hermes-webui, chrome-devtools-protocol]
|
||||
dirs: [dankclaw/hermes]
|
||||
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||
dns: [hermes.franksdank.cloud]
|
||||
probe: 127.0.0.1:8787
|
||||
|
||||
- name: hermes-dashboard
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
deployed_by: hermes
|
||||
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||
dns: [hermesadmin.franksdank.cloud]
|
||||
probe: 192.168.0.143:9119
|
||||
|
||||
- name: hermes-webui
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
deployed_by: hermes
|
||||
tests: [tests/hermes_webui.bats, tests/docs_completeness.bats]
|
||||
|
||||
# --- Observability --------------------------------------------------------
|
||||
- name: prometheus
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/observability/quadlet
|
||||
units: [prometheus]
|
||||
dirs: [shared/prometheus]
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: prometheus
|
||||
target: localhost:9090
|
||||
dns: [prometheus.franksdank.cloud]
|
||||
probe: 127.0.0.1:9090
|
||||
|
||||
- name: grafana
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/observability/quadlet
|
||||
units: [grafana]
|
||||
dirs: [shared/grafana]
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: grafana
|
||||
target: grafana:3000
|
||||
metrics_path: /metrics
|
||||
dns: [grafana.franksdank.cloud]
|
||||
probe: 127.0.0.1:3002
|
||||
|
||||
- name: loki
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/observability/quadlet
|
||||
units: [loki]
|
||||
dirs: [dankclaw/loki, shared/loki]
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: loki
|
||||
target: loki:3100
|
||||
metrics_path: /metrics
|
||||
dns: [loki.franksdank.cloud]
|
||||
probe: 127.0.0.1:3100
|
||||
|
||||
- name: alloy
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/observability/quadlet
|
||||
units: [alloy]
|
||||
dirs: [dankclaw/alloy, shared/alloy]
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
probe: 127.0.0.1:12345
|
||||
|
||||
- name: node-exporter
|
||||
# One service on two hosts: the same exporter, its own unit per host.
|
||||
hosts: [dankclaw, dankserver]
|
||||
kind: quadlet
|
||||
unit_dir:
|
||||
dankclaw: dankclaw/observability/quadlet
|
||||
dankserver: dankserver/observability/quadlet
|
||||
units: [node-exporter]
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: node-dankclaw
|
||||
target: host.containers.internal:9100
|
||||
- job: node-dankserver
|
||||
target: 192.168.0.235:9100
|
||||
probe: 127.0.0.1:9100
|
||||
|
||||
- name: alertmanager
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/observability/quadlet
|
||||
units: [alertmanager]
|
||||
# The Alertmanager config ships from Prometheus' directory, and the Signal
|
||||
# relay is a HOST unit this handler installs: a change to either reaches
|
||||
# the machine only through an alertmanager deploy.
|
||||
files:
|
||||
- shared/prometheus/alertmanager.yml
|
||||
- scripts/signal-relay.py
|
||||
- scripts/signal-relay.service
|
||||
tests: [tests/observability_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/docs_completeness.bats]
|
||||
probe: 127.0.0.1:9093
|
||||
|
||||
# --- Identity -------------------------------------------------------------
|
||||
- name: authentik-postgres
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/authentik/quadlet
|
||||
units: [authentik-postgres]
|
||||
deployed_by: authentik-server
|
||||
tests: [tests/authentik.bats]
|
||||
|
||||
- name: authentik-server
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/authentik/quadlet
|
||||
units: [authentik-server]
|
||||
tests: [tests/authentik.bats]
|
||||
dns: [auth.franksdank.cloud]
|
||||
|
||||
- name: authentik-worker
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/authentik/quadlet
|
||||
units: [authentik-worker]
|
||||
deployed_by: authentik-server
|
||||
tests: [tests/authentik.bats]
|
||||
|
||||
- name: authentik-outpost
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/authentik/quadlet
|
||||
units: [authentik-outpost]
|
||||
tests: [tests/authentik.bats]
|
||||
|
||||
- name: authentik-outpost-ldap
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/authentik/quadlet
|
||||
units: [authentik-outpost-ldap]
|
||||
tests: [tests/authentik.bats]
|
||||
|
||||
# --- The auth chain's edge, then the edge itself --------------------------
|
||||
- name: anubis
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/anubis/quadlet
|
||||
units: [anubis]
|
||||
dirs: [dankclaw/anubis]
|
||||
tests: [tests/anubis.bats, tests/caddyfile.bats, tests/homepage_caddy.bats]
|
||||
|
||||
- name: caddy
|
||||
hosts: [dankclaw]
|
||||
kind: quadlet
|
||||
unit_dir: dankclaw/caddy/quadlet
|
||||
units: [caddy]
|
||||
dirs: [dankclaw/caddy]
|
||||
files: [dankclaw/Caddyfile]
|
||||
# Retiring the old split proxy selects Caddy: it must remove the installed
|
||||
# unit and container before it can claim 80/443.
|
||||
retires: [dankclaw/gitea/quadlet/gitea-caddy.container]
|
||||
tests: [tests/caddy_quadlet.bats, tests/caddyfile.bats,
|
||||
tests/homepage_caddy.bats, tests/docs_completeness.bats]
|
||||
dns:
|
||||
- franksdank.cloud
|
||||
- www.franksdank.cloud
|
||||
- dankclaw.franksdank.cloud
|
||||
- dankserver.franksdank.cloud
|
||||
probe: 127.0.0.1:80
|
||||
|
||||
- name: homepage
|
||||
hosts: [dankclaw]
|
||||
kind: host
|
||||
dirs: [shared/homepage]
|
||||
files:
|
||||
- scripts/homepage-status.py
|
||||
- scripts/homepage-status.service
|
||||
- scripts/homepage-status.timer
|
||||
tests: [tests/homepage_caddy.bats, tests/homepage_config.bats,
|
||||
tests/homepage_deploy.bats, tests/homepage_pages.bats,
|
||||
tests/homepage_checker.bats]
|
||||
|
||||
# --- Dankserver media stack ----------------------------------------------
|
||||
# The romm trio is one handler and one directory: every file in it selects
|
||||
# all three, exactly as the deployer restarts all three.
|
||||
- name: romm-mariadb
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/romm/quadlet
|
||||
units: [romm-mariadb]
|
||||
deployed_by: romm
|
||||
dirs: [dankserver/romm]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
probe: 192.168.0.235:3306
|
||||
|
||||
- name: romm-redis
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/romm/quadlet
|
||||
units: [romm-redis]
|
||||
deployed_by: romm
|
||||
dirs: [dankserver/romm]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
|
||||
- name: romm
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/romm/quadlet
|
||||
units: [romm]
|
||||
dirs: [dankserver/romm]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [romm.franksdank.cloud]
|
||||
probe: 192.168.0.235:8085
|
||||
|
||||
- name: copyparty
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/copyparty/quadlet
|
||||
units: [copyparty]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [files.franksdank.cloud]
|
||||
probe: 192.168.0.235:3923
|
||||
|
||||
- name: jellyfin
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/jellyfin/quadlet
|
||||
units: [jellyfin]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [jelly.franksdank.cloud]
|
||||
probe: 192.168.0.235:8096
|
||||
|
||||
- name: calibre-web
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/calibre-web/quadlet
|
||||
units: [calibre-web]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [books.franksdank.cloud]
|
||||
probe: 192.168.0.235:8083
|
||||
|
||||
- name: calibre-server
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/calibre-server/quadlet
|
||||
units: [calibre-server]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
|
||||
- name: plex
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/plex/quadlet
|
||||
units: [plex]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [plex.franksdank.cloud, plex.franksdank.casa]
|
||||
probe: 192.168.0.235:32400
|
||||
|
||||
- name: tautulli
|
||||
hosts: [dankserver]
|
||||
kind: quadlet
|
||||
unit_dir: dankserver/tautulli/quadlet
|
||||
units: [tautulli]
|
||||
tests: [tests/dankserver_quadlet.bats, tests/deployment_flow.bats]
|
||||
probe: 192.168.0.235:8181
|
||||
|
||||
# --- FrankBook-Pro: the *arr stack inside the podman machine --------------
|
||||
# Every container rides gluetun's netns, and that coupling is DERIVED: each
|
||||
# arr's unit file says `Network=gluetun.container`, so a gluetun unit change
|
||||
# selects the whole stack with no declaration here.
|
||||
- name: gluetun
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/gluetun/quadlet
|
||||
units: [gluetun]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
|
||||
- name: prowlarr
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/prowlarr/quadlet
|
||||
units: [prowlarr]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [prowlarr.franksdank.cloud]
|
||||
probe: 192.168.0.112:9696
|
||||
|
||||
- name: qbittorrent
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/qbittorrent/quadlet
|
||||
units: [qbittorrent]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [qbittorrent.franksdank.cloud]
|
||||
probe: 192.168.0.112:8080
|
||||
|
||||
- name: radarr
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/radarr/quadlet
|
||||
units: [radarr]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [radarr.franksdank.cloud]
|
||||
probe: 192.168.0.112:7878
|
||||
|
||||
- name: sonarr
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/sonarr/quadlet
|
||||
units: [sonarr]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
probe: 192.168.0.112:8989
|
||||
|
||||
- name: lidarr
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/lidarr/quadlet
|
||||
units: [lidarr]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [lidarr.franksdank.cloud]
|
||||
probe: 192.168.0.112:8686
|
||||
|
||||
- name: readarr
|
||||
hosts: [frankbook-pro]
|
||||
kind: vm-quadlet
|
||||
unit_dir: frankbook-pro/readarr/quadlet
|
||||
units: [readarr]
|
||||
tests: [tests/frankbook_quadlet.bats, tests/deployment_flow.bats]
|
||||
dns: [readarr.franksdank.cloud]
|
||||
probe: 192.168.0.112:8787
|
||||
|
||||
# --- The LLM fleet (last: never implied by a generic change) --------------
|
||||
- name: frank-dragoon
|
||||
hosts: [dankclaw]
|
||||
kind: root-unit
|
||||
files:
|
||||
- scripts/frank-dragoon.env
|
||||
- scripts/frank-dragoon.service
|
||||
- scripts/serve-frank-dragoon.sh
|
||||
health: http://127.0.0.1:7770/health
|
||||
wait: 1200
|
||||
tests: [tests/frank_dragoon_unit.bats, tests/llm_units.bats,
|
||||
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||
tests/scripts_syntax.bats, tests/docs_completeness.bats]
|
||||
scrape:
|
||||
- job: frank-dragoon
|
||||
target: host.containers.internal:7770
|
||||
metrics_path: /metrics
|
||||
labels: {service: frank-dragoon}
|
||||
probe: 127.0.0.1:7770
|
||||
|
||||
- name: frank-zealot
|
||||
hosts: [dankclaw]
|
||||
kind: root-unit
|
||||
files:
|
||||
- scripts/frank-zealot.env
|
||||
- scripts/frank-zealot.service
|
||||
- scripts/serve-frank-zealot.sh
|
||||
health: http://127.0.0.1:7771/health
|
||||
wait: 900
|
||||
tests: [tests/frank_zealot_unit.bats, tests/llm_units.bats,
|
||||
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||
tests/scripts_syntax.bats]
|
||||
scrape:
|
||||
- job: frank-zealot
|
||||
target: host.containers.internal:7771
|
||||
metrics_path: /metrics
|
||||
labels: {service: frank-zealot}
|
||||
probe: 127.0.0.1:7771
|
||||
|
||||
- name: frank-archon
|
||||
hosts: [dankserver]
|
||||
kind: root-unit
|
||||
files:
|
||||
- scripts/frank-archon.env
|
||||
- scripts/frank-archon.service
|
||||
- scripts/serve-frank-archon.sh
|
||||
# The vendored chat template the engine renders with.
|
||||
- scripts/qwen38-froggeric-v22.5.jinja
|
||||
health: http://127.0.0.1:7770/health
|
||||
wait: 900
|
||||
tests: [tests/frank_archon_unit.bats, tests/llm_units.bats,
|
||||
tests/llm_deploy_gates.bats, tests/llm_secrets.bats,
|
||||
tests/scripts_syntax.bats]
|
||||
scrape:
|
||||
- job: frank-archon
|
||||
target: 192.168.0.235:7770
|
||||
metrics_path: /metrics
|
||||
labels: {service: frank-archon}
|
||||
probe: 192.168.0.235:7770
|
||||
@@ -13,11 +13,15 @@ load helpers
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "changed-test wrapper maps a compose change to compose and environment tests" {
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankserver/compose.yaml
|
||||
@test "changed-test wrapper maps a shared config change to its service seams" {
|
||||
# The compose era is retired (rank 7): the former compose fixture here
|
||||
# mapped dankserver/compose.yaml, a file that no longer exists. The
|
||||
# equivalent contract now: a service config change maps to the
|
||||
# service's seams and nothing unrelated.
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/searxng/settings.yml
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"tests/compose_validity.bats"* ]]
|
||||
[[ "$output" == *"tests/env_consistency.bats"* ]]
|
||||
[[ "$output" == *"tests/searxng_limiter.bats"* ]]
|
||||
[[ "$output" == *"tests/searxng_quadlet.bats"* ]]
|
||||
[[ "$output" != *"tests/homepage_pages.bats"* ]]
|
||||
}
|
||||
|
||||
@@ -47,3 +51,27 @@ load helpers
|
||||
[[ "$output" != *"tests/frank_medium_unit.bats"* ]]
|
||||
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
|
||||
}
|
||||
|
||||
@test "changed-test wrapper maps the frankbook unit to the frankbook and deploy-flow seams" {
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths frankbook-pro/sonarr/quadlet/sonarr.container
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"tests/frankbook_quadlet.bats"* ]]
|
||||
[[ "$output" == *"tests/deployment_flow.bats"* ]]
|
||||
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
|
||||
[[ "$output" != *"tests/frank_archon_unit.bats"* ]]
|
||||
}
|
||||
|
||||
@test "changed-test wrapper maps the inventory itself to its seam" {
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths shared/services.yml
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"tests/service_inventory.bats"* ]]
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths scripts/service-map.py
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"tests/service_inventory.bats"* ]]
|
||||
}
|
||||
|
||||
@test "changed-test wrapper refuses an unmapped path" {
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths newdir/file.txt
|
||||
[ "$status" -eq 2 ]
|
||||
[[ "$output" == *"no focused-test mapping"* ]]
|
||||
}
|
||||
|
||||
+1773
File diff suppressed because it is too large
Load Diff
@@ -101,7 +101,10 @@ webui_service_yaml() {
|
||||
}
|
||||
|
||||
@test "remote deployer accepts and dispatches hermes-webui" {
|
||||
grep -q 'dankclaw:hermes-webui' "$REMOTE_DEPLOY"
|
||||
# Acceptance is the inventory (rank 3): the old literal allow-list in
|
||||
# the remote runner is retired.
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
|
||||
[ "$status" -eq 0 ]
|
||||
# Dispatched to the unit handler, never to compose: deploy_hermes installs
|
||||
# the quadlets and retires compose-era containers.
|
||||
grep -q 'hermes|hermes-dashboard|hermes-webui) deploy_hermes' "$REMOTE_DEPLOY"
|
||||
@@ -111,5 +114,7 @@ webui_service_yaml() {
|
||||
}
|
||||
|
||||
@test "top-level deployer accepts hermes-webui as a Dankclaw service" {
|
||||
grep -q 'dankclaw:hermes-webui' "$DEPLOY"
|
||||
# deploy.sh validates --service through the inventory's allows.
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service hermes-webui
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@@ -433,11 +433,14 @@ assert any('ALERTS' in e for e in exprs), exprs
|
||||
grep -q -- '--web.listen-address=127.0.0.1:9100' "$unit"
|
||||
grep -q -- '--web.listen-address=192.168.0.235:9100' "$unit"
|
||||
! grep -qE -- '--web\.listen-address=(0\.0\.0\.0|:9100|\[::\])' "$unit"
|
||||
# One owner per container name: the compose service must be gone, or two
|
||||
# definitions fight over the same container.
|
||||
! grep -qE '^\s+node-exporter:' "$REPO_ROOT/dankserver/compose.yaml"
|
||||
# The deploy path has to know about it on this target.
|
||||
grep -q 'dankserver:node-exporter' "$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
# One owner per container name: Dankserver has no compose file at all
|
||||
# since the 2026-10-13 retirement (it lived beside diverged quadlet
|
||||
# mounts for weeks — the two-owners trap by construction).
|
||||
[ ! -e "$REPO_ROOT/dankserver/compose.yaml" ]
|
||||
# The deploy path has to know about it on this target — acceptance
|
||||
# is the inventory (rank 3), not a literal list in the runner.
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service node-exporter
|
||||
[ "$status" -eq 0 ]
|
||||
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
|
||||
--paths dankserver/observability/quadlet/node-exporter.container
|
||||
[ "$status" -eq 0 ]
|
||||
|
||||
Executable
+125
@@ -0,0 +1,125 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: the service inventory (rank 3). shared/services.yml is the one
|
||||
# inventory of deployable services; scripts/service-map.py is its only
|
||||
# reader, and every consumer (deploy-plan.py, deploy-remote.sh's
|
||||
# allow-list, deploy.sh's --service gate, test-changed.sh's seam mapper,
|
||||
# the Pi-hole hosts render, the Prometheus scrape facts, the homepage
|
||||
# probes) goes through it. These tests pin the inventory's integrity,
|
||||
# the planner's selection against a captured fixture, and the rendered
|
||||
# artifacts' drift against the tracked copies.
|
||||
|
||||
load helpers
|
||||
|
||||
@test "inventory: schema, unit claims, path claims, compose ownership, and render drift are sound" {
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" check
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "inventory: the deployers' allow-list is the inventory" {
|
||||
# Every service the inventory places on a host must pass "allows" for
|
||||
# that host, and a name the inventory never places must be refused.
|
||||
for host in dankclaw dankserver frankbook-pro; do
|
||||
while IFS= read -r svc; do
|
||||
[ -n "$svc" ] || continue
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host "$host" --service "$svc"
|
||||
[ "$status" -eq 0 ] || { echo "allows refused $host:$svc"; return 1; }
|
||||
done < <(python3 "$REPO_ROOT/scripts/service-map.py" services --host "$host")
|
||||
done
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankserver --service sonarr
|
||||
[ "$status" -ne 0 ] # the *arr stack lives on frankbook-pro, not dankserver
|
||||
run python3 "$REPO_ROOT/scripts/service-map.py" allows --host dankclaw --service frank-archon
|
||||
[ "$status" -ne 0 ] # frank-archon lives on dankserver
|
||||
}
|
||||
|
||||
@test "inventory: the planner reproduces the pinned selection fixture" {
|
||||
# tests/fixtures/service-plan-golden.tsv is the characterization of
|
||||
# path -> service selection across every tracked path, every target,
|
||||
# and the multi-path combos the old hand-written planner was fed.
|
||||
python3 - "$REPO_ROOT" <<'PYEOF'
|
||||
import importlib.util
|
||||
import sys
|
||||
|
||||
root = sys.argv[1]
|
||||
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
|
||||
sm = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(sm)
|
||||
inv = sm.Inventory.load(root + "/shared/services.yml")
|
||||
bad = []
|
||||
with open(root + "/tests/fixtures/service-plan-golden.tsv") as fh:
|
||||
for line in fh:
|
||||
host, paths, want = line.rstrip("\n").split("\t")
|
||||
got = ",".join(sm.select(inv, host, paths.split("+")))
|
||||
if got != want:
|
||||
bad.append(f"{host} {paths}\n want: {want}\n got: {got}")
|
||||
if bad:
|
||||
print(f"{len(bad)} selection mismatches:", *bad[:20], sep="\n")
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
@test "inventory: deploy-plan.py keeps its CLI contract on top of service-map" {
|
||||
# stdin paths, positional paths, and the --repo/--base/--head diff mode
|
||||
# all route to the same selection.
|
||||
run bash -c "cd '$REPO_ROOT' && printf '%s\n' dankclaw/searxng/quadlet/gluetun.container | \
|
||||
scripts/deploy-plan.py --target dankclaw"
|
||||
[ "$status" -eq 0 ]
|
||||
[ "${lines[0]}" = "gluetun" ]
|
||||
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver scripts/frank-archon.env
|
||||
[ "$status" -eq 0 ]
|
||||
[ "${lines[0]}" = "frank-archon" ]
|
||||
run "$REPO_ROOT/scripts/deploy-plan.py" --target dankclaw --repo "$REPO_ROOT" --base HEAD --head HEAD
|
||||
[ "$status" -eq 0 ]
|
||||
[ -z "$output" ]
|
||||
}
|
||||
|
||||
@test "inventory: the Pi-hole hosts file is a rendered artifact" {
|
||||
# The tracked addn-hosts must be byte-identical to the render; editing
|
||||
# it by hand is drift the inventory check also catches.
|
||||
run bash -c "cd '$REPO_ROOT' && python3 scripts/service-map.py render pihole-hosts | \
|
||||
diff -u dankclaw/pihole/addn-hosts -"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "inventory: changed-test seams for a service unit are the service's own" {
|
||||
# The rank-3 contract: a unit change selects the OWNING service's
|
||||
# declared seams, not the old shotgun of unrelated quadlet seams.
|
||||
run bash "$REPO_ROOT/scripts/test-changed.sh" --list --paths dankclaw/gitea/quadlet/gitea.network
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"tests/caddyfile.bats"* ]]
|
||||
[[ "$output" == *"tests/deployment_scoping.bats"* ]]
|
||||
[[ "$output" != *"tests/open_webui_quadlet.bats"* ]]
|
||||
[[ "$output" != *"tests/searxng_quadlet.bats"* ]]
|
||||
}
|
||||
|
||||
@test "inventory: every tracked quadlet unit is claimed by exactly one service" {
|
||||
# check units covers claim conflicts; this pins the census size so a
|
||||
# unit added WITHOUT an inventory entry (unclaimed) fails the seam.
|
||||
python3 - "$REPO_ROOT" <<'PYEOF'
|
||||
import importlib.util
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import PurePosixPath
|
||||
|
||||
root = sys.argv[1]
|
||||
spec = importlib.util.spec_from_file_location("sm", root + "/scripts/service-map.py")
|
||||
sm = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(sm)
|
||||
inv = sm.Inventory.load(root + "/shared/services.yml")
|
||||
graph = sm.graph_for(tuple(inv.hosts))
|
||||
owned = {u for u in graph.units}
|
||||
problems = []
|
||||
tracked_units = [
|
||||
p for p in sm.tracked_files()
|
||||
if PurePosixPath(p).name.endswith((".container", ".network")) and not sm.is_trash(p)
|
||||
]
|
||||
for path in tracked_units:
|
||||
parts = PurePosixPath(path).parts
|
||||
name = parts[-1]
|
||||
key = (parts[0], name.rsplit(".", 1)[0])
|
||||
if key not in graph.units and key not in graph.networks:
|
||||
problems.append(f"unclaimed unit-file path: {path}")
|
||||
if problems:
|
||||
print("\n".join(problems))
|
||||
sys.exit(1)
|
||||
PYEOF
|
||||
}
|
||||
Reference in New Issue
Block a user