Frank
327e78b950
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
57 lines
3.4 KiB
Markdown
57 lines
3.4 KiB
Markdown
---
|
|
type: reference
|
|
description: "This repository tests its configuration at defined seams."
|
|
tags:
|
|
- architecture
|
|
- testing
|
|
timestamp: 2026-10-18
|
|
---
|
|
|
|
# Testing Policy
|
|
|
|
This repository tests its configuration at defined seams. The tests run with `bats`; run them with `./tests/run.sh` before every push.
|
|
|
|
## Seams Under Test
|
|
|
|
| Seam | File | What it asserts |
|
|
|------|------|-----------------|
|
|
| Compose validity | `tests/compose_validity.bats` | YAML parses, services have images, no named volumes |
|
|
| Env consistency | `tests/env_consistency.bats` | every compose file has a **sibling** `.env.example` (`:9`) in which every `${VAR}` is defined (`:17`); placeholders only (`:31`); the live gitea database password never appears (`:37`) |
|
|
| Caddyfile validity | `tests/caddyfile.bats` | the Caddyfile lives on **dankclaw only** (`:15`, not per-host), flat `franksdank.cloud` names with no `.local` leftovers (`:20,:24`), balanced braces, `caddy validate` when the binary exists, the renderable homepage basicauth placeholder, the quadlet/Porkbun/loki/prometheus guardrails |
|
|
| Docs completeness | `tests/docs_completeness.bats` | every compose service has a docs page; `index.md` links every page; **no wikilink in the vault points at a page that does not exist** (`:37`) |
|
|
| Script syntax | `tests/scripts_syntax.bats` | `bash -n` clean, executable bits, shellcheck when available |
|
|
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
|
|
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
|
|
| E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] |
|
|
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
|
|
|
|
## Focused pre-commit testing
|
|
|
|
For normal commits, use the focused wrapper:
|
|
|
|
```bash
|
|
./scripts/test-changed.sh
|
|
```
|
|
|
|
It reads staged paths with `git diff --cached` and runs only the mapped Bats
|
|
files. The tracked `.githooks/pre-commit` hook invokes this wrapper. Enable the
|
|
repository hook path once per checkout:
|
|
|
|
```bash
|
|
git config core.hooksPath .githooks
|
|
```
|
|
|
|
Use `--list` to inspect the selection without running tests, or `--paths FILE
|
|
...` to preview a mapping. Shared test infrastructure and unmapped paths fail
|
|
closed rather than silently skipping coverage. Run the full suite with
|
|
`./tests/run.sh` when changing shared test helpers, the test runner, or when
|
|
focused coverage is not sufficient.
|
|
|
|
## Red-Green
|
|
|
|
Tests are written first. A failing test means the seam is not satisfied; the config change that makes it pass is the only acceptable fix. No `--no-verify`, no deleting tests without approval.
|
|
|
|
## See Also
|
|
|
|
- [[Deployment Flow]]
|