diff --git a/docs/Architecture/Deployment Flow.md b/docs/Architecture/Deployment Flow.md index 0f02405..18dedaa 100644 --- a/docs/Architecture/Deployment Flow.md +++ b/docs/Architecture/Deployment Flow.md @@ -60,7 +60,7 @@ service on the other host. | Service family | Runtime manager | Scope behavior | |---|---|---| | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. | -| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. | +| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. | | `frank-default-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. | | `frank-default-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. | | Dankserver media inventory | host-managed bare containers | Compose is inventory only; the scoped deploy does not create duplicate containers. | @@ -71,12 +71,11 @@ it is not restarted by an unrelated large-model deployment. ## Caddy safety -Caddy runs on Dankclaw and fronts both hosts. The dispatcher never stops -`gitea-caddy` implicitly. If the legacy `gitea-caddy` container owns ports -80/443 while the compose `caddy` container is absent or stopped, a Caddy -selection fails closed rather than replacing the live proxy. Resolve port -ownership explicitly before retrying. A Caddy reload failure falls back to a -restart of **only** the selected `caddy` container. +Caddy runs on Dankclaw and fronts both hosts, including Gitea. The former +standalone `gitea-caddy` Quadlet is retired; the canonical compose `caddy` +service is now the sole reverse-proxy owner of ports 80/443. A Caddy reload +failure falls back to a restart of **only** the selected `caddy` container. +Before deploying, verify that no retired proxy still owns either port. Homepage files, generated `deployed.js`, the status checker, and the rendered Caddyfile are updated only when their service is selected. A media-only diff --git a/docs/Dankclaw/Dankclaw Stack.md b/docs/Dankclaw/Dankclaw Stack.md index d86bc4b..d917b5b 100644 --- a/docs/Dankclaw/Dankclaw Stack.md +++ b/docs/Dankclaw/Dankclaw Stack.md @@ -1,3 +1,15 @@ +--- +type: reference +title: Dankclaw Stack +description: Service inventory and routing overview for the Dankclaw AI, development, and observability host. +tags: + - dankclaw + - services + - containers + - caddy +timestamp: 2026-08-26 +--- + # Dankclaw Stack Dankclaw (192.168.0.143) runs the AI, search, and development stack. It @@ -43,9 +55,9 @@ repo owns their scripts, units, and knowledge base (see ## Boot Persistence -The Gitea trio, the SearXNG trio (gluetun, searxng, valkey), and open-webui run as quadlet units, so they start at boot. User linger is on. ComfyUI runs as a systemd unit around a bare process. The remaining services in the table are defined in `compose.yaml` and are not yet boot-persistent. +The Gitea database and server, the SearXNG trio (gluetun, searxng, valkey), and open-webui run as Quadlet units, so they start at boot. Gitea's web UI is reverse-proxied by the canonical Caddy compose service; the former standalone Gitea Caddy unit is retired. User linger is on. ComfyUI runs as a systemd unit around a bare process. The remaining services in the table are defined in `compose.yaml` and are not yet boot-persistent. -Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw` and open-webui with `scripts/deploy-open-webui-quadlet.sh dankclaw`. The Gitea units install to `~/.config/containers/systemd/` (see [[Gitea]]). +Deploy the SearXNG trio with `scripts/deploy-searxng-quadlet.sh dankclaw` and open-webui with `scripts/deploy-open-webui-quadlet.sh dankclaw`. The Gitea units install to `~/.config/containers/systemd/` (see [[Gitea]]); Caddy is deployed through `scripts/deploy.sh dankclaw --service caddy`. ## Routing Rules diff --git a/docs/Dankclaw/Gitea.md b/docs/Dankclaw/Gitea.md index 5bbc240..0a9165d 100644 --- a/docs/Dankclaw/Gitea.md +++ b/docs/Dankclaw/Gitea.md @@ -1,8 +1,20 @@ +--- +type: configuration +title: Gitea +description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service. +tags: + - gitea + - dankclaw + - quadlet + - caddy +timestamp: 2026-08-26 +--- + # Gitea -Gitea is a self-hosted Git service with issues, pull requests, and a web UI. On Dankclaw it runs as three Quadlet-managed containers in the rootless podman session of user `dank`. +Gitea is a self-hosted Git service with issues, pull requests, and a web UI. On Dankclaw it runs as two Quadlet-managed containers in the rootless Podman session of user `dank`. The canonical Caddy on the same host reverse-proxies its web UI at `gitea.franksdank.cloud`. -**Host:** Dankclaw — **LAN access:** `http://dankclaw:3000` (or `http://192.168.0.143:3000`) — **Public domain:** `gitea.franksdank.cloud` (DNS pending) — **Docs:** https://docs.gitea.com +**Host:** Dankclaw — **LAN access:** `http://dankclaw:3000` (or `http://192.168.0.143:3000`) — **Proxy domain:** `gitea.franksdank.cloud` — **Docs:** https://docs.gitea.com ## Containers @@ -10,7 +22,6 @@ Gitea is a self-hosted Git service with issues, pull requests, and a web UI. On |-----------|------|-------|---------| | `gitea-postgres` | `gitea-postgres.service` | `docker.io/library/postgres:16-alpine` | Database | | `gitea-server` | `gitea-server.service` | `docker.io/gitea/gitea:latest` | Web UI, API, git (HTTP :3000, SSH :2222) | -| `gitea-caddy` | `gitea-caddy.service` | `docker.io/library/caddy:latest` | Reverse proxy for the public domain | The containers share the private bridge network `systemd-gitea`. The quadlet network file `gitea.network` enables DNS, so the containers resolve each other by name. The automatic `podman` default network has DNS disabled. Do not run the Gitea containers on it. @@ -29,10 +40,10 @@ Run all commands as `dank` on Dankclaw. | Action | Command | |--------|---------| -| Start all | `systemctl --user start gitea-postgres gitea-server gitea-caddy` | -| Stop all | `systemctl --user stop gitea-caddy gitea-server gitea-postgres` | +| Start all | `systemctl --user start gitea-postgres gitea-server` | +| Stop all | `systemctl --user stop gitea-server gitea-postgres` | | Restart Gitea only | `systemctl --user restart gitea-server` | -| Status | `systemctl --user status gitea-postgres gitea-server gitea-caddy` | +| Status | `systemctl --user status gitea-postgres gitea-server` | | Container logs | `podman logs -f gitea-server` | | Unit logs | `journalctl --user -u gitea-server -f` | @@ -42,14 +53,13 @@ The units start in dependency order: graph LR N[gitea-network.service] --> P[gitea-postgres.service] P --> S[gitea-server.service] - S --> C[gitea-caddy.service] ``` ## Boot persistence User-level linger is enabled for `dank` (`loginctl enable-linger dank`). The units start at host boot without a login session. Each unit carries `[Install] WantedBy=default.target`. The podman generator applies it at boot; no symlink is needed. -The containers use the quadlet fresh-start model. A stop removes the container (`podman run --rm --replace`). The next start recreates it. The bind-mounted data survives. +The containers use the quadlet fresh-start model. A stop removes the container (`podman run --rm --replace`). The next start recreates it. The bind-mounted data survives. The canonical Caddy is managed separately by the Dankclaw stack. ## Secrets @@ -61,11 +71,11 @@ Do not change the password in place. Change the PostgreSQL role first, then upda | File | Location | Repo copy | |------|----------|-----------| -| Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server,caddy}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](../../dankclaw/gitea/quadlet/) | -| Caddyfile | `/opt/gitea/data/caddy/Caddyfile` | host-only (not in repo) | +| Quadlet units | `~/.config/containers/systemd/gitea-{postgres,server}.container`, `gitea.network` | [`dankclaw/gitea/quadlet/`](../../dankclaw/gitea/quadlet/) | +| Caddyfile | `/home/dank/Containers/caddy/Caddyfile` | [`dankclaw/Caddyfile`](../../dankclaw/Caddyfile) | | Secrets | `/opt/gitea/.env` (0600) | [`dankclaw/gitea/.env.example`](../../dankclaw/gitea/.env.example) | -After editing a repo copy, copy it to the target, then run `systemctl --user daemon-reload` and restart the affected units. +After editing a repo copy, deploy through `scripts/deploy.sh dankclaw`. The Gitea database/server units are installed from this directory; the canonical Caddy and its rendered Caddyfile are deployed as the separate `caddy` service. Do not manually recreate the retired `gitea-caddy` unit. ## Data volumes @@ -73,7 +83,6 @@ After editing a repo copy, copy it to the target, then run `systemctl --user dae |----------------|-----------|----------| | `/data` | `/opt/gitea/data/gitea-gitea` | Git repositories, app.ini, attachments | | `/var/lib/postgresql/data` | `/opt/gitea/data/gitea-postgres` | PostgreSQL data | -| `/etc/caddy/Caddyfile`, `/root/.local/share/caddy` | `/opt/gitea/data/caddy/...` | Caddy config and TLS state | Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge base page [[Gitea]]. @@ -83,8 +92,8 @@ Back up `/opt/gitea/data/` (tar) or `pg_dump` the database. See the knowledge ba - **The files in the data directory are owned by host UID 100999**, not `dank` (rootless ID mapping: container uid 1000 → host 100999). Edit them inside the container with `podman exec --user 1000 gitea-server ...`. - **The host auto-reboots.** `unattended-upgrades` reboots Dankclaw when a package needs it. The quadlet units make Gitea survive that. See the 2026-08-16 incident below. - **Port 3000 vs 3001.** The live host port is 3000. The full-stack [`dankclaw/compose.yaml`](../../dankclaw/compose.yaml) maps 3001. Align the two before deploying the full stack. -- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the quadlet units on ports 3000, 2222, 80, and 443. -- **Rootless cannot bind ports below 1024 by default.** `gitea-caddy` publishes 80 and 443. Without a host sysctl, `rootlessport` fails with `permission denied` and the unit crash-loops under `Restart=always`. The fix is a one-time sysctl: `net.ipv4.ip_unprivileged_port_start=80`, persisted in `/etc/sysctl.d/99-rootless-ports.conf`. Applied 2026-08-18 after a 13,000-restart crash loop. +- **Legacy compose retired.** `/opt/gitea/podman-compose.yml` is retired to `/opt/gitea/.deprecated/`. Do not run it. Its containers collide with the Quadlet units on ports 3000 and 2222. +- **Canonical proxy.** `gitea.franksdank.cloud` is served by the Caddy service in `dankclaw/compose.yaml`, which proxies to `192.168.0.143:3000`. Do not recreate a Gitea-specific Caddy unit or publish another proxy on ports 80/443. During migration, the deployer stages any installed legacy unit under `~/.AGENTTRASH/YYYYMMDD-gitea-caddy/` before starting canonical Caddy. ## Incident: 2026-08-16 — down after unattended reboot diff --git a/docs/index.md b/docs/index.md index 1377b00..cc015e7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -158,6 +158,10 @@ The LLM inference fleet on the two hosts. Full index: - [[Sample Script Set]] — reference scripts for the PP pair - [[vLLM Pipeline Parallelism]] — vLLM's PP implementation +## Update Log + +- [[log]] — chronological record of knowledge-base changes + ## See Also - [AGENTS.md](../AGENTS.md) — repository rules diff --git a/docs/log.md b/docs/log.md new file mode 100644 index 0000000..930049b --- /dev/null +++ b/docs/log.md @@ -0,0 +1,11 @@ +# Bundle Change Log + +## 2026-08-26 +- `/docs/Dankclaw/Gitea.md` — Documented migration of Gitea web proxy ownership to canonical Caddy and retirement of the standalone proxy. +- `/docs/Dankclaw/Dankclaw Stack.md` — Updated boot-persistence and Caddy deployment descriptions for the two-unit Gitea stack. +- `/docs/Architecture/Deployment Flow.md` — Replaced the split-proxy safety description with the canonical Caddy ownership model. +- `/dankclaw/gitea/quadlet/gitea-caddy.container` — Retired the standalone Gitea Caddy Quadlet; deployment stages any live copy into home trash. +- `/scripts/deploy-plan.py` — Routes retirement of the legacy proxy to the canonical Caddy and removes it from the service inventory. +- `/scripts/deploy.sh` and `/scripts/deploy-remote.sh` — Removed standalone proxy dispatch and added idempotent retirement before canonical Caddy deployment. +- `/tests/caddyfile.bats`, `/tests/deployment_scoping.bats`, `/tests/deployment_flow.bats`, `/tests/frank_large_config.bats` — Added migration coverage and corrected the existing ubatch harness assertion. +- `/docs/index.md` — Bundle index remains the root navigation entry for the updated pages. diff --git a/scripts/deploy-remote.sh b/scripts/deploy-remote.sh index 26f22c2..cf75fd7 100755 --- a/scripts/deploy-remote.sh +++ b/scripts/deploy-remote.sh @@ -293,8 +293,8 @@ deploy_caddy() { # Retire the old split proxy before claiming ports 80/443. This is # idempotent and leaves the old unit recoverable in the target trash. retire_legacy_gitea_caddy + podman-compose -f dankclaw/compose.yaml up -d --no-deps caddy if podman container exists caddy; then - podman-compose -f dankclaw/compose.yaml up -d --no-deps caddy local reloaded=0 for _ in 1 2 3 4 5; do if podman exec caddy caddy reload --config /etc/caddy/Caddyfile \ @@ -309,8 +309,6 @@ deploy_caddy() { echo "Caddy reload failed; restarting only caddy." podman restart caddy fi - else - echo "WARN: caddy container is absent; no proxy was stopped or replaced." fi } diff --git a/tests/deployment_flow.bats b/tests/deployment_flow.bats index 85d1fcd..4b5ce35 100644 --- a/tests/deployment_flow.bats +++ b/tests/deployment_flow.bats @@ -54,6 +54,11 @@ load helpers ! grep -q 'deploy_gitea_unit gitea-caddy' "$REPO_ROOT/scripts/deploy-remote.sh" } +@test "canonical Caddy deployment recreates the container when needed" { + grep -q 'podman-compose -f dankclaw/compose.yaml up -d --no-deps caddy' "$REPO_ROOT/scripts/deploy-remote.sh" + ! grep -q 'no proxy was stopped or replaced' "$REPO_ROOT/scripts/deploy-remote.sh" +} + @test "remote runner has separate handlers for both inference services" { grep -q 'deploy_llm_large' "$REPO_ROOT/scripts/deploy-remote.sh" grep -q 'deploy_llm_small' "$REPO_ROOT/scripts/deploy-remote.sh" diff --git a/tests/frank_large_config.bats b/tests/frank_large_config.bats index 69fc2a0..663be20 100644 --- a/tests/frank_large_config.bats +++ b/tests/frank_large_config.bats @@ -70,7 +70,7 @@ TEMPLATE_FILE="$REPO_ROOT/scripts/chat-template.jinja" grep -q 'Qwen3\.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT\.gguf' "$bench" grep -q -- '--n-prompt "$CONTEXT"' "$bench" grep -q -- '--ubatch-size "$UBATCH_VALUES"' "$bench" - grep -q -- '--batch-size "$CONTEXT"' "$bench" + grep -q -- '--batch-size "$UBATCH_VALUES"' "$bench" grep -q -- '--mmap 0' "$bench" ! grep -qE '(^|[[:space:]])llama-server|(^|[[:space:]])curl|print_timing' "$bench" }