docs: record quadlet standardization of dankserver media + dankclaw data roots
Phase 1/2 of the podman standardization plan: service pages updated to the quadlet reality (units, Containers data roots, env-file secret convention, the v1-generator findings each caught live), Storage Policy documents the two rootless ownership regimes and the merge-move migration, Deployment Flow's service-manager table no longer claims dankserver media is bare inventory-only. Bundle change log entry added.
This commit is contained in:
Frank
@@ -63,7 +63,7 @@ service on the other host.
|
||||
| SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
|
||||
| `frank-default-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
|
||||
| `frank-default-small` | systemd + `systemctl` | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
|
||||
| Dankserver media inventory | host-managed bare containers | Compose is inventory only; the scoped deploy does not create duplicate containers. |
|
||||
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
|
||||
|
||||
The shared chat template maps to the inference service on the selected target.
|
||||
The inactive `frank-default-npu` unit is relink-only when explicitly selected;
|
||||
|
||||
@@ -11,30 +11,50 @@ All persistent data uses **bind mounts**: a host directory mapped directly into
|
||||
|
||||
## Conventions
|
||||
|
||||
| Stack | Host root |
|
||||
|-------|-----------|
|
||||
| Dankserver | `/opt/<service>/...` |
|
||||
| Dankclaw | `/opt/<service>/...` (Gitea keeps `/opt/gitea/data/...`) |
|
||||
| Stack | Host data root |
|
||||
|-------|----------------|
|
||||
| Dankserver | `/home/dank/Containers/<svc>/` (romm, jellyfin, calibre-web, copyparty) |
|
||||
| Dankclaw | `/home/dank/Containers/<svc>/` (caddy, gitea, authentik, hermes, …) |
|
||||
|
||||
This layout was standardized on 2026-09-01. The old `/opt/<service>` roots were
|
||||
migrated by the deployers' `migrate_data_root` (merge-move, numeric ownership
|
||||
preserved); `/opt` holds no live container data on either host.
|
||||
|
||||
## Ownership
|
||||
|
||||
Containers run as `PUID=1000` / `PGID=1000`. Host directories must be owned by `dank`:
|
||||
Containers are rootless. Two regimes exist and the deployer must respect both:
|
||||
|
||||
- **PUID/PGID images** (linuxserver calibre-web, jellyfin): data is owned by
|
||||
host uid 1000 (`dank`) — plain `chown` is safe.
|
||||
- **Mapped-uid images** (official mariadb, redis, authentik, postgres): files
|
||||
land in the subuid range (e.g. 100998 for container uid 999). Renaming such
|
||||
a directory requires root (rename needs write on the directory itself) and
|
||||
**re-chowning to 1000 breaks the container's internal user**. Migrations
|
||||
therefore use `sudo mv` (numeric-preserving) and skip the chown pass
|
||||
(`migrate_data_root … keep`).
|
||||
|
||||
```bash
|
||||
sudo chown -R 1000:1000 /opt/<service>
|
||||
stat -c '%u %n' /home/dank/Containers/romm/mariadb # 100998 = mapped, leave it
|
||||
```
|
||||
|
||||
## Secrets
|
||||
|
||||
Secrets never live in bind-mounted configs that get committed. They are:
|
||||
Secrets never live in bind-mounted configs that get committed. The quadlet
|
||||
convention on both hosts:
|
||||
|
||||
1. Set in `.env` files on the host (git-ignored, `.env.example` committed as template), or
|
||||
2. Injected as environment variables from the deploy script.
|
||||
1. One host-local env file per stack at `/home/dank/Containers/<svc>/env`
|
||||
(mode 0600), referenced from the unit's `[Service] EnvironmentFile=` so
|
||||
`${VAR}` expansion works at `ExecStart` time; never committed.
|
||||
2. `.env` files in the checkout (git-ignored, `.env.example` committed as
|
||||
template) for compose-era and Caddy-rendered secrets.
|
||||
|
||||
The live Gitea database password exists only in `.env` on Dankclaw. A test enforces that it never appears in this repo.
|
||||
Deployed units hold only variable expansions (`${COPAY_ACCOUNT}` …), never
|
||||
literals — enforced by the `no secret material` seams. The live Gitea
|
||||
database password exists only in `/home/dank/Containers/gitea/env` on
|
||||
Dankclaw. A test enforces that it never appears in this repo.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Deployment Flow]]
|
||||
- [[Gitea]] — the existing bind-mount stack being migrated
|
||||
- [[Gitea]] — the original bind-mount stack, now on the Containers layout
|
||||
- [[Backblaze Integration]] — future backup target for these mounts
|
||||
|
||||
@@ -49,12 +49,20 @@ Additional packages added: `libxfixes3` and `libasound2t64` to support epub-to-p
|
||||
|
||||
## Volumes
|
||||
|
||||
| Container path | Host path (convention) |
|
||||
|----------------|------------------------|
|
||||
| `/config` | `/opt/calibre-web/config` |
|
||||
| `/books` | The existing Calibre library directory (user-entered during setup) |
|
||||
| Container path | Host path |
|
||||
|----------------|-----------|
|
||||
| `/config` | `/home/dank/Containers/calibre-web/config` |
|
||||
| `/books` | `/home/dank/Containers/calibre-web/books` |
|
||||
|
||||
## Docker Compose
|
||||
Deployment since 2026-09-01 is a quadlet unit
|
||||
(`dankserver/calibre-web/quadlet/calibre-web.container` →
|
||||
`calibre-web.service`, deploy with
|
||||
`./scripts/deploy.sh dankserver --service calibre-web`), image pinned to
|
||||
`0.6.27-ls397`, healthcheck on `/`, `PublishPort=8083:8083`. This service was
|
||||
already on the Containers data-root layout before the standardization, so the
|
||||
cutover was a bare-container retire + unit start with no data movement.
|
||||
|
||||
## Docker Compose (reference — retired 2026-09-01; live mounts are under `/home/dank/Containers/calibre-web`)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
|
||||
@@ -80,7 +80,7 @@ takes its volume roots from argv plus `chdir: /w` in `/z/initcfg`, not from a
|
||||
rw on disk — copyparty enforces read-only at the application layer, so local
|
||||
services (Jellyfin, Calibre) keep writing into the array directly.
|
||||
|
||||
## Docker Compose
|
||||
## Docker Compose (reference — retired 2026-09-01; live deployment is the quadlet unit below)
|
||||
|
||||
```yaml
|
||||
copyparty:
|
||||
@@ -100,20 +100,28 @@ services (Jellyfin, Calibre) keep writing into the array directly.
|
||||
retries: 3
|
||||
```
|
||||
|
||||
`${COPAY_PASS}` is compose-time interpolation from `dankserver/.env` — the
|
||||
password lands in the container's **argv**, visible to anyone who can run
|
||||
`${COPAY_ACCOUNT}`/`${COPAY_PASSWORD}` are **never** written into the unit:
|
||||
they live in the host-local `/home/dank/Containers/copyparty/env` (0600),
|
||||
loaded via `[Service] EnvironmentFile=` so systemd expands them into
|
||||
`Exec=-a ${COPAY_ACCOUNT}:${COPAY_PASSWORD}` at start. The password still
|
||||
lands in the container argv — visible to anyone who can run
|
||||
`podman inspect copyparty` on the host.
|
||||
|
||||
**Applying changes:** `scripts/deploy.sh` deliberately does not touch the
|
||||
dankserver compose stack (host containers are not quadlet-managed there), so
|
||||
`command:` edits need a manual recreate on Dankserver:
|
||||
## Deployment (Quadlet)
|
||||
|
||||
```
|
||||
cd /home/dank/serverAppConfigs/dankserver && podman-compose up -d --force-recreate copyparty
|
||||
```
|
||||
Since 2026-09-01: `dankserver/copyparty/quadlet/copyparty.container` →
|
||||
`copyparty.service`, deployed with
|
||||
`./scripts/deploy.sh dankserver --service copyparty`. Cutover details:
|
||||
|
||||
Same project (`dankserver`), only copyparty is recreated; a fresh container
|
||||
answers `000` for the first seconds while it mounts.
|
||||
- v1 generator rejects `Command=` → the argv lives in `Exec=`.
|
||||
- No `HealthCmd=` — the v1 generator splits unquoted spaces in that key;
|
||||
self-healing is `Restart=always`.
|
||||
- `PublishPort=3923:3923` — the first cutover dropped it (unit joined
|
||||
`dankserver_appnet`), copyparty answered inside the netns only; the
|
||||
published-port seam test now pins 3923 alongside 8085/8096/8083.
|
||||
- Config root moved `/opt/copyparty` → `/home/dank/Containers/copyparty/cfg`.
|
||||
- The deployer refuses to start the unit before the env file exists, so a
|
||||
missing seed cannot create an admin account with a garbage password.
|
||||
|
||||
## Healthcheck
|
||||
|
||||
|
||||
@@ -31,7 +31,7 @@ graph LR
|
||||
J --> Cache[Jellyfin cache]
|
||||
```
|
||||
|
||||
## Docker Compose
|
||||
## Docker Compose (reference — retired 2026-09-01; live mounts use `/home/dank/Containers/jellyfin`, see Deployment below)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -82,11 +82,23 @@ Set **Base URL** to `/` (or the path you configure) in Jellyfin Dashboard → Ne
|
||||
|
||||
| Mount | Purpose | Size Estimate |
|
||||
|-------|---------|---------------|
|
||||
| `/opt/jellyfin/config` | SQLite database, settings, cached images | 10–100 GB |
|
||||
| `/opt/jellyfin/cache` | Transcoding segments, extracted posters | Self-cleaning per session |
|
||||
| `/home/dank/Containers/jellyfin/config` | SQLite database, settings, cached images | 10–100 GB |
|
||||
| `/home/dank/Containers/jellyfin/cache` | Transcoding segments, extracted posters | Self-cleaning per session |
|
||||
| `/mnt/entropy/Movies`, `/mnt/entropy/TVShows` | Media, **read-only binds** (`:ro`) — no automated write path | — |
|
||||
|
||||
The database must be on local storage (SSD/NVMe preferred). Media libraries can be on NFS/mergerfs. If media is on NFS, set `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`.
|
||||
|
||||
## Deployment (Quadlet)
|
||||
|
||||
Since 2026-09-01: `dankserver/jellyfin/quadlet/jellyfin.container` →
|
||||
`jellyfin.service` (generator-enabled), deployed with
|
||||
`./scripts/deploy.sh dankserver --service jellyfin`. GPU passthrough uses
|
||||
`AddDevice=/dev/dri` — plain `Device=` is **not** a quadlet key and makes the
|
||||
v1 generator reject the whole unit (learned live: "unsupported key 'Device'
|
||||
in group 'Container'"). Entropy mounts stay `:ro`; the container runs as
|
||||
root inside the user namespace, so the render-node ACLs apply via the device
|
||||
cgroup regardless of `GroupAdd=keep-groups`.
|
||||
|
||||
## Authentication
|
||||
|
||||
- **Built-in auth (default):** Per-user libraries, parental controls, max bitrate limits. Manage via Dashboard → Users.
|
||||
@@ -147,7 +159,7 @@ Run Jellyfin alongside Plex for an incremental migration. Point Jellyfin at the
|
||||
|
||||
1. Verify GPU on host: `ls -l /dev/dri/`, `lspci | grep -i vga`, `getent group render`.
|
||||
2. Install media drivers: `sudo apt install intel-media-va-driver-non-free vainfo intel-gpu-tools`.
|
||||
3. Create directories: `/opt/jellyfin/config`, `/opt/jellyfin/cache`.
|
||||
3. Create directories: `/home/dank/Containers/jellyfin/config`, `/home/dank/Containers/jellyfin/cache`.
|
||||
4. Write compose file, verify device and GID.
|
||||
5. Deploy: `podman compose up -d`.
|
||||
6. Test GPU passthrough: `podman exec jellyfin /usr/lib/jellyfin-ffmpeg/vainfo --display drm --device /dev/dri/renderD128`.
|
||||
|
||||
@@ -73,7 +73,7 @@ These are set via environment variables or in `config.yml`:
|
||||
|
||||
### Config File
|
||||
|
||||
RomM uses `config.yml` for scan settings, available through the admin UI (Scan Settings). Place the file at `/opt/romm/config/config.yml` on the host.
|
||||
RomM uses `config.yml` for scan settings, available through the admin UI (Scan Settings). Place the file at `/home/dank/Containers/romm/config/config.yml` on the host.
|
||||
|
||||
Key scan config options:
|
||||
|
||||
@@ -99,17 +99,40 @@ scan:
|
||||
export: true # Export ES-DE compatible gamelist.xml
|
||||
```
|
||||
|
||||
## Deployment (Quadlet)
|
||||
|
||||
Since 2026-09-01 the trio runs as repo-tracked quadlet units, deployed with
|
||||
`./scripts/deploy.sh dankserver --service romm` (any of `romm`,
|
||||
`romm-mariadb`, `romm-redis` triggers the full ordered cutover):
|
||||
|
||||
- Units: `dankserver/romm/quadlet/{romm,romm-mariadb,romm-redis}.container`
|
||||
→ generated `romm.service` etc., enabled via the quadlet generator's
|
||||
`default.target.wants` (`is-enabled` reports `generated`).
|
||||
- Secrets: host-local `/home/dank/Containers/romm/env` (0600, 15 app vars —
|
||||
compose-provided keys only; image-baked vars like `HOME`/`PATH` must never
|
||||
enter it, they poison the podman client's service environment).
|
||||
- Network: `dankserver_appnet` with aliases `mariadb`/`redis` (replaces
|
||||
compose service discovery; `DB_HOST`/`REDIS_HOST` still resolve).
|
||||
- Publish: `8085:8080` (`PublishPort=`, see the seam test on published ports).
|
||||
|
||||
## Volumes
|
||||
|
||||
| Container path | Host path (convention) |
|
||||
|----------------|------------------------|
|
||||
| `/romm/library` | `/opt/romm/library` |
|
||||
| `/romm/resources` | `/opt/romm/resources` |
|
||||
| `/romm/config` | `/opt/romm/config` |
|
||||
| Container path | Host path |
|
||||
|----------------|-----------|
|
||||
| `/romm/library` | `/home/dank/Containers/romm/library` |
|
||||
| `/romm/resources` | `/home/dank/Containers/romm/resources` |
|
||||
| `/romm/config` | `/home/dank/Containers/romm/config` |
|
||||
| `/var/lib/mysql` | `/home/dank/Containers/romm/mariadb` (mapped subuid owner — never chown) |
|
||||
| `/data` (redis) | `/home/dank/Containers/romm/redis` (mapped subuid owner) |
|
||||
|
||||
The library volume contains ROM files and subdirectories per platform. The resources volume stores cached artwork, metadata, and the SQLite/SQLite-backed asset cache. The config volume holds `config.yml` and other configuration.
|
||||
The library volume contains ROM files and subdirectories per platform. The resources volume stores cached artwork, metadata, and the asset cache. The config volume holds `config.yml` and other configuration.
|
||||
|
||||
## Docker Compose
|
||||
The cutover rescued RomM's anonymous `/romm` volume (assets, plus empty
|
||||
mountpoint stubs that shadowed nothing) into the data root before first unit
|
||||
start — the mechanism behind the original "RomM data wiped" incident is now
|
||||
structurally impossible: the units declare bind mounts only.
|
||||
|
||||
## Docker Compose (reference — retired 2026-09-01, superseded by the quadlet units above)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
|
||||
@@ -11,6 +11,45 @@ timestamp: 2026-09-14
|
||||
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-09-01 — dankserver media stack standardized to quadlet; dankclaw data roots migrated
|
||||
- **Incident context**: Dankserver hard-crashed and rebooted 2026-09-01; every
|
||||
bare compose-era media container (RomM trio, Jellyfin, Calibre-Web,
|
||||
Copyparty) stayed dead at boot — the failure mode the Boot Persistence rule
|
||||
exists for. Recovery doubled as the Phase 1 cutover.
|
||||
- **Phase 1 (dankserver)**: six quadlet units shipped
|
||||
(`dankserver/{romm,jellyfin,calibre-web,copyparty}/quadlet/`) + first-cutover
|
||||
deploy handlers (retire bare containers → rescue RomM's anonymous `/romm`
|
||||
volume → merge-move `/opt/<svc>` into `/home/dank/Containers/<svc>` →
|
||||
ordered start with health gates). All six verified: units `active`,
|
||||
`is-enabled=generated` (wants in the generator dir), ports 8085→8080,
|
||||
8096, 8083, 3923 answering, jellyfin holds `/dev/dri` render nodes,
|
||||
entropy binds provably ro except copyparty's `/w`. MariaDB continuity
|
||||
proven via the real `dank` admin row (created 2026-08-31), not a fresh
|
||||
bootstrap. Stale compose-era `container-*.service` generate-systemd units
|
||||
retired to `~/.AGENTTRASH/20260901-stale-generate-systemd`.
|
||||
- **Deployer findings fixed** (each caught live, each with a new seam):
|
||||
v1 generator rejects `Command=` (→ `Exec=`), `Device=` (→ `AddDevice=`,
|
||||
whole-unit rejection is silent), and splits spaces in `HealthCmd=`
|
||||
(→ omitted for copyparty); `migrate_data_root` must merge into pre-seeded
|
||||
destinations and rename mapped-subuid roots (100998) with `sudo mv`
|
||||
(numeric-preserving, `keep` mode skips the chown — chowning mapped data
|
||||
breaks the in-container users); quadlet unit dir must be created on a host
|
||||
that never had quadlet; missing `PublishPort` = app answers inside the
|
||||
netns only (copyparty 3923); seeded env files must carry **compose-provided
|
||||
vars only** — the image-baked `HOME=/root` in a `[Service]
|
||||
EnvironmentFile=` kills the podman client itself.
|
||||
- **Phase 2 (dankclaw)**: data roots of gitea, authentik, hermes trios moved
|
||||
to `/home/dank/Containers/<svc>/` while running (rename is inode-bound —
|
||||
zero downtime), units repointed, redeployed through `deploy.sh --service`;
|
||||
auth chain verified end-to-end (outpost ping → 302 to flow).
|
||||
`deploy_hermes` ordering bug fixed: generated units must be reloaded
|
||||
**after** unit files are copied, not before (stale ExecStart crash-looped
|
||||
the trio on the old path).
|
||||
- **Docs**: [[RomM]], [[Jellyfin]], [[Calibre-Web]], [[Copyparty]] updated to
|
||||
the quadlet reality; [[Storage Policy]] now documents the Containers layout
|
||||
+ the two ownership regimes; [[Deployment Flow]] service-manager table
|
||||
corrected (dankserver media is no longer "bare containers, inventory only").
|
||||
|
||||
## 2026-08-30 (night II) — frank-default-sightsound deployed; NPU SVA/IOMMU root cause
|
||||
- **Deployed** `frank-default-sightsound` on Dankclaw: FLM v1.0.1 (`/usr/bin/flm`,
|
||||
dpkg `fastflowlm`) serving the whole local registry (36 tags) on port 52626;
|
||||
|
||||
Reference in New Issue
Block a user