Compare commits

..
Author SHA1 Message Date
dank 0ecfe830f7 Merge pull request 'fix(deploy): relay firewall DROP must be appended, not -I-inserted above its ACCEPTs' (#75) from fix/signal-relay-fw-order into main 2026-10-21 16:59:16 -07:00
Frank ce370a5b28 fix(deploy): relay firewall DROP must be appended, not -I-inserted above its ACCEPTs
iptables -I prepends, so the install block's final -I of the blanket DROP
landed it at position 1, above all three ACCEPTs: every packet to
18093/tcp died there (live 2026-10-21: 159 packets on the DROP, 0 on each
ACCEPT). The host's own health probe timed out beside a LISTENing socket
and the whole dankclaw drain aborted at the alertmanager gate twice.

Install now rebuilds the tail: delete any blanket DROP wherever it sits,
keep the ACCEPTs at the head, append the DROP last - converges on hosts
poisoned by the old order. New bats case runs install_signal_relay for
real against an iptables sandbox seeded with the poisoned order (red on
the old code, green on this one).
2026-10-21 16:52:40 -07:00
Frank bfe4d185bd docs(vault): OpenWebUI picker opened to all three models for every account (2026-10-21)
Per Frank, frank-archon is public alongside dragoon/zealot: three-leg DB
trio (conn enable, model row ctx 200000, access_grant principal='*'),
verified as role=user. Token-mint traps recorded (cwd, PID-1 secret).
2026-10-21 16:48:32 -07:00
dank 94a9f3aa31 Merge pull request 'feat(security): digest-pin every registry image; drop batch-killing HostAddr' (#74) from security/quadlet-digest-pins into main 2026-10-21 16:32:44 -07:00
4 changed files with 109 additions and 19 deletions

No files matched your search

+12 -4
View File
@@ -252,16 +252,24 @@ install_signal_relay() {
# The relay binds the host's LAN IP (the one address a bridge container # The relay binds the host's LAN IP (the one address a bridge container
# can reach as host.containers.internal), so keep the unauthenticated # can reach as host.containers.internal), so keep the unauthenticated
# endpoint off the rest of the LAN: admit loopback, the host itself # endpoint off the rest of the LAN: admit loopback, the host itself
# (its own health probe) and the podman bridge subnet only. Idempotent: # (its own health probe) and the podman bridge subnet only.
# -C before -I. # ORDER IS THE GATE: `iptables -I` inserts at position 1, so the old
# shape (-C/-I x3 ACCEPT, then -C/-I the DROP) landed the blanket DROP
# ABOVE the ACCEPTs and every probe timed out beside a LISTENing socket
# (live 2026-10-21: 159 packets on the DROP, 0 on each ACCEPT; the whole
# dankclaw drain aborted at the alertmanager gate twice before the cause
# was read off `nft list ruleset`). Rebuild the tail instead of inserting
# it: delete any blanket DROP wherever it sits, keep the three ACCEPTs
# at the head (-C before -I, unchanged), then APPEND the DROP so it is
# last and the ACCEPTs decide first. Converges on poisoned hosts.
run_privileged iptables -D INPUT -p tcp --dport 18093 -j DROP 2>/dev/null || true
run_privileged iptables -C INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT 2>/dev/null \ run_privileged iptables -C INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT 2>/dev/null \
|| run_privileged iptables -I INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT
run_privileged iptables -C INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT 2>/dev/null \ run_privileged iptables -C INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT 2>/dev/null \
|| run_privileged iptables -I INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT
run_privileged iptables -C INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT 2>/dev/null \ run_privileged iptables -C INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT 2>/dev/null \
|| run_privileged iptables -I INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT
run_privileged iptables -C INPUT -p tcp --dport 18093 -j DROP 2>/dev/null \ run_privileged iptables -A INPUT -p tcp --dport 18093 -j DROP
|| run_privileged iptables -I INPUT -p tcp --dport 18093 -j DROP
# The interpreter needs a moment to bind after restart; a one-shot curl # The interpreter needs a moment to bind after restart; a one-shot curl
# raced the socket on 2026-09-17 and failed a healthy deploy. # raced the socket on 2026-09-17 and failed a healthy deploy.
local i local i
+26 -10
View File
@@ -34,15 +34,16 @@ https://github.com/open-webui/open-webui
The picker is built from the `model` table in `webui.db` (one row per model) The picker is built from the `model` table in `webui.db` (one row per model)
plus the `model_ids` of each **enabled** OpenAI connection, NOT from live plus the `model_ids` of each **enabled** OpenAI connection, NOT from live
connection discovery — so the picker shows exactly what is seeded there. connection discovery — so the picker shows exactly what is seeded there.
As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`** *(unverified Since 2026-10-21 the picker exposes **all three fleet models to every
2026-10-18: `webui.db` is the authority; the fleet itself serves all three ids account**: `frank-dragoon`, `frank-archon`, `frank-zealot` (per Frank: open
`frank-dragoon`/`frank-zealot`/`frank-archon`, per the cards in `frank-archon` to everyone; the other two were already public). Proven with
`shared/homepage/services.json`)*. a `role=user` token: `/api/models` lists all three ids and a chat turn to
`frank-archon` answers end-to-end.
| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Endpoint page | | Connection (`openai.api_configs` idx) | `openai.api_base_urls` (live 2026-10-21) | Enabled | `model_ids` | Endpoint page |
|---|---|---|---|---| |---|---|---|---|---|
| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | [[frank-dragoon]] | | 0 | `http://host.containers.internal:7770/v1` | yes | `frank-dragoon` | [[frank-dragoon]] |
| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | [[frank-archon]] | | 1 | `http://192.168.0.235:7770/v1` | yes | `frank-archon` | [[frank-archon]] |
| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | [[frank-zealot]] | | 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | [[frank-zealot]] |
The URLs in the first column are the DB's recorded values and predate the 2026-08-22 The URLs in the first column are the DB's recorded values and predate the 2026-08-22
@@ -50,9 +51,10 @@ bridge move; on this network they are wrong — see the reachability gotcha belo
is the corrected law. The picker row for `frank-zealot` was re-keyed on 2026-10-18 after is the corrected law. The picker row for `frank-zealot` was re-keyed on 2026-10-18 after
that day's payload swap (see [[frank-zealot]] for what was deployed and when). that day's payload swap (see [[frank-zealot]] for what was deployed and when).
The `model` table carries exactly two decoration rows, `frank-zealot` and The `model` table carries three decoration rows — `frank-zealot`,
`frank-dragoon`. Both the context length and the description live **inside `frank-dragoon`, `frank-archon` (200,000 context, added 2026-10-21).
JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds Both the context length and the description live **inside JSON columns** —
`params` holds `{"context": {"length": …}}` and `meta` holds
`{"description": …}`; there is no `context_length` or `description` column, so `{"description": …}`; there is no `context_length` or `description` column, so
`sqlite3` one-liners against those names fail. `params.context.length` is a copy `sqlite3` one-liners against those names fail. `params.context.length` is a copy
of the endpoint's per-slot window, so the authoritative number lives on the of the endpoint's per-slot window, so the authoritative number lives on the
@@ -380,6 +382,20 @@ neither.
name: /selected model/i })`) and the welcome text "How can I help you name: /selected model/i })`) and the welcome text "How can I help you
today?"; that snapshot is also the cheapest proof the picker is populated. today?"; that snapshot is also the cheapest proof the picker is populated.
- **2026-10-21 — picker opened to all three models, for everyone.** Per
Frank: `frank-archon` accessible to everyone. Three-leg repair on
`webui.db` (snapshot `webui.db.bak-20261021` first): connection idx 1
`enable: true`, a `model` row `frank-archon` (params
`{"context":{"length":200000}}`, meta description from the endpoint card),
and an `access_grant` row `(resource_type='model', resource_id='frank-archon',
principal_id='*', 'read')` — `principal_id='*'` is the public grant, not a
role row. Verified as a plain `role=user` token: `/api/models` lists all
three ids, chat to `frank-archon` answers. Token-mint gotchas: the backend
package imports only with cwd `/app/backend`, and `WEBUI_SECRET_KEY` must
come from **PID 1's environ** (`podman exec` sessions do not inherit the
app process env; the `.webui_secret_key` file happens to equal it — use it
explicitly, `podman exec -e`).
## See Also ## See Also
- [[Dankclaw Stack]] - [[Dankclaw Stack]]
@@ -74,11 +74,19 @@ published ports — node `:9100`, cadvisor `:8080`, gpu `:9401` all answer there
— while the host's own LAN IP is refused (`192.168.0.143:8080`, `:9401`), so — while the host's own LAN IP is refused (`192.168.0.143:8080`, `:9401`), so
every same-host Prometheus target uses the gateway and only cross-host targets every same-host Prometheus target uses the gateway and only cross-host targets
(dankserver) use plain LAN. Whether LAN works at all is network-specific: the (dankserver) use plain LAN. Whether LAN works at all is network-specific: the
relay at `192.168.0.143:18093` IS reached from alertmanager's network, with relay at `192.168.0.143:18093` is reached from alertmanager's network only
iptables INPUT admitting loopback, the host itself and the podman bridge subnet while `install_signal_relay`'s INPUT rules are in force: loopback, the host
`10.89.0.0/16` before the DROP (`scripts/deploy-remote.sh:255-262`). Never itself and the podman bridge subnet `10.89.0.0/16` accepted **above** a
point a scrape target at a host LAN IP from its own host on the strength of trailing blanket DROP. The law the poisoned host taught (2026-10-21):
one working example; dial the gateway and probe. `iptables -I` prepends, so inserting the DROP last lands it first — the old
install order dropped every packet (159 on the DROP, 0 on the ACCEPTs), the
deployer's health gate failed beside a LISTENing socket, and the drain
aborted twice before `nft list ruleset` named the cause. The install now
rebuilds the tail (delete the DROP wherever it sits, ACCEPTs at the head,
DROP appended); verify any change to it with the sandboxed bats case, not
with `-C` greps — `-C` answers existence, never order. Never point a scrape
target at a host LAN IP from its own host on the strength of one working
example; dial the gateway and probe.
## Deployment ## Deployment
+58
View File
@@ -559,6 +559,64 @@ assert any('ALERTS' in e for e in exprs), exprs
grep -q -- '--dport 18093 -j DROP' "$handler" grep -q -- '--dport 18093 -j DROP' "$handler"
} }
@test "the deployer installs the relay firewall with the ACCEPTs deciding before the DROP" {
# Behavioral, run for real against a sandbox that plays the kernel: the
# rule set is a LIST and iptables -I prepends, so the shape this function
# had until 2026-10-21 (-C/-I ACCEPT x3, then -C/-I the blanket DROP)
# landed the DROP at position 1 and every packet died there — the drain
# failed its health gate beside a LISTENing socket twice. Seeding the
# sandbox with that poisoned order also pins the convergent repair: a
# re-deploy must rewrite the tail, not skip because -C finds every rule.
local sandbox="$BATS_TEST_TMPDIR/fw"
mkdir -p "$sandbox/bin"
cat > "$sandbox/bin/iptables" <<'EOF'
#!/usr/bin/env bash
# ordered rule list in $FW_STATE: -C lookup, -I at head, -A at tail, -D delete.
state="${FW_STATE:?}"
touch "$state"
op="$1"; shift
case "$op" in
-C) grep -Fxq -- "$*" "$state" ;;
-I) grep -Fxq -- "$*" "$state" || { { echo "$*"; cat "$state"; } > "$state.new"; mv "$state.new" "$state"; } ;;
-A) grep -Fxq -- "$*" "$state" || echo "$*" >> "$state" ;;
-D) if grep -Fxq -- "$*" "$state"; then
grep -Fxv -- "$*" "$state" > "$state.new"; mv "$state.new" "$state"
else exit 1; fi ;;
*) exit 0 ;;
esac
EOF
chmod +x "$sandbox/bin/iptables"
# Poisoned state exactly as the pre-fix deployer left Dankclaw.
printf '%s\n' \
'INPUT -p tcp --dport 18093 -j DROP' \
'INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT' \
'INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT' \
'INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT' \
> "$sandbox/rules"
(
cd "$REPO_ROOT"
export PATH="$sandbox/bin:$PATH" FW_STATE="$sandbox/rules" HOME="$sandbox/home"
mkdir -p "$HOME"
eval "$(sed -n '/^install_signal_relay()/,/^}/p' scripts/deploy-remote.sh)"
run_privileged() { "$@"; }
systemctl() { :; }
curl() { return 0; }
sleep() { :; }
install_signal_relay
)
# Exactly one blanket DROP, and it sits after every ACCEPT.
[ "$(grep -cF -- 'INPUT -p tcp --dport 18093 -j DROP' "$sandbox/rules")" -eq 1 ]
local drop_at total
drop_at="$(grep -nF -- 'INPUT -p tcp --dport 18093 -j DROP' "$sandbox/rules" | cut -d: -f1)"
total="$(wc -l < "$sandbox/rules")"
[ "$drop_at" -eq "$total" ]
local s seen
for s in 127.0.0.1 192.168.0.143 10.89.0.0/16; do
seen="$(grep -nF -- "-s $s -j ACCEPT" "$sandbox/rules" | cut -d: -f1)"
[ -n "$seen" ] && [ "$seen" -lt "$drop_at" ]
done
}
@test "the planner selects alertmanager for its config and relay files" { @test "the planner selects alertmanager for its config and relay files" {
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankclaw \ run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankclaw \
--paths shared/prometheus/alertmanager.yml --paths shared/prometheus/alertmanager.yml