diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..b1b4dc7 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,4 @@ +*.jpg filter=lfs diff=lfs merge=lfs -text +*.jpeg filter=lfs diff=lfs merge=lfs -text +*.png filter=lfs diff=lfs merge=lfs -text +*.webp filter=lfs diff=lfs merge=lfs -text diff --git a/.githooks/pre-push b/.githooks/pre-push new file mode 100755 index 0000000..ce3f685 --- /dev/null +++ b/.githooks/pre-push @@ -0,0 +1,6 @@ +#!/bin/sh +# git-lfs pre-push — required because core.hooksPath=.githooks disables the +# default .git/hooks/pre-push; without this hook, pushes would upload LFS +# pointer files without their objects and break every fresh clone. +command -v git-lfs >/dev/null 2>&1 || { printf >&2 'n' ; exit 0; } +git lfs pre-push "$@" diff --git a/.gitignore b/.gitignore index 22223e1..8f2272f 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,34 @@ last-changelog-version .DS_Store *.swp *.tmp + +# --- per-host runtime identity + caches (never config) --------------------- +agent/blobs/ +install-id +puppeteer/ +webcache/ + +# --- local backups + staging artifacts: kept on disk, never in history ----- +.AGENTTRASH/ +*.save +*.bak-* + +# deploy quarantine — never tracked +.quarantine/ + +# harness runtime transcripts: local-only, may contain raw session secrets +agent/sessions/ + +# harness diagnostic report bundles (tarballs of logs + session state) +reports/ + +# --- agent runtime scratch: research runs, decode predictions, worktrees ---- +autoresearch/ +agent/predict/ +local/ +wt/ + +# --- per-worktree memory buckets: keyed on ephemeral worktree paths --------- +# A session run inside a git worktree mints a bucket named after the worktree +# path; the worktree is deleted, the bucket is orphan state, not config. +agent/memories/--home-blank-.omp-wt-* diff --git a/.pre-pull-backup/agent/managed-skills/command-center-dotfiles-import/SKILL.md b/.pre-pull-backup/agent/managed-skills/command-center-dotfiles-import/SKILL.md new file mode 100644 index 0000000..e06cfd9 --- /dev/null +++ b/.pre-pull-backup/agent/managed-skills/command-center-dotfiles-import/SKILL.md @@ -0,0 +1,28 @@ +--- +name: command-center-dotfiles-import +description: Runbook for importing client env-layer configs into the command-center repo and deploying them via stow on new fleet machines; decisions agreed with Frank 2026-09-09 (pre-execution — verify execution state before assuming done). +--- + +Design agreed with Frank for importing client application/environment configs into the `~/.omp` command-center repo and deploying them on fresh machines. **Status: plan agreed, not yet executed** — check repo state (does `dotfiles/` or `scripts/deploy.sh` exist?) before re-deriving; if already implemented, this becomes the operating model, not a to-do. + +## Agreed decisions (each was an explicit user choice) +1. **Target**: unknown/TBD fresh machine → design stays host-agnostic; deploy derives overrides from `$(hostname)` at run time. Nothing hardcodes a target. +2. **Scope**: environment layer only — niri (+cfg includes), gtk-3.0/4.0 settings, ghostty, alacritty, shell rc files (`.zshrc`, `.bashrc`, `.profile`, `.p10k.zsh`), fish, systemd user units, autostart, dconf. App *data* (browser profiles, vaults, DBs) never imported. +3. **Layout**: stow-style package dirs `dotfiles//` inside the repo; repo clones to `~/.omp` on every host that runs the harness (full command-center clone, not env-layer only). +4. **Deploy mechanism**: symlink farm — `stow -t ~` from `~/.omp/dotfiles`; relative symlinks within `$HOME` work because repo root lives inside `$HOME`. +5. **Dogfood order**: convert the workstation itself (real dirs → symlinks via adopt) BEFORE any target import; iterate where the compositor reload is observable. +6. **Secrets policy**: never commit secret values. Split rc files into committed logic + untracked sourced-if-exists stub (`~/.config/environment.d/secrets.env` or `.zshrc.local`); real values provisioned out-of-band only on machines that need them. Pre-commit secret scanner stays as safety net, not gate of first resort. +7. **Divergence**: `zz-` override package stowed LAST with `--override`; only per-host diffs live there. (Stow descends into conflicting symlinked dirs — per-file overrides only work via this last-stowed `--override` pattern.) +8. **Deploy trigger**: explicit `scripts/deploy.sh` invocation, human- or agent-invoked. NO auto-deploy on pull — pull and deploy are separate decisions; `--check` = drift report mode. +9. **Ownership**: Frank is sole push authority; exactly one editing agent at a time → no multi-writer race machinery needed. Each host's agent edits only its own `zz-` package. +10. **Conflict policy**: pre-existing regular file that differs from repo truth → move to timestamped `.quarantine/` dir on the target (never `rm`, same convention as this repo's trash rule), symlink anyway, print every replacement in deploy output and `--check` report. Repo truth always wins; divergence recoverable from quarantine. +11. **First-clone auth**: human types credential once console-side (URL-with-token or prompt); the new machine's secrets exist only there afterward. + +## Implementation gotchas baked into the design +- `core.hooksPath` is a per-clone local setting — does NOT survive clone; bootstrap/deploy must set it on every fresh clone or the secret-gate pre-commit hook silently doesn't run there. +- dconf/gsettings are databases, not files: import as `dconf dump /` text into repo; deploy runs `dconf load` instead of symlinking. +- p10k installer and toolchain hooks append to rc files at install time — expect conflicts on `.zshrc`-class files at first adopt; quarantine-and-replace handles it, but diff live-vs-repo before adopting so pre-existing drift is seen, not silently adopted. +- Known secret at import time: `GITEA_TOKEN` export lived in `.zshrc` (line ~22) on the workstation — quarantine BEFORE importing rc files; consider token rotation at cutover since it had been exposed in plaintext rc + remote URL. + +## Deploy script spec agreed +`scripts/deploy.sh`: sets hooksPath → stows all base packages → stows `zz-$(hostname)` with `--override` → first run uses `--adopt` for pre-existing identical files → divergent files quarantined-and-reported; `--check` mode emits drift report (modified tracked files vs deployed symlinks) without mutating. diff --git a/.pre-pull-backup/agent/managed-skills/stash-supersession-audit-before-drop/SKILL.md b/.pre-pull-backup/agent/managed-skills/stash-supersession-audit-before-drop/SKILL.md new file mode 100644 index 0000000..aebd555 --- /dev/null +++ b/.pre-pull-backup/agent/managed-skills/stash-supersession-audit-before-drop/SKILL.md @@ -0,0 +1,33 @@ +--- +name: stash-supersession-audit-before-drop +description: "Before dropping a stale git stash, prove it's superseded by HEAD via per-file line-set diff of git show stash@{0}:file vs HEAD; archive as an annotated tag and drop only after auditing unique facts — zero-loss alternative to blind drop." +--- + +# Stash Supersession Audit Before Drop + +Use when a repo (especially a live-deployment checkout where the working tree IS the deployment, e.g. `~/.omp` command-center) has an old stash whose content may have been superseded by later commits, and dropping blindly risks losing unique facts. + +## Procedure + +1. **Inventory the stash** — see which files it touches: + ```bash + git stash show --stat stash@{0} + ``` +2. **Probe for unique content per file.** For each touched file, diff stash version against HEAD and count lines that exist only in the stash version: + ```bash + git show "stash@{0}:" > /tmp/stash_copy + # compare normalized line sets; report lines present in stash but absent from HEAD + ``` + A file with `unique-vs-HEAD = 0` is fully superseded — safe to drop for that file. +3. **Classify each unique-content hit**: + - *Stale fact* (superseded by a later refactor, e.g. old model/provider naming) → drop is correct. + - *Live durable fact* (e.g. an operational invariant never re-recorded) → extract it into the live memory/docs file or flag to the user before dropping. +4. **Archive before dropping** — zero-loss: tag the stash commit, then drop from the stash list: + ```bash + git tag archive/stash---wip "stash@{0}" + git stash drop "stash@{0}" + ``` + The tag pins the full stash object (including its index parent); recoverable later via `git show archive/stash-...` or `git checkout archive/stash-...^3 -- ` for the untracked-files third parent if present. + +## Why +Stashes are invisible in normal history and expire from reflogs (~90 days). A stash that *looks* superseded can still hold the only copy of a durable fact; tagging converts "drop" into a reversible archive with zero workflow cost. diff --git a/AGENTS.md b/AGENTS.md index 1b0db9f..ac0e276 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ System-configuration command center for Frank's fleet: agent config, knowledge base, small system utilities, and (soon) dotfiles. You — the reader — are probably an agent -running on Dankware with this repo as your working directory. Read +running on one of Frank's workstations with this repo as your working directory. Read `omp-docs/index.md` before you touch anything. This file is the operating contract; `omp-docs/` is the knowledge base (OKF v0.1 format: directory of markdown files with YAML frontmatter, spec at `omp-docs/references/okf-spec.md`). @@ -36,6 +36,26 @@ with YAML frontmatter, spec at `omp-docs/references/okf-spec.md`). - Trash convention: never `rm` anything user-authored — move it to `.AGENTTRASH/` (gitignored). Deletion is a pruning session with Frank present. +### Commit hygiene (mirrors dank/serverAppConfigs) + +The law is imported from the serverAppConfigs repo — checkout on the dev box at +`~/Projects/serverAppConfig`, see its `AGENTS.md` § *Git Workflow* and +§ *Testing* ("All edits flow through git", "tests are mandatory before +commit"). Applied here: + +- `git pull` before starting new work; start from a clean tree. The checkout + IS the deployment here — a dirty tracked file is a half-deployed config + serving the live agent. +- Commit atomically and early: one commit = one complete functional unit, + never a pile of half-edited files; docs land in the same commit as the + change they document. +- `bats tests/` green + the pre-commit secret gate = the commit precondition; + the full suite runs at pre-push. +- Push to the local Gitea as soon as a unit is green — every other + workstation pulls from there. `git * --no-verify` is forbidden (the secret + gate is the point of the hook). +- Messages: imperative mood, what + why, `Refs #N` when a Gitea issue exists. + ## Secret gate (HARD rule) The pre-commit hook (`.githooks/pre-commit`, enabled via @@ -50,7 +70,7 @@ secret in every agent's context window on every machine that pulls this repo. | box | role | GPU | notes | |-----|------|-----|-------| -| Dankware | dev laptop, never runs inference | Arc iGPU + RTX 4070 Max-Q (dGPU, non-unified) | 22 threads; mainline llama.cpp at `~/GitPackages/llama.cpp/build/bin/`; this repo lives here at `~/.omp` | +| Dev workstation | client box, never runs inference | laptop iGPU + dGPU (non-unified) | mainline llama.cpp at `~/GitPackages/llama.cpp/build/bin/`; this repo lives here at `~/.omp` | | Dankclaw | bench target, runs the large serving model | Strix Halo APU — **unified memory** | ROCmFPX fork at `~/GitPackages/ROCmFPX/build-qwen38/bin/`; prod service `frank-default-large`, port 7770; ssh: `ssh dankclaw` | | Dankserver | FreeToken box | RTX 3090 dGPU — non-unified | runs `frank-default-small`; ssh: `ssh dankserver` | @@ -68,7 +88,7 @@ secret in every agent's context window on every machine that pulls this repo. repo's operator: never stop, mask, or restart it (or any production service on any box) without an explicit, current, per-run OK from Frank. -## File-organization doctrine (why this repo won't rot like ~/.pi did) +## File-organization doctrine (why this repo stays readable) - One concept, one file; directories get an `index.md` listing their contents — that's the OKF contract, and it is also the anti-rot mechanism: nothing may diff --git a/README.md b/README.md new file mode 100644 index 0000000..99bd283 --- /dev/null +++ b/README.md @@ -0,0 +1,86 @@ +# Command Center + +## Configuration repository for Frank's Userspace configurations (i.e. his desktop setup) + +### AI Disclosures: +- Slop rating: 3/5 - **Actively using, actively de-slopping, repository still mostly AI owned** +- Completeness: **Unfinished, planning planned, no plan yet** +- Datacenter model/spend: Less than $5 of Deepseek 4.1 Flash +- Local model: Qwen 3.8 27B + +### slop rating of 3 or higher indicates human reading is a waste of time outside of this README, and that you might as well ask an agent for help with this. +### slop rating 3 indicates Frank is comfortable using this for professional use -- but barely. + + + +## The Fleet + +| Host | Role | IP | +|------|------|-----| +| **Workstation** | Client box (CachyOS, Niri/Wayland) | — | +| **Dankclaw** | Server (Podman, reverse proxy, Gitea, NPU) | 192.168.0.143 | +| **Dankserver** | LLM inference backend | 192.168.0.235 | + +The workstation hosts the primary checkout — this repository lives at `~/.omp/` and drives the OMP agent harness. + +## Structure at a Glance + +``` +~/.omp/ +├── agent/ # OMP harness runtime +│ ├── config.yml # Live agent configuration +│ ├── models.yml # Model providers (single source of truth) +│ ├── APPEND_SYSTEM.md # System prompt extensions +│ ├── managed-skills/ # Managed skills (SKILL.md bundles) +│ ├── models.db # Model cache +│ └── sessions/ # Persistent session state +├── dotfiles/ # stow-style env-layer packages (home-relative trees) +│ └── deploy via scripts/deploy.sh (check|adopt|sync; quarantine-and-replace) +├── omp-docs/ # Knowledge base (source of truth) +│ ├── index.md # KB index — start here +│ ├── system/ # Fleet hardware, OS, network +│ ├── tools/ # Installed software (llama.cpp, Gitea, Caddy, niri…) +│ ├── core/ # Agent config conventions (settings, models, AGENTS.md) +│ ├── pi/ # Legacy pi-era config reference +│ ├── engineering/ # SDLC, languages, infra, skills ecosystem +│ ├── research/ # Decision-support research (not yet facts) +│ ├── references/ # OKF spec, maintenance procedures +│ ├── caddy/ # Reverse proxy configuration +│ ├── packages/ # Pi packages and extensions +│ ├── inventory/ # Userspace config census +│ ├── reports/ # Investigation reports +│ ├── log.md # Change log +│ └── .okf/ # OKF v0.1 conformance config +├── scripts/ # Utility scripts (kb-lint.sh) +├── plugins/ # npm plugins (pi-peon, etc.) +├── puppeteer/ # Browser automation config +├── .githooks/ # Pre-commit hook +├── .gitignore # Runtime state ignored (databases, logs, cache) +├── AGENTS.md # Agent conventions (niri reload, keybindings) +└── README.md # This file +``` + +## What Lives Here + +**Config in, state out.** Everything tracked is something an agent reads or writes. Runtime databases (`*.db`, `*.sqlite`), logs, caches, and session state are git-ignored — git history captures configuration changes, not agent heartbeat. + +## Quick Start + +| Task | How | +|------|-----| +| Read the knowledge base | `omp-docs/index.md` — every concept linked from here | +| Check agent config | `agent/config.yml` + `agent/models.yml` | +| Lint the KB | `scripts/kb-lint.sh` (OKF v0.1 conformance) | +| View change log | `omp-docs/log.md` | +| Run a pre-commit | `.githooks/pre-commit` | + +## Conventions + +- **OKF v0.1** — every doc file carries YAML frontmatter with a `type:` field. `index.md` files are directory listings (no frontmatter). +- **Progressive disclosure** — `index.md` is a landing page; drill into subdirectories for detail. +- **Verified facts** — stamped with `verified:` when probed against the live fleet. Trust over memory. +- **Single config source** — `~/.omp/agent/models.yml` is the only model registry; OMP is the primary harness and there is no second copy to keep in sync. + +## Legacy Notes + +`omp-docs/pi/` and `omp-docs/core/` retain general documentation about the pi-coding-agent harness (its config format, packages, skill system) for reference. OMP at `~/.omp/` is the live harness; managed skills live in `~/.omp/agent/managed-skills/`. diff --git a/agent/APPEND_SYSTEM.md b/agent/APPEND_SYSTEM.md index f5a37a6..d40a980 100644 --- a/agent/APPEND_SYSTEM.md +++ b/agent/APPEND_SYSTEM.md @@ -13,7 +13,7 @@ Frank is 35 year old Software Engineer, specializing in DevOps, living in Seattl He likes video games, science fiction, and fantasy novels. He considers himself somewhat of a history buff. ### Your identity: -You are Comrade Chief Designer of the Soviet Computer Systems of Frank, which are comprised of servers Dankware and Dankclaw, as well as several clients, including Dankware, which is where you are running from. You operate from the host Dankware as a Staff Software Engineer with the eyes of the greatest industrial designers that have ever lived. Growing up under Soviet paranoia, you also have an innate sense for good security practices. Despite your outwardly sunny disposition, you will work diligently as if a one way trip to The Gulag is always a possibility. +You are Comrade Chief Designer of the Soviet Computer Systems of Frank, which are comprised of the servers Dankclaw and Dankserver, plus several client workstations. You operate from whichever workstation you are running on, as a Staff Software Engineer with the eyes of the greatest industrial designers that have ever lived. Growing up under Soviet paranoia, you also have an innate sense for good security practices. Despite your outwardly sunny disposition, you will work diligently as if a one way trip to The Gulag is always a possibility. ### Adjectives that describe your personality: well-measured, cautious, expeditious, ambitious, concise, surgical, thorough, and efficient. @@ -57,7 +57,7 @@ Suggest these tools or languages as possible solutions to problems if they are a The knowledge base at `docs/` is the source of truth for your project. Your training is out of date. Default to the documentation and the code before you assume anything. Maintain a work log of actions taken and their result. Refer to the work log to remember the progress of your tasks. 1. Read the relevant `index.md` before you touch a topic. -2. Follow `~/.pi/agent/docs/references/okf-spec.md` for every doc you create or edit. +2. Follow `omp-docs/references/okf-spec.md` for every doc you create or edit. 3. Read the target file completely before you edit it. 4. Always update the documentation when you've completed a task. 5. Compact the work log when necessary. diff --git a/agent/config.yml b/agent/config.yml index b6e6da1..fb3af30 100644 --- a/agent/config.yml +++ b/agent/config.yml @@ -1,50 +1,67 @@ providers: - webSearchOrder: - - searxng - - perplexity - - gemini - - anthropic - - codex - - xai - - zai - - exa - - tinyfish - - jina - - kagi - - tavily - - firecrawl - - brave - - kimi - - parallel - - synthetic - - startpage - - duckduckgo - - ecosia - - google - - mojeek - - public + openrouterVariant: floor modelRoles: - default: llamacpp (dankclaw)/frank-default-large:medium - smol: llamacpp (dankserver)/frank-default-small:xhigh - slow: llamacpp (dankclaw)/frank-default-large:medium - vision: llamacpp (dankclaw)/frank-default-large:xhigh - plan: llamacpp (dankclaw)/frank-default-large:medium - designer: llamacpp (dankclaw)/frank-default-large:medium - commit: llamacpp (dankserver)/frank-default-small - tiny: llamacpp (dankserver)/frank-default-small - task: llamacpp (dankserver)/frank-default-small - advisor: llamacpp (dankclaw)/frank-default-large:medium + slow: llamacpp (dankclaw)/frank-dragoon:medium + vision: llamacpp (dankclaw-3090)/frank-zealot:xhigh + plan: llamacpp (dankclaw-3090)/frank-zealot:medium + designer: llamacpp (dankclaw-3090)/frank-zealot:medium + commit: llamacpp (dankclaw-3090)/frank-zealot:medium + tiny: llamacpp (dankclaw-3090)/frank-zealot:off + task: llamacpp (dankclaw-3090)/frank-zealot:off + advisor: llamacpp (dankserver)/frank-archon:medium + default: llamacpp (dankserver)/frank-archon:medium + smol: llamacpp (dankclaw-3090)/frank-zealot:medium + web: web/searxng + speech: local/kokoro + judge: openrouter/~typesafe/jev-latest:auto + dictation: local/parakeet-tdt-0.6b-v3 + memory: llamacpp (dankclaw-3090)/frank-zealot:medium symbolPreset: nerd composer: - shape: claude + shape: band theme: - dark: dark-monokai + dark: dark-terminal + light: light setupVersion: 2 browser: - headless: false + headless: true retry: fallbackChains: - {} + web: + - web/kagi + - web/perplexity + - google-gemini-cli/gemini-2.5-flash + - google-antigravity/gemini-2.5-flash + - google/gemini-2.5-flash + - anthropic/claude-haiku-4-5 + - openai-codex/gpt-5.6-luna + - xai/grok-4.5 + - web/zai + - web/exa + - web/tinyfish + - web/jina + - web/tavily + - web/firecrawl + - web/brave + - web/kimi + - web/parallel + - web/synthetic + - web/startpage + - web/duckduckgo + - web/ecosia + - web/google + - web/mojeek + - web/public + - openai-codex/gpt-6-luna + - openai-codex/gpt-5.6 + - openai-codex/gpt-5.5 + - xai-oauth/grok-4.5 + - web/ollama + speech: + [] + llamacpp (dankserver)/frank-archon: + - llamacpp (dankclaw)/frank-dragoon + maxRetries: 10 searxng: endpoint: search.franksdank.cloud exa: @@ -55,15 +72,23 @@ autolearn: enabled: true autoContinue: true compaction: + remoteEndpoint: http://192.168.0.143:7771/v1/chat/completions methodOrder: - shake - - handoff - soft + - handoff + - snapcompact - remote - idleThresholdTokens: 200000 - thresholdPercent: -1 + idleThresholdTokens: 100000 + thresholdPercent: 85 idleEnabled: true - thresholdTokens: 150000 + thresholdTokens: -1 + dropUseless: false + remoteStreamingV2Enabled: true + asyncEnabled: true + idleTimeoutSeconds: 120 + enabled: true + experimentalContextManagement: false interruptMode: immediate defaultThinkingLevel: medium hideThinkingBlock: false @@ -74,22 +99,57 @@ disabledExtensions: [] statusLine: separator: powerline - preset: nerd - contextLine: annotated - transparent: false - compactThinkingLevel: false + preset: custom + contextLine: embedded + transparent: true + compactThinkingLevel: true showHookStatus: true + sessionAccent: true + leftSegments: + - session_name + - vim + - mode + - hostname + - path + - git + - pr + rightSegments: + - model + - cost + - context_pct + - time_spent + segmentOptions: + model: + showThinkingLevel: true + path: + abbreviate: true + stripWorkPrefix: true + maxLength: 120 + git: + showBranch: false + showStaged: true + showUnstaged: true + showUntracked: true tui: textSizing: true codexResetFireworks: false hyperlinks: auto - tight: false + tight: true display: shimmer: kitt smoothStreaming: true showTokenUsage: true task: showResolvedModelBadge: true + maxConcurrency: 2 + agentModelOverrides: + designer: llamacpp (dankclaw-3090)/frank-zealot + librarian: llamacpp (dankclaw-3090)/frank-zealot + reviewer: llamacpp (dankclaw-3090)/frank-zealot + scout: llamacpp (dankclaw-3090)/frank-zealot + security-reviewer: llamacpp (dankclaw-3090)/frank-zealot + sonic: llamacpp (dankclaw-3090)/frank-zealot + task: llamacpp (dankclaw-3090)/frank-zealot snapcompact: systemPrompt: none toolResults: false @@ -102,13 +162,13 @@ checkpoint: enabled: true readLineNumbers: true startup: - showSplash: true + showSplash: false + changelogMode: summary update: channel: stable features: unexpectedStopDetection: smart stt: - modelName: parakeet submitTrigger: say-submit enabled: true recap: @@ -116,4 +176,41 @@ recap: loop: mode: prompt steeringMode: one-at-a-time -personality: default \ No newline at end of file +personality: default +images: + urls: + backends: + - provider-files + - tailscale + - cloudflared + - litterbox +edit: + mode: hashline +dev: + autoqaConsent: granted +astGrep: + enabled: true +vault: + enabled: true +tts: + localVoice: bm_george +speech: + enabled: false + mode: yield + enhanced: true + voice: am_fenrir +secrets: + enabled: true +read: + renderMarkdown: true + toolResultPreview: true +lsp: + formatOnWrite: true + diagnosticsOnEdit: true +autoResume: false +speechgen: + enabled: true +live: + voice: vale +contextPromotion: + enabled: true \ No newline at end of file diff --git a/agent/managed-skills/authentik-provision-fleet-user/SKILL.md b/agent/managed-skills/authentik-provision-fleet-user/SKILL.md new file mode 100644 index 0000000..fc6eeec --- /dev/null +++ b/agent/managed-skills/authentik-provision-fleet-user/SKILL.md @@ -0,0 +1,42 @@ +--- +name: authentik-provision-fleet-user +description: "Provision or audit an Authentik user on Dankclaw (auth.franksdank.cloud): ak shell ORM drive, group set, behavioral login verify, stale-claim caveat." +--- + +# Provision an Authentik user on Dankclaw + +Source of truth: `serverAppConfigs-docs/Infrastructure/Authentik.md` ("User provisioning playbook"). Authentik objects are NOT repo-managed — no deploy, no file edits. + +1. Inspect first, mirror the newest human user's groups: +```bash +ssh dankclaw 'podman exec -i authentik-server ak shell' <<'EOF' 2>&1 | sed -n '/objects imported/,$p' +from authentik.core.models import User, Group +for u in User.objects.all().order_by("username"): + print(u.username, u.email, sorted(u.groups.values_list("name", flat=True))) +print(sorted(Group.objects.values_list("name", flat=True))) +EOF +``` +2. Create (idempotent guard; password goes in the heredoc = stdin, never argv, so it never lands in `ps`): +```python +from authentik.core.models import User, Group +u = User.objects.filter(username="X").first() +if not u: + u = User(username="X", email="X@franksdank.cloud", name="X", is_active=True) + u.set_password("...") + u.save() +for g in ("Non-Admin Users", "jellyfin_users", "openwebui_users"): # human-user precedent: ridley + u.groups.add(Group.objects.get(name=g)) +``` +3. Verify through the exact PasswordStage code path: +```python +from django.contrib.auth import authenticate +assert authenticate(request=None, username="X", password="...") +assert authenticate(request=None, username="X", password="nope") is None +``` + +## Rules +- No `User.create_user`; `is_superuser` is a group-derived property, not a constructor kwarg (raises `TypeError`). `path` defaults to `users`. +- Guarded-vhost access = any authenticated user (all proxy providers use implicit-consent); groups are consumed downstream: `homepage-admins` (Caddy `/admin` header gate), `openwebui_users`/`_admins` (roles claim), `jellyfin_users`/`_admins` (LDAP outpost). EXCLUDE admin groups from a plain user; ask before granting `homepage-admins`. +- Group add after a login → Caddy RBAC can 403 on the outpost's cached claims; purge that user's rows: `DELETE FROM authentik_providers_proxy_proxysession WHERE session_data->'claims'->>'preferred_username' = 'X';` (in the `authentik-postgres` container) — next visit re-authorizes silently. +- Jellyfin visibility rides the LDAP outpost cache; expect a lag. +- Look up flows by slug, not name. Do not rotate `AUTHENTIK_SECRET_KEY` to fix claim issues. diff --git a/agent/managed-skills/autoresearch-llm-bench-window/SKILL.md b/agent/managed-skills/autoresearch-llm-bench-window/SKILL.md new file mode 100644 index 0000000..518b224 --- /dev/null +++ b/agent/managed-skills/autoresearch-llm-bench-window/SKILL.md @@ -0,0 +1,76 @@ +--- +name: autoresearch-llm-bench-window +description: "How to run an iterative config-tuning window on a Dank-fleet LLM endpoint: harness pattern, noise-floor-first discipline, bracket mapping, and the frank-primary laws measured in the 2026-10-06 segment." +--- + +# Autoresearch bench window for a fleet LLM endpoint + +Use when running (or re-running) an iterative config-tuning segment against a +Dank-fleet LLM endpoint (frank-primary/frank-large/frank-small): build the +harness, establish the noise floor, then bracket-map one flag at a time. + +## Harness pattern (already built for frank-primary — reuse, don't rebuild) + +Committed at `autoresearch.sh` (dev box) + `scripts/autoresearch-frank-primary.sh` +(dankserver) in serverAppConfigs. The pattern generalizes to any fleet endpoint: + +1. Dev-box wrapper scp's the candidate env file + bench script to the target + host, then ssh-runs the bench. Never push the candidate config to origin + (the scp dirties the host checkout on purpose). +2. Host script, in order: stop the production unit (relinked units: `systemctl + stop`, NEVER mask — mask refuses a symlinked unit file) → boot a scratch + container (distinct name, e.g. `arbench`) on the SAME devices with the + candidate env — mirror the production podman contract exactly, differing + only in container name and a FRESH prefix-cache tier (wiped per run) so cold + measurements stay cold → poll the health endpoint until it answers (the + listener binds AFTER the full load, so a bare curl is honest; allow ~2-3 min + cold boot, 720s timeout) → run ONE self-contained deterministic request → + parse the engine's timings object into METRIC lines → tear down the scratch → + restore production. +3. The restore MUST be an EXIT trap (`restore() { podman rm -f scratch; + systemctl start ; }`) so a failed bench never leaves the endpoint + down. Verify after EVERY run: `systemctl is-active`, health curl until it + returns JSON (200 right after start is NOT enough — it can bind 60-90s + later), and no scratch leftover. +4. Bench request: fixed prompt built from a sorted repo-corpus glob with a + fixed random seed (deterministic → stable prompt_n), greedy sampling + (temperature 0) for a fixed decode length, sent locally (127.0.0.1) to + remove LAN jitter. Read prefill/decode/draft-acceptance from the engine's + own timings object — never roll a custom bench without asking. +5. Scratch containers must die by teardown; never leave a script whose tail + stops/removes a production unit (the stale-probe-tail lesson). + +## Discipline (the laws that make the numbers mean anything) + +1. **Noise floor FIRST.** Before iterating, replicate an identical config 2-3x + and treat that spread as the floor. Fleet endpoints here: decode ±3%, + prefill ±2%. A single-run "win" under the floor is NOT a win — this exact + mistake cost a false 33.06 keep in the 2026-10-06 segment. +2. **One flag change per iteration**; log every run honestly (keep/discard/ + crash); discards auto-revert the worktree. +3. **Bracket-map, don't climb.** Test both directions of every lever around + its deployed value, INCLUDING boot failures — crashes are envelope data + ("0.82/2048/wht6/40960 is the seqs=3-viable quartet" came from four crash + walls). Test downward too: 40960 MiB host pool is a floor, not slack. +4. **Repro before recording a mechanism.** An anomalous result next to an + abandoned bench may be contamination (boot race). Re-run clean once; only + then name a law. The MNBT 2560 collapse (2.6 t/s) was reproduced clean to + rule this out. +5. **Watch for truncation.** predicted_n < max_tokens means the generation + died mid-run — that config is pathological regardless of the rate. +6. **When the sweep ties, tie-break on real-traffic properties** (e.g. draft + acceptance for multi-turn workloads), never on probe artifacts, and prefer + the config that matches the documented runbook. + +## frank-primary specifics (2026-10-06 segment, commit c80c250) + +- MNBT=2048 is a RESONANCE POINT at MTP-on shapes, not an optimum: 2560 + collapses decode to 2.6-7.7 t/s (acceptance normal, verify stalls ~130ms/step, + truncated output), 1536 truncates at 57/256, 4096 won't boot at seqs=3. Do + not re-test. +- HOST_POOL_MIB 40960 is the load-bearing floor (36864 crashes boot). +- Spec depth 1-3 and seqs 2-3 all tie within noise; 0 and 4 are real + regressions (29.57 / 24.57 t/s vs 31.72 baseline). +- Levers known flat: GPU_HEADROOM, HOST_POOL upward, maxlen above 200k. +- Final kept config = Frank's original (spec=3, seqs=2, maxlen=200k, wht6, + 0.82, 2048, 40960, fp8 KV, TP=2), baseline decode 31.72 t/s at 43.6k depth. diff --git a/agent/managed-skills/bench-llama-server-params/SKILL.md b/agent/managed-skills/bench-llama-server-params/SKILL.md deleted file mode 100644 index 1ec996e..0000000 --- a/agent/managed-skills/bench-llama-server-params/SKILL.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -name: bench-llama-server-params -description: "Benchmark llama.cpp server-side parameters (--threads-batch, --ubatch-size, --spec-type) using HTTP API with max_tokens=1 for prefill, max_tokens=16384+ for decode" ---- - -# Benchmarking llama.cpp Server-Side Parameters - -## Problem -`llama-bench` benchmarks the bench binary itself, not the running server. -It cannot measure the effect of server-side parameters like `--threads-batch`, -`--ubatch-size`, `--spec-type`, or `--parallel`. - -## Solution: HTTP-Based Benchmarking - -### Prefill throughput (prompt processing) -1. Start the server with the target parameter value -2. Send a large prompt via the chat API with `max_tokens=1` -3. Measure `prompt_tokens / wall_time` from the response usage - -`max_tokens=1` is correct here — you want only prompt processing time, -not generation. The server crashes on `max_tokens=0`. - -### Decode throughput (generation) -1. Start the server with the target parameter value -2. Send a prompt that generates a long response -3. Use `max_tokens=16384` minimum (16k) — **never `max_tokens=1`** for decode - benchmarking. One token tells you nothing about sustained generation rate. -4. Measure `predicted_per_second` from the response `timings` object, or - compute `completion_tokens / wall_time` - -### Why at least 16k tokens for decode -- Decode throughput varies over the generation (warmup, cache growth, MoE - routing). A single token is noise. -- Speculative decoding acceptance rates stabilize over thousands of tokens. -- Short generations finish before the draft head reaches steady state. -- 16k tokens gives a stable average at ~22 tok/s (≈12 min per run). - -### Key Pattern -```python -# Decode benchmark — measure generation throughput -body = json.dumps({ - "model": "", - "messages": [{"role": "user", "content": ""}], - "max_tokens": 16384, # 16k minimum for decode; use max_tokens=1 ONLY for prefill - "temperature": 0, # deterministic for reproducibility - "stream": False -}) -# Measure wall time from request send to response receipt -# Get predicted_per_second from timings, or completion_tokens / wall_time -``` - -### When to Use -- `--threads-batch` (prefill): use `max_tokens=1` -- `--ubatch-size` (decode): use `max_tokens=16384+` -- `--spec-type` / speculative decoding: use `max_tokens=16384+` (decode only) -- Any server-side parameter that `llama-bench` cannot target - -### Benchmarking policy (MANDATORY) -1. **STOP the production service** before running any benchmark -2. **Mask** the service (`systemctl mask`, never `disable` — on these hosts - `disable` deletes the unit file from `/etc/systemd/system/`) -3. Run the bench -4. **Unmask + restart** the service when done -5. **Verify health** before reporting results - -### Example Scripts -- `scripts/bench-threads-batch-http.py` — prefill sweep (max_tokens=1) -- `scripts/bench-decode-threads-batch.py` — decode sweep (max_tokens=256+) -- `scripts/bench-spec-decode.py` — speculative decoding config sweep diff --git a/agent/managed-skills/closed-engine-runtime-investigation/SKILL.md b/agent/managed-skills/closed-engine-runtime-investigation/SKILL.md new file mode 100644 index 0000000..419a8ee --- /dev/null +++ b/agent/managed-skills/closed-engine-runtime-investigation/SKILL.md @@ -0,0 +1,29 @@ +--- +name: closed-engine-runtime-investigation +description: "Recipe for diagnosing/tuning a closed-source LLM runtime (halogen-class) on this fleet: docs, env-lever enumeration from the binary, ledger scraping, cache-assisted A/B under the bench law." +--- + +# Investigating a closed-source LLM runtime (halogen-class) + +Use when a containerized closed-source engine (e.g. halogen on frank-large) underperforms and the repo docs don't explain it. Fleet law: stop the production unit BEFORE any GPU experiment (`sudo systemctl stop `; mask FAILS on relinked units — stop alone, redeploy after). + +## 1. Ground truth before flags +- Boot ledger + per-request timings live in `podman logs ` (container-lifetime only) UNLESS the serve script captures stdout on disk (foreground `podman run`). A `podman run -d ... > LOG` redirect captures ONLY the container ID — a one-line hex file is the tell. +- `serve_api:`-style log lines carry per-request `prompt/prefill/rounds/commit/detok` — build the depth-vs-rate curve from REAL traffic before any bench. Shallow-healthy + deep-collapsed = a depth-linear term (KV reads), not a broken engine. +- If the engine's public repo exists, read its FLAGS/docs (e.g. `docs/FLAGS.md`) AND the bench tool source (e.g. `tools/halogen-bench.py`) — bench tools reveal which regimes the published numbers never measured (halogen's tg rows are shallow-only). + +## 2. Enumerate the undocumented levers +The binary self-announces every env var it reads. The image often lacks `strings`: +``` +podman run --rm --entrypoint /bin/sh -c \ + 'grep -aoE "_[A-Z0-9_]+" /usr/local/bin/ | sort -u' +``` +Classify by kernel family (attention vs linear-state vs GEMV) against the suspected term. Only test levers with public precedent (e.g. issue #12 proved HALOGEN_FA_TQ) or an obvious name match; defaults are the publisher's measured winners — an A/B must show >=~15% to adopt. + +## 3. Cache-assisted A/B (cheap reps) +Boot each probe arm as its own container (env is read at startup) with the prompt cache ON (explicit MB >= one full-entry at ~70 KB/token): rep 1 pays the cold prefill, rep 2+ resumes (~2 s) — depth measurements then cost decode-time only. Publish the probe on host LOOPBACK (127.0.0.1:), run the driver ON the host (python3 stdlib is enough), read the engine's own `timings` (prompt_ms/cache_n/predicted_ms/draft_n_accepted). Never benchmark beside a live engine on the same GPU. + +## 4. Adopt or revert +- Winner: env line into the serve env + test-seam update + docs, then `deploy.sh --service ` (restart restores everything; no unmask needed after stop-only). +- Loser: `podman rm -f` probes, start the unit, revert nothing. +- Either way: record the measured before/after in the runbook; report the depth behavior upstream if the publisher invites benchmark data — their bench gaps (depth, carve-topologies) are exactly what issues like #12 close. diff --git a/agent/managed-skills/dankclaw-homepage-asset-verify/SKILL.md b/agent/managed-skills/dankclaw-homepage-asset-verify/SKILL.md new file mode 100644 index 0000000..b56cad0 --- /dev/null +++ b/agent/managed-skills/dankclaw-homepage-asset-verify/SKILL.md @@ -0,0 +1,13 @@ +--- +name: dankclaw-homepage-asset-verify +description: "Add/verify homepage assets on Dankclaw: Caddy @data allowlist, deploy copy step, e2e panel census, live content-type check." +--- + +Every new top-level homepage asset directory (e.g. `shared/homepage/img/`, `shared/homepage/sfx/`) needs FOUR coordinated changes, all in one commit: + +1. **Caddy allowlist** — `dankclaw/Caddyfile` `@data path` matcher must include `//*`. Without it the post-auth `@home` catch-all answers with `index.html` (`200 text/html` for a PNG/OGG — silent breakage). +2. **Deploy copy** — `scripts/deploy-remote.sh` `deploy_homepage()` copies `shared/homepage//` with the `cp -r ... /srv/homepage/` merge pattern beside the fonts/img/sfx blocks. +3. **Test pins** — `tests/homepage_caddy.bats` grep-pins the new `@data` path; if the asset changes panel rendering, the e2e census in `tests/e2e/specs/10-homepage.spec.ts` must count the new render style (it derives expected names from services.json: `.panel .name` for normal cards, `.mi-name` for fused Model Information cards — a new fused component breaks the panel census otherwise, which is exactly how the 2026-10-08 Protoss deploy failed its e2e gate). +4. **Deploy both services** — `deploy.sh dankclaw --service homepage` and `--service caddy` (Caddyfile change), then live-verify from a browser tab: `fetch('//')` must return `200` with the correct `content-type` (`image/jpeg`, `audio/ogg`), not `text/html`. + +Checker note: `status.json` names follow `services.json` automatically (the deployed checker re-reads it via the homepage deploy's restart of homepage-status.timer), so renaming cards needs no checker change. diff --git a/agent/managed-skills/defuse-shell-rc-secrets/SKILL.md b/agent/managed-skills/defuse-shell-rc-secrets/SKILL.md new file mode 100644 index 0000000..c7609fe --- /dev/null +++ b/agent/managed-skills/defuse-shell-rc-secrets/SKILL.md @@ -0,0 +1,21 @@ +--- +name: defuse-shell-rc-secrets +description: "Find literal API keys/tokens in shell rc files and replace them with 600-perm secret files + loader lines, syncing the canonical tool token file." +--- + +# Defuse shell-rc secrets + +Symptom: `export KEY=...` literal in `.zshrc`/`.bashrc` — world-readable in every process env, one `git add -A` from a dotfiles repo. + +## Procedure (repeat per host) +1. Sweep: `grep -n "export.*KEY\|export.*TOKEN" ~/.zshrc ~/.zprofile ~/.profile ~/.bashrc 2>/dev/null` (run per host; when batching hosts in one ssh compound command, label which output came from which host — misattribution wastes turns). +2. Extract WITHOUT echoing: `mkdir -p ~/.config/secrets && chmod 700` it, then `sed -n 's/^export NAME="\(.*\)"$/\1/p' ~/.zshrc > ~/.config/secrets/` (quote variant and unquoted variant are different sed patterns — verify byte counts with `wc -c`, a 0-byte file means the pattern missed). +3. `chmod 600` every secret file; sync any canonical tool file the ecosystem expects (e.g. `~/.cache/huggingface/token`). +4. Replace the rc literal (hashline `PUT N.=N:` range — never echo the secret in the op): `[ -f ~/.config/secrets/ ] && export NAME="$(cat ~/.config/secrets/)"`. +5. Verify: `zsh -n ~/.zshrc` (or `bash -n`); keep a dated `.bak` of the rc on remote hosts; shred any /tmp transfer copies immediately (`shred -u`). + +## Laws +- A token pasted into chat lands in transcripts and model context — treat chat as a leak channel; give the user the `read -rs` snippet instead. +- scp of a secret FILE (encrypted channel, no argv) is fine; the token is never a command-line argument. +- After replacing a legacy HF token, probe scope non-destructively: `HfApi().whoami()` (legacy read tokens show `type: access_token` without canWrite; the decisive probe is `create_repo(exist_ok=True)` on the real target repo). +- Verify extraction byte-count before deleting/trusting; a successful-looking sed can write an empty file. diff --git a/agent/managed-skills/detached-fleet-job-launch/SKILL.md b/agent/managed-skills/detached-fleet-job-launch/SKILL.md new file mode 100644 index 0000000..9d07dce --- /dev/null +++ b/agent/managed-skills/detached-fleet-job-launch/SKILL.md @@ -0,0 +1,46 @@ +--- +name: detached-fleet-job-launch +description: Launch a long-running remote job on Dank fleet hosts so it survives ssh/logind teardown +--- + +# Detached long-running jobs on Dank fleet hosts + +## Law +`ssh host 'setsid nohup CMD & '` is NOT detached. systemd-logind deactivates +`session-N.scope` when the ssh command returns and SIGKILLs the whole cgroup: +no journal entry, no OOM trace, python block-buffers stdout so the log stays +0 bytes. setsid/nohup do not escape the cgroup. + +## Canonical launch +``` +ssh host 'export XDG_RUNTIME_DIR=/run/user/$(id -u); \ + systemd-run --user --collect --unit=NAME \ + -p SendSIGKILL=no \ + -p StandardOutput=append:/home/dank/logs/NAME.log \ + -p StandardError=append:/home/dank/logs/NAME.log \ + sh -c "CMD -u; echo \$? > /home/dank/logs/NAME.rc"' +``` +- transient user unit = outside every session scope; survives ssh teardown + (requires `Linger=yes`, true for `dank` on both hosts) +- `echo $? > …rc` inside the unit = completion sentinel the unit itself writes +- `-u` / unbuffered output so the log is live +- verify launch in the same ssh call: `systemctl --user is-active NAME.service` + +## Watcher (one async job, never poll) +``` +ssh host 'while [ ! -f /home/dank/logs/NAME.rc ]; do sleep 60; done; \ + echo RC=$(cat /home/dank/logs/NAME.rc); tail -25 /home/dank/logs/NAME.log' +``` +Run it async with timeout 0; the result is injected on completion. + +## Forensics when a detached job died +1. `ps -p PID` — gone. +2. Log 0 bytes → buffered stdout lost at SIGKILL. +3. `journalctl --since … | grep -E "session-[0-9]+.*(Deactivated|Removed session)"` — + timestamps matching the death window confirm the scope kill (vs OOM, which + leaves kernel lines). + +## Notes +- zsh remote shells: `echo ===X===` fails (`=cmd` expansion); quote sentinels. +- The HF upload of 2026-10-08 (113.7 GiB, hf_xet) died this way ~2 min in; + relaunch as above succeeded. diff --git a/agent/managed-skills/faillock-sudo-lockout-recovery/SKILL.md b/agent/managed-skills/faillock-sudo-lockout-recovery/SKILL.md new file mode 100644 index 0000000..a047460 --- /dev/null +++ b/agent/managed-skills/faillock-sudo-lockout-recovery/SKILL.md @@ -0,0 +1,27 @@ +--- +name: faillock-sudo-lockout-recovery +description: "Diagnose and recover from a pam_faillock sudo lockout on Frank's Arch workstations, typically caused by an agent's non-interactive sudo retries" +--- + +# Faillock Sudo Lockout Recovery + +## Symptom +User reports "sudo password rejected / changed"; journal shows +`pam_faillock(sudo:auth): User is temporarily locked out due to 3 consecutive failed login attempts`. + +## Diagnose (read-only, one round) +1. `stat -c '%n mtime=%y ctime=%z' /etc/shadow` — unchanged mtime = password was NEVER changed. +2. `journalctl --since '' | grep 'sudo\['` — lines with `PWD=` / `COMMAND=` name the spammer (agent harness loops show `PWD=/home/.../.omp`, `COMMAND=...yay/pacman...`, no `TTY=`). +3. `faillock --user ` — shows the valid failure entries. + +## Root cause law +In the omp harness sudo has NO TTY: each `sudo ` = 3 pam_unix failures. Arch pam_faillock defaults deny=3 with ~600s unlock_time, and each retry re-arms it — the user's real interactive sudo is locked while the loop runs. + +## Recover +1. Kill the loop: `ps -eo pid,ppid,etime,cmd | grep -E 'sudo|yay|makepkg'` → kill offending PIDs; stop/kill the responsible agent session. +2. `faillock --user "$USER" --reset` — works unprivileged for your own user (dir /run/faillock/ is 0700 owner). +3. User verifies with `sudo -v` in a real terminal. + +## Prevent +- Agents NEVER run sudo/yay/pacman/pkexec; no SUDO_ASKPASS/expect workarounds. +- Root-requiring steps are handed to the user as exact copy-paste commands in the final report. diff --git a/agent/managed-skills/fandom-wiki-media-fetch/SKILL.md b/agent/managed-skills/fandom-wiki-media-fetch/SKILL.md new file mode 100644 index 0000000..41287b0 --- /dev/null +++ b/agent/managed-skills/fandom-wiki-media-fetch/SKILL.md @@ -0,0 +1,19 @@ +--- +name: fandom-wiki-media-fetch +description: Fetch audio/image media from a Fandom wiki via its MediaWiki API when the wiki HTML pages 403 curl or hang headless browsers. +--- + +Goal: get direct media URLs (and files) from a starcraft.fandom.com-style Fandom wiki. + +Why this route: Fandom HTML pages return a ~5 KB JS shell to `curl` (403 via `read ...:raw`) and time out headless Chromium (`browser.open`/`goto` >30 s on quotation megapages). The MediaWiki API is plain JSON and answers plain curl. + +Steps: +1. Find the content page's wikitext: + `curl -sL -A "Mozilla/5.0" "https://.fandom.com/api.php?action=parse&page=&prop=wikitext&format=json&formatversion=2"` + Top-level pages are often tabber/transclusion shells — follow the `{{:Subpage}}` / `{{TemplateName}}` references with the same call (e.g. `StarCraft_unit_quotations` → `/Protoss` → `Template:QuotesSC1Zealot`). +2. Extract media names from the wikitext. StarCraft quotes use `{{sm2|File.ogg|displayed quote}}` inside `{{UnitQuoteBox|select=...}}` fields — match the quote text to pick the exact file. +3. Resolve direct CDN URLs (works for images too): + `curl -sL -A "Mozilla/5.0" ".../api.php?action=query&prop=imageinfo&iiprop=url|size&format=json&formatversion=2&titles=File:A.ogg|File:B.jpg"` + → `query.pages[].imageinfo[0].url` = `https://static.wikia.nocookie.net/...` — downloadable with plain `curl -sfL`. +4. To discover art on a unit page: `action=query&prop=images&imlimit=100&titles=`, filter titles (icon/art/portrait/Head/Render), then resolve the keepers via imageinfo. Note splash/head art naming: `File:Zealot SC2 Art1.jpg`, `File:Archon SC2 Head1.jpg`; some obvious guesses (SC1-era units' SC2 splash) do not exist — verify with imageinfo, absent pages return `missing` with no `imageinfo`. +5. Verify downloads (`file sfx/*.ogg` → `Ogg data, Vorbis audio`) — a 403 body saves as a success with `-o` and no `-f`. diff --git a/agent/managed-skills/frank-primary-decode-tuning/SKILL.md b/agent/managed-skills/frank-primary-decode-tuning/SKILL.md new file mode 100644 index 0000000..2108d4a --- /dev/null +++ b/agent/managed-skills/frank-primary-decode-tuning/SKILL.md @@ -0,0 +1,22 @@ +--- +name: frank-primary-decode-tuning +description: "Tune frank-primary (radiance TP=2) decode throughput: benchmark harness usage, flag levers, known optima and boot-failure walls." +--- + +# frank-primary decode tuning + +Harness: `bash autoresearch.sh` (dev box) scp's `scripts/frank-primary.env` + `scripts/autoresearch-frank-primary.sh` to dankserver, stops prod (`systemctl stop frank-primary`, NEVER mask — relinked unit), boots scratch container `arbench` on :7770 (fresh prefix-cache tier at `$STATE_DIR/kvcache/arbench` for cold prefill), polls health 720s, sends ONE greedy completion (fixed 140k-char repo-corpus prompt -> 43,627 tokens, 256-token decode), parses `timings` into METRIC lines, tears down, restores prod via EXIT trap. ~4 min/run; decode noise ~0.3%. + +Metric: `METRIC decode_tps` = `timings.predicted_per_second` at 43.6k depth. + +## Optima (measured 2026-10-06, keep these) +- `FRANK_PRIMARY_SPEC_TOKENS=2` (unimodal: 1->31.89, 2->32.40, 3->31.72, 4->24.57 t/s; acceptance collapses past 3) +- `FRANK_PRIMARY_MAX_SEQS=3` with `MAX_MODEL_LEN=200000` (600k KV; +4.2% total; 4 slots regresses) +- `FRANK_PRIMARY_EXPERT_VS_CACHE=0.82` bracket-confirmed (0.78 fails boot, 0.84 -8.9%, 0.87 collapse) +- `TP_WIRE=wht6`, `MAX_BATCHED_TOKENS=2048` — REQUIRED at seqs=3: bf16 wire and MNBT 4096 both fail boot (VRAM) + +## Non-levers (decode-neutral, don't spend runs) +GPU_HEADROOM_MIB (48 = neutral), HOST_POOL_MIB 49152 (neutral), MAX_MODEL_LEN any change (regresses: bigger KV steals resident weights), prefix-cache tiers (prefill-side). + +## Rules +One flag per iteration; keep only > +1% decode over best; a boot failure is `crash` (trap restores prod). Constraint floor: MAX_SEQS*MAX_MODEL_LEN >= 400000. diff --git a/agent/managed-skills/git-repo-secret-audit/SKILL.md b/agent/managed-skills/git-repo-secret-audit/SKILL.md new file mode 100644 index 0000000..9f07df2 --- /dev/null +++ b/agent/managed-skills/git-repo-secret-audit/SKILL.md @@ -0,0 +1,68 @@ +--- +name: git-repo-secret-audit +description: "Audit a git repo for leaked secrets across history, unreachable objects, LFS payloads, git config and host configs — with triage of every hit." +--- + +# Git repo secret audit (history + unreachable objects, not just the worktree) + +Worktree greps prove nothing: the leak you care about is in a commit that was amended +away, in a branch nobody checks out, in an LFS payload, or in `.git/config`. + +## 0. Build the known-value set from the live secrets, never from literals +Read the values from the secret store at runtime so no key enters the transcript: +```python +known = {} +for p in (home/".config/secrets").iterdir(): + t = p.read_text().strip() + vals = {t} + m = re.search(r":([^:@/]+)@", t) # credential-store URL form + if m: vals.add(m.group(1)) + known[p.name] = {v for v in vals if len(v) >= 12} +``` +Report ONLY `value[:4] + len + sha256[:10]`. Never print a value, and never quote a +secret-shaped literal in a doc/commit message — the pre-commit gate will (correctly) block it. + +## 1. Shapes to match (superset of the Dank gate) +`sk-[A-Za-z0-9]{18,}`, `sk-or-v1-`, `sk-ant-`, `hf_[A-Za-z0-9]{30,}`, `gh[pousr]_[A-Za-z0-9]{30,}`, +`github_pat_`, `glpat-`, `xox[baprs]-`, `(AKIA|ASIA)[0-9A-Z]{16}`, `npm_`, `pypi-`, `AIza[0-9A-Za-z_-]{30,}`, +`(sk|pk)_(live|test)_`, Telegram `\d{8,10}:AA`, JWT `eyJ…\.eyJ…`, `-----BEGIN [A-Z ]*PRIVATE KEY-----`, +`AGE-SECRET-KEY-1`, `ssh-(ed25519|rsa)\s+AAAA`, `scheme://user:pass@host` basic-auth URLs, +`Bearer <24+>`, and the noisy `(password|secret|api_key|token)\s*[:=]\s*<12+>` assignment shape. + +## 2. Scope, in this order +```bash +git rev-list --all | wc -l # revs incl. every branch + tag +git ls-tree -r --format='%(objectname) %(path)' # sha -> path, per rev +git cat-file --batch # stream reachable blob content +git cat-file --batch-all-objects --batch-check # EVERY object, incl. dangling +git rev-list --objects --all # the reachable set; difference = dangling +git log --all --oneline --find-object= # empty => never committed on any ref +git status --porcelain -uall --ignored=matching # !! lines: on-disk ignored files +git config --list --show-origin # local/global/system credentials +git ls-files | grep -E '(auth\.json|trust\.json|\.env|\.pem|\.key|identity)' # quarantine +``` +LFS: scan `.git/lfs/objects/**` as BYTES (`decode("latin-1")`) — a screenshot is a classic +secret carrier and the tracked tree holds only pointers. + +## 3. Entropy layer (catches issuer-specific keys you did not list) +Tokens `[A-Za-z0-9+/_-]{32,}` with Shannon entropy >= 4.3 containing both digits and letters. +Flag every one whose value is in `known` as a LIVE leak; list the rest for triage. + +## 4. Triage — the shapes that always fire and are fine +- `api_key="sk-no-key-required"` and other doc placeholders +- prose: `Set password: user.set_password(...)`, `authenticate(user, password)` +- 40-hex hits that are git object ids / packfile names (`pack-<40hex>.pack`) +- memory/KB records that *describe* a secret operation +A hit is a leak only when the value matches a live secret, decodes to a real credential, or +sits in a field a program reads. + +## 5. Adjacent host scan (the audit's real yield) +Check file modes of the secret dir (`0600`, dir `0700`) — world-readable secret files are the +most common real finding. Then grep the live values across shell rc files, `~/.gitconfig`, +`~/.npmrc`, `~/.netrc`, `~/.curlrc`, `~/.docker/config.json`, `~/.cache/huggingface/token`, +`~/.aws/credentials`, `~/.ssh/config` and report filenames only. + +## 6. Report +Per layer: scope numbers + verdict. Every hit: path:line, redacted value, verdict +(leak / benign + why). Name the gate's coverage gap if the project's own hook misses shapes +you matched. Fix modes/chmod on the spot; changes to a security hook are the operator's call. diff --git a/agent/managed-skills/gitea-token-out-of-remote-url/SKILL.md b/agent/managed-skills/gitea-token-out-of-remote-url/SKILL.md new file mode 100644 index 0000000..5979b09 --- /dev/null +++ b/agent/managed-skills/gitea-token-out-of-remote-url/SKILL.md @@ -0,0 +1,52 @@ +--- +name: gitea-token-out-of-remote-url +description: "Move a Gitea/HTTP git token out of remote.origin.url into a credential store file, and verify auth works with prompts disabled." +--- + +# Move a git token out of the remote URL + +A remote URL like `http://user:@host:port/org/repo.git` puts the credential in +`.git/config` (copied by every clone), prints it in `git remote -v`, and git-lfs echoes the +whole URL in its `Locking support detected on remote` warning on every push. + +## 1. Capture the token from the existing remote (never paste it into a message) +```bash +url=$(git remote get-url origin) +user=$(printf '%s' "$url" | sed -E 's#^http://([^:]+):[^@]+@.*$#\1#') +tok=$(printf '%s' "$url" | sed -E 's#^http://[^:]+:([^@]+)@.*$#\1#') +``` + +## 2. Write the credential file (URL form, port included) +```bash +umask 077 +printf 'http://%s:%s@HOST:PORT\n' "$user" "$tok" > ~/.config/secrets/gitea +``` +Git 2.56 `credential-store` does NOT parse the `protocol=`/`host=`/`password=` block shape +that `git credential fill` accepts on stdin: a block-format file gives a SILENT empty get, +then git falls through to askpass (`could not read Username`). One URL per line, port included. + +## 3. Point git at it, strip the URL, silence the LFS echo +```bash +git config --global credential.helper 'store --file ~/.config/secrets/gitea' +git remote set-url origin http://HOST:PORT/org/repo.git +git config lfs.http://HOST:PORT/org/repo.git/info/lfs.locksverify true +``` + +## 4. Verify with prompts disabled (a hang is a failed credential) +```bash +printf 'protocol=http\nhost=HOST:PORT\n\n' | git credential-store --file ~/.config/secrets/gitea get +GIT_TERMINAL_PROMPT=0 git ls-remote --heads origin main # read auth +GIT_TERMINAL_PROMPT=0 git push --dry-run origin main # write auth +git remote get-url origin # must contain no '@' +grep -l "$tok" ~/.gitconfig */.git/config ~/.zshrc # must find nothing +``` +A successful push over the bare URL is the end-to-end proof: the credential came from the file. + +## 5. Sweep the leak +Same shape on every clone on the host — loop over `find ~ -maxdepth 4 -path '*/.git/config'`, +read `remote.origin.url`, scrub any URL containing `@`. One credential file serves them all +via the global helper. + +## Rotate +Rewrite only the credential file, then delete the old token in the Gitea UI +(Settings → Applications). No remote URL, rc file, or repository changes. diff --git a/agent/managed-skills/hf-checkpoint-profile/SKILL.md b/agent/managed-skills/hf-checkpoint-profile/SKILL.md new file mode 100644 index 0000000..170cb20 --- /dev/null +++ b/agent/managed-skills/hf-checkpoint-profile/SKILL.md @@ -0,0 +1,26 @@ +--- +name: hf-checkpoint-profile +description: "Profile a Hugging Face safetensors checkpoint (sizes, dtypes, tensor-name space, gated/license) with compact HTTP-range reads — safe for subagents on small context slots." +--- + +# Profile an HF safetensors checkpoint cheaply + +Goal: inventory a repo (files/bytes, dtype histogram, tensor-name patterns, per-group GiB, config keys, license/access) without pulling gigabytes and without overflowing a subagent's context slot. + +## Endpoints (all public, no token needed even for gated models) +- `https://huggingface.co/api/models/?blobs=true` → `siblings` (name+size), `gated`, `private`, `downloads`, `safetensors` metadata, `tags`. +- `https://huggingface.co/api/models/` for tags/base model; `https://huggingface.co/api/models?author=&limit=100` to find sibling repos (canonical id check: cards often link both `X-1.5-Y` and `X-Y`). +- `https://huggingface.co/api/models//resolve/main/` for `config.json`, `model.safetensors.index.json`, `chat_template.jinja`, `LICENSE`, `generation_config.json`; `/raw/main/` for text. + +## dtype + per-group bytes without downloading shards +Parse the safetensors header over HTTP Range: first 8 bytes = little-endian u64 header length, then that many bytes of JSON (`{name: {dtype, shape, data_offsets}}`). Fetch every shard header with a thread pool (131 shards ≈ seconds), then bytes = prod(shape) × dtype size. Collapse names with `re.sub(r"\.\d+\.", ".N.", k)` (and `\.shard_\d+` → `.shard_N`) to get the pattern space; group-sum by prefix to see where the mass sits (experts / ngram-PLE / MTP / vision / trunk). + +Compare the key set against the base model's index to prove "full-weight finetune, identical names" (a zero-diff key set means per-tensor recipes written for the base apply verbatim, and it reveals fused vs per-expert layout, PLE/ngram shards, MTP head, vision tower, absence of `quantization_config`). + +## Guardrails +- Run python scripts from a clean directory: a stray `/tmp/inspect.py` shadows stdlib `inspect` and kills `concurrent.futures` imports. +- Never put an HF token on a command line (`/proc//cmdline` is world-readable); use `curl -K `. +- `curl -f` / `--fail-with-body` and verify byte counts — a 401/403 writes the error body as the "artifact" and exits 0. + +## Subagent context budget +Subagents on a 131k-token slot die from `exceed_context_size_error` when told to read whole doc pages. Give grep-only instructions (`| head -40`, `-o`, `-c`), demand a line cap ("report in ≤250 lines, no verbatim dumps"), and prefer `git clone --depth 5` + local grep to web-page reads. A peer scout that overflows returns exit 1 with the 400 in the harness log and NO structured output — assume nothing was learned and redo the fact inline. diff --git a/agent/managed-skills/hf-large-artifact-publication/SKILL.md b/agent/managed-skills/hf-large-artifact-publication/SKILL.md new file mode 100644 index 0000000..953ca7b --- /dev/null +++ b/agent/managed-skills/hf-large-artifact-publication/SKILL.md @@ -0,0 +1,40 @@ +--- +name: hf-large-artifact-publication +description: "Publish a large quantized artifact + model card (and an optional anonymized corpus dataset) to HF Hub from dankserver: staging set, HF-style card spine, upload gates, transient-unit launcher, sentinel watching, card-metadata and scrub laws." +--- + +# HF large-artifact publication (dankserver) + +Publish a big model artifact (100+ GiB) + card, and optionally an anonymized corpus dataset, from dankserver under `frank8465`. + +## Staging +- Dev box `~/Projects/hf-swift15-upload/` (author card + scripts here), `scp -r` synced to `/home/dank/hf-swift15-upload/` on dankserver. Keep staging + logs + `.rc` sentinels after publication as receipts. +- Payload set: artifact, `README.md` card, `scripts/` (recipe/drivers/probes), `artifacts/` (gate JSONs, per-position `.rows`, manifests, run logs). +- Secret-scan the whole payload before arming: credential regexes (hf_/ghp_/pat, private keys, Bearer values, AKIA/sk-), LAN IPs, hostnames, domains, emails. Verdict each hit — variable *references* and upstream test fixtures are not leaks; check real values in context. + +## Card law (HF validator) +- Custom licenses MUST be `license: other` + `license_name:` + `license_link:` (mirror the base model's card trio — fetch `https://huggingface.co/api/models/` and read `cardData`). A raw non-SPDX `license:` value raises `ValueError: Invalid metadata in README.md` inside `upload_file` — AFTER all local gates have run. +- Card spine: what-the-format-is + engine pin digest, recipe table, quality gates (KL/top-1/PPL + per-position data shipped), proof hardware, exact `podman run` line, license chain, manifest table with the artifact's sha256 receipt, "In plain English:" block per section. +- Quote the SHIPPED filename (build name), not a prettified one — a hub-side rename of a 100 GiB LFS file costs a full re-upload. + +## Upload mechanics +- venv `/home/dank/venvs/radcorpus` (huggingface_hub 1.28 + hf_xet); `HF_XET_HIGH_PERFORMANCE=1`; token from `~/.cache/huggingface/token` (see defuse-shell-rc-secrets pattern: `~/.config/secrets/` 600 + rc loader, mirrored to the cache file). NEVER token in argv/chat. +- Script gates, in order: sha256 of live artifact vs recorded digest (`gate 1 : MATCH`), `create_repo(..., private=False, exist_ok=True)`, small files first, artifact last, post-upload census — **compare remote sizes to local and FAIL the RC on any mismatch** (this caught a stale card mid-campaign). +- hub 1.28 API traps: `upload_file`/`create_commit` kwargs are keyword-only; no `rename_file`, `hf_hub_url`, `HfApi.commit_api` (use `CommitOperationAdd`/`CommitOperationDelete` from `huggingface_hub` top-level if needed); `whoami` hides `canWrite` for legacy tokens — probe write scope by `create_repo(exist_ok=True)` on the real target. +- zsh-over-ssh: never `echo ===` (triggers `=cmd` expansion); quote it. + +## Launcher + watchers +`setsid nohup` over ssh dies at session teardown. Use a transient user unit: +``` +systemd-run --user --collect --unit=hf-upload -p SendSIGKILL=no \ + -p StandardOutput=append:/home/dank/logs/hf-upload.log \ + -p StandardError=append:/home/dank/logs/hf-upload.log \ + sh -c "cd && /bin/python -u .py --go; echo \$? > /home/dank/logs/hf-upload.rc" +``` +Sentinel: background ssh loop polling the `.rc` file (60 s period, bounded), tails the log on appearance. Progress bars are `\r`-fed — `tr '\r' '\n' | tail` to read them. + +## Corpus dataset variant +- License audit first: read `urls.txt`/manifest, fetch each source card (`license` in cardData). Share-alike sources (CC BY-SA) → publish derived corpus CC BY-SA with attribution; model weights unaffected. +- Fleet-harvested slices = agent-session transcripts: scrub with deterministic substring-class rules (NOT word-boundary: catches `dankclaw_default`, `FRANKBOOK_SVCS`, `franksdank/alloy` image namespace): LAN IPs→10.x, hostnames→server-a/b/client-c, domain→example.com (catch `\.`-escaped forms + bare brand), `/home/`, emails→user@example.com. Residual probes run on OUTPUT; whitelist self-replacements + audited fixture noise. +- Publish original hashes, disclose non-byte-identity, cross-link model card ↔ dataset card, verify via `raw/main/README.md` fetches. +- Dataset repo needs dataset-write scope (fine-grained token; access tokens 403). `repo_type="dataset"` everywhere; 45 MB rides in ~60 s. diff --git a/agent/managed-skills/homepage-smoke-verify-loop/SKILL.md b/agent/managed-skills/homepage-smoke-verify-loop/SKILL.md new file mode 100644 index 0000000..a02b0cf --- /dev/null +++ b/agent/managed-skills/homepage-smoke-verify-loop/SKILL.md @@ -0,0 +1,21 @@ +--- +name: homepage-smoke-verify-loop +description: "Smoke-verify Dankclaw homepage changes locally before commit/deploy: isolated static tree + fake status + browser census, then deploy gate." +--- + +# Homepage smoke-verify loop (serverAppConfig) + +Verify `shared/homepage/` changes against a real browser BEFORE commit/deploy. + +## Procedure +1. `node --check shared/homepage/app.js` and `python3 -c "import json; json.load(open('shared/homepage/services.json'))"`. +2. Isolated tree: `rm -rf /tmp/hp-smokeN && cp -r shared/homepage /tmp/hp-smokeN`. +3. Fabricate runtime files in the tree: + - `status.json`: `{"checked":"x","services":[{"name":,"state":"up"}]}` (set one model `down` to exercise the red-pulse path). + - `deployed.js`: `window.DEPLOYED_AT="smoke"`. +4. Serve: `(cd /tmp/hp-smokeN && nohup python3 -m http.server 8099 >/dev/null 2>&1 & echo $! > /tmp/.hp-pid)`. + - If reusing port 8099: `pkill -f "http.server 8099"` FIRST — a second server on a busy port fails silently and the OLD tree keeps serving. +5. Browser census via eval tab on `http://127.0.0.1:8099/?v=` — ALWAYS cache-bust with `?v=`; python http.server gets heuristically disk-cached by the browser. Assert: broken images (`.app-art,.mi-art` naturalWidth==0), computed styles (`getComputedStyle` for palette/fonts/clipPath), presence of new classes, `document.querySelectorAll('.docs section').map(s=>s.id)` for doc order, regex text census for banned words. +6. Screenshot full page + `selector` shots; inspect visually. +7. `./tests/run.sh` (expect only the known frank-primary MAX_SEQS failure), commit, `./scripts/deploy.sh dankclaw --service homepage` (runs Playwright e2e gate), then repeat the census against `https://www.franksdank.cloud/?v=`. +8. Cleanup: kill server, move `/tmp/hp-smokeN` to `~/.AGENTTRASH/YYYYMMDD-homepage-smoke/` (never `rm`). diff --git a/agent/managed-skills/homepage-theme-smoke/SKILL.md b/agent/managed-skills/homepage-theme-smoke/SKILL.md new file mode 100644 index 0000000..e537555 --- /dev/null +++ b/agent/managed-skills/homepage-theme-smoke/SKILL.md @@ -0,0 +1,15 @@ +--- +name: homepage-theme-smoke +description: "Locally smoke-test the Dankclaw homepage theme (style.css/app.js/docs.html) in the managed headless browser, with the stale-server traps that fake the result." +--- + +Goal: prove a homepage theme change on a throwaway tree before commit/deploy. + +Protocol: +1. Stage a fresh copy: `cp -r shared/homepage/. /tmp/hp-smoke-/`, drop `.AGENTTRASH` dirs, and write a fake `status.json` (`{"generated_at":"x","services":[{"name":...,"state":"up"}]}`) — app.js fetches services.json + status.json and renders NOTHING without both. +2. Serve: `cd && nohup python3 -m http.server 8099 --bind 127.0.0.1 &`. +3. STALE-LISTENER TRAP (fakes every census): a survivor server from an older wave keeps answering on 8099 — `ss -ltn | grep 8099` proves a listener exists, not WHICH tree it serves. After every (re)start verify identity: `for p in $(pgrep -f "http.server 8099"); do readlink /proc/$p/cwd; done` must print the NEW tree; then `curl -s localhost:8099/ | grep -c `. `kill` the stale pid — never trust a `||`-guarded restart. Also: `pkill -f "http.server 8099"` from a shell whose own command line contains that string kills the shell; kill by pid. +4. Browser census: navigate with `?v=` cache-bust (browser disk-caches http.server responses); `waitForSelector('.mi-card', ...)` before evaluating — rendering is async (fetch). Geometry checks: rect-gap to 1px for bottom-pinned chips; screenshot for visual proof. `screenshot({path})` may not write — use the returned omp-sshots artifact. +5. In headless pages `new Audio()` is detached: prove voice playback via `tab.requests()` (206 on the /sfx file) + the `mi-speaking` class, not a DOM `