58 lines
3.5 KiB
Markdown
58 lines
3.5 KiB
Markdown
---
|
|
name: open-notebook-provision-model-rest
|
|
description: "Provision or repair an open-notebook model (LLM/TTS/embedding) over its REST API on Dankclaw: credential→model→defaults order, the defaults-store-the-ID trap, the /api/embed 500-that-succeeds, and the two-note ranking proof for embeddings."
|
|
---
|
|
|
|
# open-notebook model provisioning over REST
|
|
|
|
Use when adding/repairing a model row (embedding, LLM, TTS) in open-notebook on Dankclaw, when semantic
|
|
search is dead, or when an embed error appears in the worker log. Config lives in the **database**
|
|
(encrypted with `OPEN_NOTEBOOK_ENCRYPTION_KEY`) — no repo file mirrors it, REST or the UI are the only doors.
|
|
|
|
## Access (never put the bearer in argv)
|
|
|
|
The API bearer IS `OPEN_NOTEBOOK_PASSWORD`; the API publishes on loopback `127.0.0.1:5056` only, so run
|
|
from Dankclaw over ssh. `ssh dankclaw 'curl -H "Authorization: Bearer …"'` leaks the secret into
|
|
`/proc/<pid>/cmdline` — use a mode-600 curl config:
|
|
|
|
```bash
|
|
PW=$(grep -m1 '^OPEN_NOTEBOOK_PASSWORD=' /home/dank/Containers/open-notebook/env | cut -d= -f2- | tr -d '"')
|
|
cfg=$(mktemp); chmod 600 "$cfg"; printf 'header = "Authorization: Bearer %s"\n' "$PW" > "$cfg"
|
|
B=http://127.0.0.1:5056 # not 5055: seerr holds the host's 5055
|
|
```
|
|
|
|
## Order, and the address rule
|
|
|
|
`POST /api/credentials` → `POST /api/models` (`credential: <credential id>`) → `PUT /api/models/defaults`
|
|
→ any profile that names a model (speaker `voice_model`, episode `speaker_config`/`outline_llm`).
|
|
|
|
- Credential provider for any in-house OpenAI-compatible server: **`openai_compatible`** (not `openai`),
|
|
with a dummy `api_key` when the upstream does not authenticate.
|
|
- The app's URL guard refuses base URLs resolving in `169.254.0.0/16`, so `host.containers.internal`
|
|
cannot be stored; a **container name on `dankclaw_default`** (e.g. `http://frank-dragoon-halogen:8731/v1`)
|
|
or another machine's RFC1918 IP both pass.
|
|
|
|
## The two traps that eat the most time
|
|
|
|
1. **`PUT /api/models/defaults` takes the model ID, never the name.** `"default_embedding_model":
|
|
"qwen3-embedding-0.6b"` is accepted and echoed back, then every job dies in the worker with
|
|
`Model with ID qwen3-embedding-0.6b not found`. Use the row's `model:…` id (GET defaults first, change
|
|
one key, PUT the whole object back — the other keys in that response are ids and must be preserved).
|
|
2. **`POST /api/embed` answers 500 *after* the work succeeds.** Upstream `EmbedResponse.command_id`
|
|
declares `str` and the worker returns a `RecordID`, so FastAPI raises in serialization only. The truth is
|
|
`podman logs open-notebook | grep "Successfully embedded"`, never the HTTP code. Note saves
|
|
(`POST /api/notes`) submit `embed_note` themselves; `/api/embed` is only for backfills.
|
|
|
|
## Proof it actually works
|
|
|
|
- Per row: `POST /api/models/<id>/test` → `Embedding dimensions: 1024` (embeddings) or a real completion.
|
|
- Embeddings, discriminating test: create **two** unrelated notes, then
|
|
`POST /api/search` `{"query": "...", "type": "vector", "search_notes": true, "limit": 5}` — the enum is
|
|
`text|vector` (not `semantic`) and the score field is `similarity`. Each query must rank its own note
|
|
high and the other low (measured 0.659/0.292 vs 0.906/0.318). One note proves nothing: `total_count` is 1
|
|
either way. Delete the probe notes afterwards (`DELETE /api/notes/<id>`).
|
|
- Confirm which engine served it: `podman logs <engine container> | grep "POST /v1/embeddings"`.
|
|
|
|
Owner pages: `serverAppConfigs-docs/Dankclaw/Open Notebook.md` (app-side shapes), and the endpoint's own
|
|
page for whatever serves the model.
|