serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-ciru-orca.md

---
type: reference
title: CIRU v4.4.1 Orca on Dankclaw (frank-large)
description: The active frank-large runbook — CIRU v4.4.1 (QSA window hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the Dankclaw Strix Halo iGPU: the private host build, the runtime gate, the payload, the deployed config, MTP and PLE, the client contract, and the failure modes worth knowing.
tags:
  - llm
  - ciru
  - dankclaw
  - frank-large
  - strix-halo
  - gfx1151
  - qwen4exp
  - rocm
  - mtp
timestamp: 2026-09-19
---

# CIRU v4.4.1 Orca on Dankclaw (frank-large)

> **ACTIVE since 2026-09-19.** It replaced halogen-flash-server (see
> [[halogen-flash-server]], the current rollback), which had replaced
> ik_llama.cpp (see [[frank-large-ik-uncensored]]). Deployed as a **root
> systemd unit** (`scripts/frank-large.service`, `User=dank`), port 7770, on the
> same gfx1151 iGPU the halogen quadlet used. The RTX 3090 it does not touch
> belongs to [[frank-small-gemma4-12b]].

CIRU is a llama.cpp fork tuned for the Qwen3.8-Flash-Next ("qwen4exp")
architecture on AMD gfx1151 (Strix Halo). v4.4.1 carries a **QSA window
hotfix** that is what makes vision and the MTP drafter work on this build; the
pre-applied hotfix source ships in the model repo under
`runtime/v4.4.1/qsa-window-hotfix/`. Like every other fleet endpoint it is a
llama.cpp runtime — `llama-bench` is its own harness.

## Shape

One root unit, one process, one published port:

| half | port | reachable |
|---|---|---|
| llama-server (OpenAI front-end + engine, single process) | 7770 | published as host **7770** |

Unlike the halogen predecessor (two ports: an unauthenticated engine on 8730
and an OpenAI front-end on 8731), CIRU is a single `llama-server` process —
there is no separate engine port to keep private.

`/v1/models` reports the loaded model name, **`Qwen3.8-Flash-CIRU-STRIX-Orca`**
(the launcher sets no `--alias`). The endpoint serves **whatever model name a
request names** — llama.cpp does not reject an unknown id — so the harness's
`frank-large` label in `models.yml` works without a rename.

## Runtime

A **private host build**, not a container image. The v4.4.1 source with the QSA
window hotfix, built on the host against the private ROCm SDK that
`scripts/ciru/setup-linux-amd.sh` installs into the checkout's own venv:

| path | what |
|---|---|
| `/home/dank/GitPackages/ciru-runtime-v4.4.1-qsa-hotfix/` | the checkout (source + build + venv) |
| `build-gfx1151-sdk/bin/llama-server` | the built binary |
| `.venv-rocm/` | the private ROCm SDK venv (~5.7 GB); the binary resolves its libs through RUNPATH |
| `scripts/ciru/run-orca-server.sh` | the launcher (`exec`'d by the serve script) |
| `profiles/strix-halo-production.env` | the 36-variable kernel bundle + PLE pager defaults, sourced by the launcher AFTER `frank-large.env` (every line is `${VAR:-default}`, so a value set in `frank-large.env` wins) |

### The runtime gate

`run-orca-server.sh` sources `run-server.sh`, which **exits 2** unless
`${repo_root}/runtime/hip/lib/libamdhip64.so` and `runtime/rocr/lib/
libhsa-runtime64.so` exist (unversioned, no suffix). It prepends both to
`LD_LIBRARY_PATH` and `ldd`-checks that the built `libllama-common.so` carries
the "Qwen4Exp MTP image position" marker and the HIP lib carries the
`flash_attn_index_mask_clear/set/empty` symbols before exec.

The build links the venv through RUNPATH, so those two files are an existence
gate, not a real dependency. `serve-frank-large.sh` satisfies the gate
idempotently: it finds the venv's unversioned devel symlinks
(`.venv-rocm/lib/python*/site-packages/_rocm_sdk_devel/lib/`) and `ln -sfn`-s
both into `runtime/{hip,rocr}/lib/`. **The first deploy of this endpoint failed
exactly here** — the build linked the venv but `runtime/` did not exist, so the
launcher exited 2 and the unit crash-looped until the serve script created the
symlinks.

## Payload

`scripts/download-frank-large-orca.sh` assembles `jcbtc/Qwen3.8-Flash-CIRU-
STRIX-Orca` (HuggingFace, public and ungated) into
`/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca/`, sha256-verified. The
launcher refuses to start without the main GGUF, the MTP head and the PLE
sidecar:

| file | bytes | role |
|---|---|---|
| `Qwen3.8-Flash-CIRU-STRIX-Orca.gguf` | 79,397,818,912 | the main model (Orca variant) |
| `mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf` | 4,135,893,440 | the MTP drafter head |
| `ple/ple.payload.bin` | 52,429,053,952 | the per-layer-embedding (PLE) payload — mmap'd and paged, not resident |
| `ple/ple.manifest.json` | — | PLE manifest |
| `ple/ple.scale.bf16` | — | PLE scale |
| `vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj` | 904,004,320 | the F16 vision projector (on disk, `ENABLE_VISION=0`) |

The main GGUF ships as **transport parts** that the download script assembles
and verifies; the PLE is the large one (52.4 GB) and is paged by the engine's
own workers, not held in RAM.

## Deployed config

Tunables live in `scripts/frank-large.env`; the launcher owns the flag list
(`-sm none` one iGPU no split, `-ngl all`, f16 KV, flash attention,
cont-batching, the PLE sidecar, the MTP drafter, the production kernel-bundle
profile). The departures worth naming:

| flag | value | why |
|---|---|---|
| `CONTEXT_SIZE` | `262144` | the model's FULL trained window (the operator's "full context" call); the GGUF carries no rope scaling |
| `PARALLEL_SLOTS` | `1` | one stream — and the MTP drafter **requires exactly one slot** in this runtime, so the operator's ask and the drafter's constraint are the same number |
| `ENABLE_MTP` | `1` | the MTP drafter, depth 4 (the Orca variant's own `mtp_depth_default`) |
| `ENABLE_VISION` | `0` | text-only for now; the F16 projector is on disk, one flip to enable. The fleet's image path is [[frank-small-gemma4-12b]] either way |
| `PLE_CACHE_MIB` | `4096` | the PLE pager's hot window kept resident; the vendor production-profile default. The 52.4 GB payload is mmap'd and paged around it |
| `HOST` | `0.0.0.0` | the launcher consumes it for `--host`; the value doubles as a JIT-cache directory name, which is why `0.0.0.0` is safe (HIP reads `HOST` bare — see the AGENTS.md note) |

`TEMP`, `HOST` and `GPU_MAX_HEAP_SIZE` are the three names to avoid for any
new tunable: HIP reads them bare.

## MTP

The Multi-Token Prediction drafter is a **separate Q8_0 head** (not inside the
main GGUF), loaded by the launcher. It requires exactly one slot, which is why
`PARALLEL_SLOTS=1`. Depth 4 is the Orca variant's default.

Verified at deploy (2026-09-19): the per-task `draft acceptance = 0.65625
(21/32), mean len = 3.62` — the drafter is engaged and accepting, not silently
standing down.

## PLE

The per-layer-embedding payload (52.4 GB) is the Qwen3.8-Flash-Next
architecture's signature: token embeddings live outside the main weight set and
are paged on demand by the engine's own workers (`GGML_QWEN4EXP_PLE_WORKERS=16`,
`GGML_QWEN4EXP_PLE_IO_WORKERS=32` in the production profile). `PLE_CACHE_MIB=4096`
is the hot window kept resident; the rest is mmap'd file cache. **Do not treat
the 52.4 GB as resident memory** — `free` and the dashboards will not show it
pinned, and that is correct.

## Client contract

The differences from a stock llama.cpp endpoint that bite:

- **`reasoning_content`** carries the reasoning trace (the standard OpenAI
  field). The serve script passes `--chat-template-kwargs
  '{"preserve_thinking":true}'` so prior thinking blocks are kept across turns;
  a request-level `chat_template_kwargs` still wins.
- **`reasoning_effort` is gated.** Qwen3.8's template accepts only
  `xhigh`, `medium`, `low` and **500s on anything else** (not a clamp). The
  harness folds every level onto that set in `models.yml`
  (`reasoningEffortMap`: minimal/low→low, medium→medium, high/max→xhigh) —
  subagents run at `high`, which is why the map exists.
- **Text only.** `ENABLE_VISION=0`, so image content parts are refused. The
  fleet's image path is [[frank-small-gemma4-12b]].
- **Sampling.** The server has no sampler defaults of its own; the harness
  applies the per-model recipe from `models.yml` (temp 1.0 / top_p 0.95 / top_k
  20 / min_p 0.0). A client that stays silent gets the launcher's default.
- **`tools` works** — the launcher passes `--jinja`, and the Qwen3.8 template
  supplies the tool-call grammar. Verified at deploy: a tool call returns
  `tool_calls`.

## Deploy and operate

```bash
./scripts/deploy.sh dankclaw --service frank-large
```

The handler pulls the checkout (ff-only), (re)links the unit, and gates on
`/health`. First boot is the slow one (model load ~46 s + graph capture); a
warm restart is fast.

```bash
sudo systemctl status frank-large            # the root unit
sudo journalctl -u frank-large -n 60         # startup account: runtime check, model load, "listening on http://0.0.0.0:7770", MTP draft loaded
curl -s http://127.0.0.1:7770/health         # ok
curl -s http://127.0.0.1:7770/v1/models      # Qwen3.8-Flash-CIRU-STRIX-Orca
```

**Stop and mask before any GPU experiment** (the AGENTS.md bench protocol):
`sudo systemctl stop frank-large && sudo systemctl mask frank-large`. The
frank-large serve process is a `llama-server` **and so is frank-small on the
same host** — never `pkill -x llama-server`. If you must kill by pattern, use
`pkill -f 'Qwen3.8-Flash-CIRU-STRIX-Orca'` (the model path is unique to this
endpoint and does not appear in the deploying shell's own command line).

## Troubleshooting

| symptom | cause |
|---|---|
| launcher exits 2: "The v4.4 HIP/ROCr runtime is missing" | `runtime/{hip,rocr}/lib/*.so` absent — the serve script's `ln -sfn` did not run, or the venv was moved. Re-run the serve script or rebuild: `bash $RUNTIME_DIR/scripts/ciru/setup-linux-amd.sh --install-host-deps` |
| "missing SDK venv runtime under .../.venv-rocm" | the venv's unversioned devel symlinks are gone — rebuild the runtime |
| "missing target model / missing payload" | a download did not finish — re-run `scripts/download-frank-large-orca.sh` |
| loads, `/health` ok, first completion slow (~45-60 s) | the PLE pager pulling the hot window in from file cache — not a crash, do not restart over it |
| a subagent 500s naming `reasoning_effort` | the Qwen3.8 template gate — the harness `reasoningEffortMap` must fold the level onto {xhigh, medium, low} |
| `free` / dashboards do not show 52 GB pinned | correct — the PLE is mmap'd file cache, not resident; only `PLE_CACHE_MIB` is hot |

Retrieval: this is a **root** unit, so `journalctl -u frank-large` is the
durable record (no `--user`, no `sudo` needed for the journal once the unit is
readable). There is no container log — there is no container.

## Rollback

The predecessor is intact: the halogen payload stays at
`/home/dank/models/halogen` (retired, not deleted), and the halogen quadlet is
archived under `.AGENTTRASH/`. Rolling back is a `git revert` of this change set
plus a deploy — **but the root unit and the halogen quadlet both claim port
7770**, so the handler's port-free gate covers that race. See
[[halogen-flash-server]].

The `amd_iommu=off` kernel argument is separate and reverts on its own — see
[[dankclaw-iommu-translation-off]].

## Citations

- [jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca](https://huggingface.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca) — weights, MTP head, PLE payload, and the v4.4.1 QSA window-hotfix source tarball
- [CIRU runtime v4.4.1](https://github.com/ciru) — `scripts/ciru/run-server.sh` (runtime gate), `profiles/strix-halo-production.env` (kernel bundle), `docs/BUILD_LINUX.md`