Files

T

Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.

2026-10-19 05:00:43 -07:00

4.5 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

dealign Ornith-1.5-35B-A3B CRACK on Dankserver

Active 2026-08-25. frank-zealot serves this model. It replaced the CRACK Q8_0 file on the same upstream llama.cpp binary (b10121). The Q6_K file was downloaded from the model repository and validated with llama-gguf before installation.

frank-zealot (port 7770) serves Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf from dealignai on Hugging Face. It is a MoE model (256 experts, 8 active) with a hybrid GatedDeltaNet (SSM) + attention architecture, abliterated (uncensored). The GGUF also contains a bundled MTP head, but this deployment uses ngram-mod instead.

Model Files

File	Size
`Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf`	27.2 GiB
`mmproj-Ornith-1.5-35B-A3B-f16.gguf` (vision)	0.84 GiB

Store the files under /home/dank/models/Ornith-1.5-35B-A3B-CRACK/.

Layer Audit and Expert Offload

The Q6_K file contains 41 transformer blocks and 753 tensors. Each block contains 630 MiB of expert tensors. Blocks 3, 7, 11, 15, 19, 23, 27, 31, 35, and 39 contain full attention. Block 40 contains the bundled MTP head.

The non-expert tensors total 2.02 GiB. The layer totals are 660.96 MiB for ordinary SSM blocks, 655.81 MiB for full-attention blocks, and 662.40 MiB for block 40. The expert portion is 630 MiB in every block.

The selected configuration keeps all non-expert tensors on CUDA. It also keeps expert tensors from every full-attention block, plus every block from 35 through 40, on CUDA. The unique GPU expert set is 3,7,11,15,19,23,27,31,35,36,37,38,39,40. Expert tensors from all other blocks move to CPU through this explicit override:

OVERRIDE_EXPERT='blk\.(0|1|2|4|5|6|8|9|10|12|13|14|16|17|18|20|21|22|24|25|26|28|29|30|32|33|34)\.ffn_(up|down|gate|gate_up)_(ch|)exps'
OVERRIDE_BUFT=CPU

The 16K benchmark found the hybrid policy fastest among the three tested placements. A later no-spec/ubatch sweep established that ubatch=1024 is a stable low-memory choice. The current service uses ngram-mod only; MTP is intentionally disabled. The full-attention expert set and final blocks are kept on CUDA.

To list the buffer type names on the host, run a bogus override:

llama-server --model <ANY> -c 64 -ot foo=BADTYPE

On Dankserver the available buffer types are CPU and CUDA0.

Speculative Decoding

The GGUF contains a bundled MTP head, but this deployment deliberately does not use it. The current service enables only ngram-mod, with a 48-token minimum, 64-token maximum, and 24-token lookup match:

--spec-type ngram-mod
--spec-ngram-mod-n-min 48
--spec-ngram-mod-n-max 64
--spec-ngram-mod-n-match 24

In the controlled 8K test, no speculation produced approximately 53.7–53.9 generation tokens per second. MTP was slower for that workload, so it remains disabled.

Launch Excerpt

llama-server \
  -m Ornith-1.5-35B-A3B-CRACK-Q6_K.gguf \
  --mmproj mmproj-Ornith-1.5-35B-A3B-f16.gguf \
  --chat-template-file chat-template.jinja \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-min 48 \
  --spec-ngram-mod-n-max 64 \
  --spec-ngram-mod-n-match 24 \
  --override-tensor "blk\\.(0|1|2|3|4|5|6|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|28|29|30|32)\\.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU" \
  -ngl all -c 491520 -np 3 \
  --ubatch-size 1024 \
  --cache-type-k f16 --cache-type-v f16 --no-mmap \
  --temp 1.0 --top-p 0.95 --top-k 20

Known Issues

The 27.2 GiB Q6_K file and 0.84 GiB vision projector fit the 62 GiB RAM box with the selected expert offload.
The context pool contains 491520 cells. Three slots receive 163840 cells each, which is 96K below the model context per stream.
--n-cpu-moe remains supported. This service prefers the explicit tensor override because the 16K benchmark found the hybrid placement fastest. It preserves every full-attention expert block and blocks 35–40 on CUDA.
MTP is bundled in the GGUF but intentionally disabled; ngram-mod is the only active speculative method.
The service uses the Qwen-Sharp template from peculiar-ragdoll/Qwen-Sharp-Chat-Templates through scripts/chat-template.jinja.

4.5 KiB Raw Blame History Unescape Escape