serverAppConfigs

1,013 Commits 42 Branches 8 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
Comrade Chief Designer	fd864b5a42	frank-primary: compile changelist — real diffs, break diagnostics, patch 04, corrected overclaim Working the dyno name-map blocker one step at a time, with the tools to see it: - All four patch artifacts are now genuine diff -u patches (they were prose + pseudo hunks before, useless to patch(1)). Verified against the pinned image: each applies clean and reproduces its recorded patched sha256 (6e9cb81b… / 0751852e… / 732363bb… / 84969993…). - New: scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch — takes GLM's MLA block off the fused-norm JIT kernel (R4D_GLM_FUSE_QKV_RMSNORM=0; the flag is already a constructor arg glm5next sets at attention.py:583). Reason: in models/common/ops/fused_qk_rmsnorm.py the kernel object is bound at line 223, AFTER its use at line 211 — the launcher family (@kernel_launcher, 39 call sites) is what breaks dynamo's break analysis. - New diagnostics, both reusable and recorded: TORCH_LOGS=graph_breaks + VLLM_USE_AOT_COMPILE=0 to name each break with its user stack, and /home/dank/dynamo-probe (sitecustomize.py on PYTHONPATH, VLLM_DYNAMO_DUMP_ATTRS=1) to print the dynamo name-map miss detail. Gotchas: a .pth on PYTHONPATH is NOT executed, and the scratch launcher reset ENVARGS (now ${ENVARGS:-}), which silently dropped env vars and wasted a boot. Measurement discipline correction: I first recorded the 'Read uninitialized cell' and 'Unsupported method call' failures as fatal blockers. They are not established as such — they appear only when TORCH_LOGS=graph_breaks is set, while a flag-free break-tolerance run gets past break handling and dies on the name-map KeyError. The likely mechanism is that graph_breaks logging makes dynamo build a break explanation, and that analysis itself trips on the uninitialized cell. Patch 04's header, the runbook and the work log now carry the confound explicitly rather than the overclaim. tests/run.sh: 362 seams pass.	2026-09-30 13:59:09 -07:00
Comrade Chief Designer	bd8c72ff47	frank-primary: profile the GLM decode step and stage the compile changelist A torch-profiler kernel-time window (496 decode tokens, rank0 trace parsed for device-activity time) on the deployed GLM-5.3-Flash shape falsifies this page's own "the expert gather is the bound" explanation: 46.18 s of kernel time / 496 tokens = ~93 ms per token (actual step ~104 ms) 34.8 % vectorized_elementwise_kernel 708,500 launches (1,428/token) 10.0 % r4d_gemm_moe_w4a16_nt_b16_m64 (the MoE GEMM) 6.7 % elementwise_kernel_manual_unroll 6.4 % wvSplitK_hf_sml_ (split-K GEMM) 8.9 % router: sbtopk + warpMergeSortTopK + bitonicSortKVInPlace 6.5 % r4d_mhc_* (hyper-connections) 1.05 % r4d_moe_lru_gather_kernel <-- the retracted suspect The earlier arithmetic was circular (it assumed the gather consumed the whole step). The real bound is unfused pointwise glue: the fork compiles its own Qwen trunk (support_torch_compile in models/qwen4_exp/amd/model.py) and not its GLM trunk, so the engine boots with "provides neither a compiled submodule nor breakable CUDA graphs. Overriding to FULL_DECODE_ONLY." That also explains why MTP4 measured flat — pointwise cost scales with the token count, so spec decode amortizes nothing here — and it retires arena/hit-rate tuning (the gather is 1 %). Staged as documented patch artifacts (not wired into any serve script; the engine does not yet boot with them) — scripts/patches/glm53-compile-0{1,2,3}*.patch: -01 models/glm5next/common/model.py decorate Glm5NextModel (R4D_GLM_COMPILE) -02 model_executor/kernels/r4d_lib.py freeze 10 capability probes that do os.path.exists/.so lookups inside the traced forward. functools.lru_cache does NOT protect them: dynamo re-traces the wrapped body even when warm (verified in-image); a plain module-global read compiles. -03 compilation/wrapper.py make fullgraph=True opt-out, because suppress_errors does NOT cover graph breaks and this trunk can never satisfy fullgraph (indexer metadata syncs to host — .item() at glm5next/amd/sparse_indexer.py:263,419). With all three the trunk traces (Dynamo bytecode transform time 1.08 s) and dies on KeyError: in_proj_qkvbfg_a in the mHC/KDA path — a latent bug the compile exposes, and the remaining blocker. Also records two structural constraints: no new HIP kernels are possible (r4d.so ships without sources), which binary-blocks the fused router at GLM's 288 experts; and DFlash2 for GLM is a TARGET-side port, not a drafter port (incoai/GLM-5.3-Flash-DFlash2 is architectures ["DFlash2DraftModel"] + model_type qwen3, already mapped at registry.py:636; glm5next lacks only the aux-hidden-state plumbing deepseek_v2.py demonstrates). Runbook frontmatter, intro, mermaid and wall section, AGENTS.md and the work log corrected to match the measurement. tests/run.sh: 362 seams pass.	2026-09-30 09:36:48 -07:00

Author

SHA1

Message

Date

Comrade Chief Designer

fd864b5a42

frank-primary: compile changelist — real diffs, break diagnostics, patch 04, corrected overclaim

Working the dyno name-map blocker one step at a time, with the tools to see it:

- All four patch artifacts are now genuine diff -u patches (they were prose + pseudo
hunks before, useless to patch(1)). Verified against the pinned image: each applies
clean and reproduces its recorded patched sha256 (6e9cb81b… / 0751852e… / 732363bb…
/ 84969993…).
- New: scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch — takes GLM's
MLA block off the fused-norm JIT kernel (R4D_GLM_FUSE_QKV_RMSNORM=0; the flag is
already a constructor arg glm5next sets at attention.py:583). Reason: in
models/common/ops/fused_qk_rmsnorm.py the kernel object is bound at line 223, AFTER
its use at line 211 — the launcher family (@kernel_launcher, 39 call sites) is what
breaks dynamo's break analysis.
- New diagnostics, both reusable and recorded: TORCH_LOGS=graph_breaks +
VLLM_USE_AOT_COMPILE=0 to name each break with its user stack, and
/home/dank/dynamo-probe (sitecustomize.py on PYTHONPATH, VLLM_DYNAMO_DUMP_ATTRS=1)
to print the dynamo name-map miss detail. Gotchas: a .pth on PYTHONPATH is NOT
executed, and the scratch launcher reset ENVARGS (now ${ENVARGS:-}), which silently
dropped env vars and wasted a boot.

Measurement discipline correction: I first recorded the 'Read uninitialized cell' and
'Unsupported method call' failures as fatal blockers. They are not established as
such — they appear only when TORCH_LOGS=graph_breaks is set, while a flag-free
break-tolerance run gets past break handling and dies on the name-map KeyError. The
likely mechanism is that graph_breaks logging makes dynamo build a break
*explanation*, and that analysis itself trips on the uninitialized cell. Patch 04's
header, the runbook and the work log now carry the confound explicitly rather than
the overclaim.

tests/run.sh: 362 seams pass.

2026-09-30 13:59:09 -07:00

Comrade Chief Designer

bd8c72ff47

frank-primary: profile the GLM decode step and stage the compile changelist

A torch-profiler kernel-time window (496 decode tokens, rank0 trace parsed for
device-activity time) on the deployed GLM-5.3-Flash shape falsifies this page's
own "the expert gather is the bound" explanation:

  46.18 s of kernel time / 496 tokens = ~93 ms per token (actual step ~104 ms)
   34.8 %  vectorized_elementwise_kernel      708,500 launches (1,428/token)
   10.0 %  r4d_gemm_moe_w4a16_nt_b16_m64      (the MoE GEMM)
    6.7 %  elementwise_kernel_manual_unroll
    6.4 %  wvSplitK_hf_sml_                    (split-K GEMM)
    8.9 %  router: sbtopk + warpMergeSortTopK + bitonicSortKVInPlace
    6.5 %  r4d_mhc_*                           (hyper-connections)
    1.05 % r4d_moe_lru_gather_kernel           <-- the retracted suspect

The earlier arithmetic was circular (it assumed the gather consumed the whole
step). The real bound is unfused pointwise glue: the fork compiles its own Qwen
trunk (support_torch_compile in models/qwen4_exp/amd/model.py) and not its GLM
trunk, so the engine boots with "provides neither a compiled submodule nor
breakable CUDA graphs. Overriding to FULL_DECODE_ONLY." That also explains why
MTP4 measured flat — pointwise cost scales with the token count, so spec decode
amortizes nothing here — and it retires arena/hit-rate tuning (the gather is 1 %).

Staged as documented patch artifacts (not wired into any serve script; the
engine does not yet boot with them) — scripts/patches/glm53-compile-0{1,2,3}*.patch:

  -01 models/glm5next/common/model.py    decorate Glm5NextModel (R4D_GLM_COMPILE)
  -02 model_executor/kernels/r4d_lib.py  freeze 10 capability probes that do
      os.path.exists/.so lookups inside the traced forward. functools.lru_cache
      does NOT protect them: dynamo re-traces the wrapped body even when warm
      (verified in-image); a plain module-global read compiles.
  -03 compilation/wrapper.py             make fullgraph=True opt-out, because
      suppress_errors does NOT cover graph breaks and this trunk can never
      satisfy fullgraph (indexer metadata syncs to host — .item() at
      glm5next/amd/sparse_indexer.py:263,419).

With all three the trunk traces (Dynamo bytecode transform time 1.08 s) and dies
on KeyError: in_proj_qkvbfg_a in the mHC/KDA path — a latent bug the compile
exposes, and the remaining blocker.

Also records two structural constraints: no new HIP kernels are possible (r4d.so
ships without sources), which binary-blocks the fused router at GLM's 288
experts; and DFlash2 for GLM is a TARGET-side port, not a drafter port
(incoai/GLM-5.3-Flash-DFlash2 is architectures ["DFlash2DraftModel"] +
model_type qwen3, already mapped at registry.py:636; glm5next lacks only the
aux-hidden-state plumbing deepseek_v2.py demonstrates).

Runbook frontmatter, intro, mermaid and wall section, AGENTS.md and the work log
corrected to match the measurement. tests/run.sh: 362 seams pass.

2026-09-30 09:36:48 -07:00