docs(radiance): correct the tokeniser corpus name and statsdump auth note
This commit is contained in:
1 parent
6e80cd44f0
commit
bdc0613739
2 files changed
+3
-2
No files matched your search
@@ -181,7 +181,8 @@ refusal**, not an engine fault: read the journal line and the argv, not the GPU.
|
||||
proposals held, KV bytes and the tokens they address, stream misses and bytes, prefix hits and
|
||||
evictions, preemptions and queue depth. `/stats`,
|
||||
`/graph`, `/sessions` back the dashboard. `scripts/statsdump.py` records `/stats` to JSONL with
|
||||
time and gap, for a machine with no scraper.
|
||||
time and gap, for a machine with no scraper (its key comes from `--api-key` or
|
||||
`RADIANCE_API_KEY`).
|
||||
- `RADIANCE_PROFILE_EVERY`, `RADIANCE_DUMP_BUF`/`RADIANCE_DUMP_DIR`, `RADIANCE_OP_ORACLE`(+) and the
|
||||
`RADIANCE_DEBUG_*` family are debug instruments; they are not deployment settings.
|
||||
|
||||
|
||||
@@ -255,7 +255,7 @@ eight-rank all-reduce shard weights and times `lockstep`/`ring`/`split` forms at
|
||||
```
|
||||
rad-tokcheck [options] <tokenizer.json> <corpus-file>...
|
||||
|
||||
--hf-dir DIR compare against HuggingFace ids in DIR/<corpus>.ids
|
||||
--hf-dir DIR compare against HuggingFace ids in DIR/<corpus basename>.ids
|
||||
(little-endian int32, encode(text, add_special_tokens=False))
|
||||
--no-ref skip the reference encoder (slow)
|
||||
--user encode with parse_special off
|
||||
|
||||
Reference in new issue
Block a user