diff --git a/omp-docs/tools/llama-cpp/index.md b/omp-docs/tools/llama-cpp/index.md index 8425263..b086ec3 100644 --- a/omp-docs/tools/llama-cpp/index.md +++ b/omp-docs/tools/llama-cpp/index.md @@ -124,31 +124,8 @@ print(completion.choices[0].message.content) --- -## Docker - -```bash -docker run -p 8080:8080 -v /models:/models \ - ghcr.io/ggml-org/llama.cpp:server \ - -m /models/model.gguf --host 0.0.0.0 --port 8080 - -# CUDA variant -docker run -p 8080:8080 --gpus all \ - ghcr.io/ggml-org/llama.cpp:server-cuda \ - -m /models/model.gguf --n-gpu-layers 99 -``` - ---- - -## Environment Variables - -All CLI options can be set via `LLAMA_ARG_` prefix: - -```bash -LLAMA_ARG_MODEL=/models/model.gguf LLAMA_ARG_CTX_SIZE=8192 llama-server -``` - ---- - ## Cross-References -- See [Setup Guide](setup.md) for build instructions, GPU offloading, and deployment. +- See [Setup Guide](setup.md) for build instructions, GPU offloading, the `LLAMA_ARG_*` + environment form, the Docker Compose example, and running under systemd. Deployment lives + there; this page is the API surface and the GGUF format. diff --git a/omp-docs/tools/llama-cpp/setup.md b/omp-docs/tools/llama-cpp/setup.md index 1bc7b35..b220d9b 100644 --- a/omp-docs/tools/llama-cpp/setup.md +++ b/omp-docs/tools/llama-cpp/setup.md @@ -111,6 +111,14 @@ cmake --build build --config Release -t llama-server | `-lora FNAME` | LoRA adapter (repeatable) | — | | `-ngl, --gpu-layers N` | Layers to offload to GPU | auto | +### Environment Variables + +Every CLI option can be set from the environment instead: prefix the upper-snake form of the +long flag with `LLAMA_ARG_` — `LLAMA_ARG_MODEL=/models/model.gguf LLAMA_ARG_CTX_SIZE=8192 +llama-server`. This is how the container and systemd units pass configuration (see the Compose +example below), because a unit file with one flag per line is auditable and a shell command line +with forty is not. + ### Context & Performance | Flag | Description | Default |