frank-large: mmap the CPU-overridden weights (no CUDA_Host pinning)
This commit is contained in:
1 parent
fe3fc62c41
commit
0022ba96ad
4 files changed
+143
-33
No files matched your search
@@ -7,6 +7,11 @@ Wants=network-online.target
|
||||
Type=simple
|
||||
User=dank
|
||||
Group=dank
|
||||
# Staging buffers for the CUDA transfers are page-locked; the weights
|
||||
# themselves stay mmapped (GGML_CUDA_NO_PINNED_WEIGHTS=1), but the staging
|
||||
# budget still needs room. Parity with the halogen quadlet's
|
||||
# Ulimit=memlock=-1:-1.
|
||||
LimitMEMLOCK=infinity
|
||||
Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
|
||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
|
||||
Restart=on-failure
|
||||
|
||||
Reference in new issue
Block a user