chore(llama-server): grow the model cache to 120Gi for candidate evaluation

The embedding model was picked from a recommendation and never compared. MTEB's German retrieval results put the deployed Qwen3-Embedding-0.6B well down the field, so a dozen candidates are being measured against the real archive with scripts/embed-eval-real.sh — and they will not fit in 40Gi alongside the two deployed models. Growing the shared cache rather than using a scratch volume is deliberate: on a slow connection the download is the long pole, and a shared cache lets the large candidates pull in the background while the small ones are already being measured. It also means nothing is fetched twice across runs, and a candidate that wins is already local when it is promoted. 120Gi against 309 GiB currently free in cephfs-data0. CephFS has allowVolumeExpansion, so this grows online without disturbing the running pods. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 11:10:04 +02:00
1 changed files with 16 additions and 6 deletions
@@ -1,10 +1,20 @@
 ---
-# Holds the GGUF weights, downloaded once by the Deployment's initContainer.
-# Prune-disabled so retiring this Kustomization does not throw away several
-# GiB of model files that would have to be pulled from Hugging Face again.
+# Holds the GGUF weights, downloaded once by llama-server itself via --hf-repo.
+# Prune-disabled so retiring this Kustomization does not throw away tens of GiB
+# of model files that would have to be pulled from Hugging Face again.
 #
-# ReadWriteMany so a second replica or a one-off benchmark Job can mount the
-# same weights; the models are read-only at inference time.
+# ReadWriteMany so a second replica, a one-off benchmark Job, or a candidate
+# model being evaluated can mount the same weights; the models are read-only at
+# inference time.
+#
+# Sized well beyond the two deployed models. The cache is shared by every
+# candidate under evaluation (scripts/embed-eval-real.sh), and on a slow
+# connection the download is the long pole — a shared cache lets the large
+# candidates pull in the background while the small ones are already being
+# measured, and means a model is never fetched twice across runs.
+#
+# CephFS supports online expansion (allowVolumeExpansion on the storage class),
+# so growing this does not disturb the running pods.
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
@@ -18,4 +28,4 @@ spec:
  storageClassName: ceph-filesystem
  resources:
    requests:
-      storage: 40Gi
+      storage: 120Gi