Local model orchestration
Run Ollama or llama.cpp on your own GPU nodes, alongside commercial providers, from the same Helm chart.
- Ollama deploys as a DaemonSet on GPU-labeled nodes by default, or a fixed-replica pool Deployment for mixed-GPU clusters, sharing a ReadWriteMany PVC model cache.
- llama.cpp renders one DaemonSet + Service per configured model, sharing the same PVC-backed model cache.
- Optional GPU device-plugin sub-charts for NVIDIA (with GFD + driver stack), AMD ROCm, and Intel, each disabled by default.