Run Ollama on a GPU
Start a small model, verify the API and keep model data in a persistent Docker volume.
1. Prepare
Complete GPU setup and set ARVICA_OLLAMA_IMAGE to a tested official image tag. A Docker volume persists across container removal but remains on this host; it is not a managed network volume or backup.
2. Start and verify
Run on the GPU host. Expect a text response and an API model list containing llama3.2:1b. Review the model licence for your intended use. This is a smoke test, not a performance claim.
#!/usr/bin/env bash
set -euo pipefail
: "${ARVICA_OLLAMA_IMAGE:?Set a tested ollama/ollama image tag or digest}"
docker run -d --name arvica-ollama --gpus all \
-p 127.0.0.1:11434:11434 -v arvica-ollama:/root/.ollama \
"$ARVICA_OLLAMA_IMAGE"
docker exec arvica-ollama ollama pull llama3.2:1b
docker exec arvica-ollama ollama run llama3.2:1b "Reply with OK"
curl --fail http://127.0.0.1:11434/api/tags
3. Access and troubleshoot
Create the SSH tunnel below on your computer. If the model download fails, check network and disk space. For GPU detection errors, check the host driver and Docker GPU runtime. Inspect docker logs arvica-ollama. Stop/remove only the named container when finished; the named volume remains.
ssh -i ~/.ssh/arvica_ed25519 -N -L 11434:127.0.0.1:11434 ubuntu@INSTANCE_IP