Docker containers on Thunder Compute instances now come with GPU support enabled by default through the “thunder” runtime. This means you can run Docker containers with GPU access without any additional configuration. For more information about GPU compatibility, see our compatibility guide.

Thunder compute is incompatible with the base nvidia-container-toolkit. Trying to remove the existing container toolkit and installing your own will lead to issues running docker containers.

​
Getting Started

  1. Connect to a Thunder Compute instance using the quickstart guide

  2. Run your Docker containers as normal - GPU support is automatically enabled.

If you don’t need GPU capabilities in the docker container at all, it makes more sense to run with docker’s runc runtime. No need to add the —runtime flag in this case, runc is set as the default runtime.

​
Example

# Run a container with GPU support
docker run --runtime=nvidia ubuntu:22.04 nvidia-smi
# Run Ollama server (see our [Deepseek guide](/guides/deepseek-r1-running-locally-on-thunder-compute) for an example use case)
docker run --runtime=nvidia -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

In a docker-compose file, that would look like this:

services:
  ollama:
    image: ollama/ollama
    runtime: nvidia
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    restart: unless-stopped

  gpu-test:
    image: ubuntu:22.04
    runtime: nvidia
    command: nvidia-smi

volumes:
  ollama:

If you get an error that looks like docker: unexpected EOF, try running the command again. For more troubleshooting tips, see our troubleshooting guide.

​
Additional Info

​
Supported Base Images

Most modern Docker images are supported:

  • Ubuntu 22.04 and newer base images are fully supported
  • Ubuntu 20.04 base images are supported in beta
  • Other distributions like Alpine and Debian are supported

​
Thunder Runtime

Thunder Compute instances repalce the nvidia runtime with the thunder runtime for all Docker containers. The “thunder” runtime is identical to the nvidia runtime while injecting requirements needed for Thunder Compute GPU support.

​
Need Help?

If you encounter any issues or have questions about Docker support, please contact our support team.