GPU sandboxes
for agents
GPU sandboxes for RL environments and evals. Scale from zero to hundreds of parallel agents in seconds with a simple SDK.
Zero to hundreds in seconds
Boots in 1-2s. Fan out one sandbox per rollout or eval task and run them in parallel. Stop each one when its work ends to scale back to zero.
Agent-first by design
Create, exec and stop are single SDK, MCP or REST calls, so your agent runs the whole lifecycle without a human in the loop.
One sandbox per rollout, one per task
Fan out in parallel, get results back, and let every sandbox expire.
RL environments
Fan out every rollout to its own GPU sandbox with a real kernel. Run them side by side, and terminate each one after its rollout finishes.
- One sandbox per rollout, created in a single call
- Its own kernel, so environments can’t see each other
- A TTL cleans up anything your trainer forgets
import thunder_sandbox as thunder
runs = []
for _ in range(64):
box = thunder.Sandbox.create(
gpu_type=thunder.GPUType.H100, gpu_count=1,
timeout=900,
)
box.wait_until_ready()
box.upload("rollout.py", "/home/ubuntu/rollout.py")
run = box.exec("python3", "/home/ubuntu/rollout.py",
"--episodes", "64")
runs.append((box, run))
for box, run in runs:
run.wait()
box.terminate()
Evals
Score every task in its own sandbox, in parallel, so one bad sample can’t affect the rest. Close the network and the model can’t look answers up.
- One sandbox per task, with nothing shared between them
- Network closed or allowlisted, per sandbox
- Per-second billing, so short tasks stay cheap
import thunder_sandbox as thunder
def score(task):
box = thunder.Sandbox.create(
gpu_type=thunder.GPUType.H100, gpu_count=1,
timeout=600, block_network=True,
)
box.wait_until_ready()
box.upload(task, "/home/ubuntu/task", recursive=True)
run = box.exec("python3", "/home/ubuntu/task/grade.py")
exit_code = run.wait()
box.terminate()
return exit_code
GPU CI/CD
Let your pipeline or coding agent run CUDA tests on an NVIDIA GPU for every push, each in a fresh VM. A four-minute job bills four minutes.
- A clean VM per run, with nothing left over from the last
- Build the image from a Dockerfile in your repo
- A TTL ends hung jobs before they run up a bill
import thunder_sandbox as thunder
ci = thunder.Sandbox.create(
gpu_type=thunder.GPUType.H100, gpu_count=1,
timeout=1800,
)
ci.wait_until_ready()
ci.upload("repo", "/home/ubuntu/repo", recursive=True)
tests = ci.exec("python3", "-m", "pytest", "-x", "tests/gpu",
workdir="/home/ubuntu/repo")
exit_code = tests.wait()
ci.terminate()
Your agent runs the whole lifecycle
An MCP server for coding agents, a Python SDK for your code, and a REST API for everything else.
import thunder_sandbox as thunder
sandbox = thunder.Sandbox.create(
gpu_type=thunder.GPUType.H100, gpu_count=1,
)
sandbox.wait_until_ready()
process = sandbox.exec("nvidia-smi")
import thunder_sandbox as thunder
sandbox = thunder.Sandbox.create(
cpu=4, memory=32, storage=50,
gpu_type=thunder.GPUType.H100, gpu_count=1,
timeout=900,
)
sandbox.wait_until_ready()
process = sandbox.exec("nvidia-smi")
Nothing left running when the work is done
Every sandbox has a TTL. Stop it from code or let it expire on its own.
Billed by the second, down to zero
A100 from $1.95/GPU-hr; H100 from $2.95/GPU-hr, billed per second.
A full VM for untrusted code
Every sandbox boots its own Ubuntu microVM on Firecracker, not a shared container.
Isolation
Each sandbox runs its own kernel, so agent-written code can’t reach other workloads.
Compatibility
systemd, docker compose and the rest of a normal Linux box work like they do on a server.
Bring your own image
Build from a Dockerfile or pull from a registry.
Decide what each sandbox can reach
Close the network for evals so the model can’t look answers up. Pick a policy at create time, or switch a running sandbox.
Full outbound internet. Private and cloud metadata networks stay blocked.
sandbox.update_network_policy(
block_network=False,
)
- Allowed · last 60s
- 1,284
- Refused · last 60s
- 37
Pick the right tool for the job
Sandboxes for parallel, automated work. Instances for long-running development.
| Sandboxes | Instances | |
|---|---|---|
| Lifetime | Minutes to hours, ends on a TTL | Runs until you delete it |
| Created from | Python SDK, MCP or REST API | Console, CLI or VS Code |
| Disk | Wiped when it ends | Kept, with snapshots |
| Billing | Per second | Per minute |
| Best for | RL, evals, agents and CI | Development, training, notebooks |
Give every agent its own sandbox
Fan out hundreds of RL rollouts and evals from a few lines of Python or one MCP call. Request access and we'll get your team set up.