Containers

The Containers page lets you launch ephemeral cloud compute instances — containers backed by CPU or GPU hardware — that act as agent runtimes, custom inference servers, or general-purpose execution environments for your AI workloads.

Containers run OCI-compatible Docker images and are accessible via a dedicated endpoint URL when running. They are billed hourly at a rate determined by the hardware spec.

When to use Containers

Scenario	Container role
Run a self-hosted model server (vLLM, Ollama, TGI)	Container hosts the inference process; endpoint URL exposed to the actor framework
Execute a data processing job or batch inference script	Container runs the script and exits
Host a custom tool server that Actors can call	Container stays running, exposing an HTTP API
Fine-tune a model locally before uploading	GPU container runs the training script
Sandbox code execution for an AI agent	Container provides an isolated shell

Available hardware specs

Spec	CPU	RAM	GPU	Hourly rate
cpu-2-ram-8	2 vCPU	8 GB	—	$0.10/hr
cpu-4-ram-16	4 vCPU	16 GB	—	$0.20/hr
cpu-8-ram-32	8 vCPU	32 GB	—	$0.40/hr
gpu-a100-40gb	16 vCPU	128 GB	A100 40GB	$5.00/hr

Container costs are deducted from your workspace credit wallet at the same rate as API token usage. Charges accrue while a container has status running. Stopped containers do not incur charges, but the container record is retained and can be started again.

Creating a container

1. Navigate to Containers in the sidebar.

2. Click Create.

3. The creation wizard has three steps:

   Step 1 — Image Enter the OCI image to run. The default is ubuntu:22.04. You can use any publicly reachable image (ghcr.io, Docker Hub, etc.) or a private registry image if your cluster has pull credentials configured.

   Examples:

   • ubuntu:22.04 — general-purpose Linux shell
   • python:3.11-slim — Python scripting
   • ghcr.io/huggingface/text-generation-inference:2.0 — TGI server
   • vllm/vllm-openai:latest — vLLM OpenAI-compatible server

   Step 2 — Hardware Select the hardware spec from the dropdown. The spec determines CPU cores, RAM, GPU availability, and hourly cost. Choose the smallest spec that meets your workload requirements.

   Step 3 — Name Give the container a descriptive name (e.g. vllm-llama-3, batch-pipeline-v2). Names do not need to be unique but should be human-readable.

4. Click Create. The container is created and starts in the creating → running state within 15–60 seconds depending on image pull time.

Container lifecycle

Status	Meaning	Charges accruing?
creating	Image is being pulled and container is initializing	Yes
running	Container is live; endpoint URL is active	Yes
stopped	Container is halted but preserved; no compute allocated	No
destroying	Container is being permanently removed	No

Starting a stopped container

Click the ▶ Play icon on the container row. The container restarts using the same image, spec, and configuration. Any ephemeral data written during the previous run is lost unless your image writes to a mounted volume.

Stopping a container

Click the ■ Stop icon. The container receives a SIGTERM, then SIGKILL after a grace period. The container record is preserved in stopped state; click Start to bring it back. Stopped containers cost nothing.

Destroying a container

Click the 🗑 Delete icon. This permanently removes the container and all associated data. Destruction cannot be undone. Destroy containers you no longer need to avoid accumulating unused records.

Endpoint URL

When a container's entrypoint exposes an HTTP service, CleverThis assigns it a public HTTPS endpoint URL. This URL is shown in the Endpoint column of the container table.

If your image runs a server process, it should listen on port 8080 (the convention for container endpoint exposure). Traffic to the endpoint URL is proxied to port 8080 on the running container.

Specifying a different port

If your image uses a different port (e.g. 8000 for vLLM, 11434 for Ollama), set the PORT environment variable when creating the container. CleverThis proxies the endpoint URL to whatever port you specify:

Image	Default port	Set PORT to
vllm/vllm-openai	8000	8000
ollama/ollama	11434	11434
ghcr.io/huggingface/text-generation-inference	80	80
Custom Python server	8080 (if using our convention)	—

Using the endpoint in an Actor

agents:
  - id: call_vllm
    type: tool
    tool: http
    url: "{{ env.CONTAINER_ENDPOINT_URL }}/v1/chat/completions"
    method: POST
    body_template: |
      {
        "model": "llama3",
        "messages": {{ messages | tojson }}
      }
    headers:
      Content-Type: "application/json"
    inject_as: system
    result_template: "{{ response.body }}"

Set CONTAINER_ENDPOINT_URL in the namespace's Actor environment variables to avoid hard-coding the URL in the YAML.

Environment variables

Environment variables let you pass configuration into containers at startup without modifying the image.

Setting environment variables

In the Create container wizard (or when creating via API), add environment variables as key-value pairs in the Environment section:

Variable	Example value	Common use
PORT	8000	Tell CleverThis which port to proxy
MODEL	meta-llama/Meta-Llama-3-8B	Which model to load (for vLLM/TGI images)
HF_TOKEN	hf_abc123...	HuggingFace Hub API token for gated models
QUANTIZATION	awq	Quantization mode for supported inference servers
MAX_MODEL_LEN	4096	Maximum context length
DTYPE	float16	Weight precision

Referencing API keys securely

Rather than hard-coding API keys in environment variables visible in the UI, store them as Namespace secrets and reference them:

# In an Actor that calls this container:
env:
  CONTAINER_ENDPOINT_URL: "{{ secrets.CONTAINER_URL }}"
  API_KEY: "{{ secrets.INFERENCE_API_KEY }}"

For container environment variables themselves, any value entered in the Create dialog is encrypted at rest and only decrypted when the container starts.

Startup command override

To override the default entrypoint command (useful for debugging or running a different process in the same image), set the COMMAND environment variable:

COMMAND=python -m vllm.entrypoints.openai.api_server --model mistralai/Mistral-7B-v0.1 --port 8000

Private registry images

By default, containers pull from public registries (Docker Hub, ghcr.io, etc.). For private images, configure pull credentials on your workspace.

Supported registries

Registry	Authentication method
Docker Hub (private repos)	Username + access token
GitHub Container Registry (ghcr.io)	GitHub PAT with read:packages scope
AWS ECR	IAM credentials (access key + secret)
Google Container Registry / Artifact Registry	Service account JSON key
Azure Container Registry	Service principal or admin credentials
Self-hosted (Harbor, Gitea)	Username + password

Adding registry credentials

1. Go to Settings → Registries.
2. Click Add Registry.
3. Enter the registry URL (e.g. ghcr.io or registry.your-company.com).
4. Enter credentials appropriate for your registry type.
5. Click Save. Credentials are stored encrypted.

Once credentials are configured, enter the full image path in the container creation wizard:

ghcr.io/your-org/your-private-image:latest

CleverThis uses the configured credentials automatically at pull time.

Storage and volumes

Ephemeral storage

By default, a container's filesystem is ephemeral: any files written during the container's lifetime are lost when the container is destroyed (or when it exits, even if you click Start again). Stopped containers do retain their filesystem until they are explicitly destroyed.

Persistence across restarts

Data written during a run is preserved in stopped state — if you stop the container and start it again, the filesystem is intact. Only destroying a container clears the data.

This means for one-off jobs that write output files, you can:

1. Start the container.
2. Wait for it to complete and write its results.
3. Stop it (data preserved).
4. Download or access the results before destroying it.

Shared storage for multi-container workflows

For workflows where multiple containers need to share data, use external storage:

Option	Setup	Best for
S3-compatible object storage	Mount with s3fs or write directly with boto3	Large files, model weights, datasets
NFS mount	Mount NFS path in entrypoint script	Datasets shared across containers in the same network
Redis or PostgreSQL (another container)	Connect via endpoint URL	Structured data, job queues, state sharing

To mount an S3 bucket inside a container, include the setup in your entrypoint:

# Example entrypoint.sh
apt-get install -y s3fs
s3fs my-bucket /data -o use_path_request_style -o url=https://s3.us-east-1.amazonaws.com
python process.py

Pass AWS credentials via environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.

Networking

Endpoint access

Each container gets one public HTTPS endpoint URL (e.g. https://endpoint-abc123.containers.cleverthis.com). The endpoint proxies to the port specified by the PORT environment variable (default 8080).

Endpoint URLs are:

• Active immediately when the container reaches running state
• Accessible over the public internet (no authentication by default)
• Deactivated when the container is stopped

Securing endpoints

If your container hosts a service that should not be publicly accessible, add HTTP basic authentication or API key validation inside the service itself. The CleverThis endpoint URL provides HTTPS but no built-in authentication.

For Actor-to-container communication, pass an authentication header in the Actor YAML:

headers:
  Authorization: "Bearer {{ secrets.CONTAINER_API_KEY }}"

Multiple ports

CleverThis exposes one proxied endpoint URL per container. If your service needs multiple ports (e.g. HTTP on 8080 and gRPC on 9090), either:

• Use a reverse proxy inside the container (nginx) to route both to port 8080.
• Run separate containers for each port.

Inter-container networking

Containers do not share a private network by default. To connect containers:

1. Use the public endpoint URL of one container as the target in another.
2. For Actors, reference the endpoint URL via environment variables or namespace secrets so it can change without updating the Actor YAML.

Viewing container logs

Click any container row to open the detail panel. Select the Logs tab to see:

• Stdout/stderr from the container process, streamed in real time
• Lines from container startup (image pull output, initialization messages)
• Any crash output if the container exited unexpectedly

Log output is shown in reverse-chronological order. Use Search logs to filter by keyword. Logs are retained for 7 days after a container is destroyed.

Following logs live

Click Follow to auto-scroll to new log entries as they arrive. This is useful for monitoring a batch job's progress or debugging a server startup sequence.

Downloading logs

Click Download to save all logs as a .txt file. Useful for long-running jobs where you want to post-process the output.

Common container patterns

Pattern: Batch inference job

Run a script that reads a dataset, generates predictions, writes results, and exits:

1. Create a container with python:3.11-slim (CPU) or gpu-a100-40gb (GPU).
2. Set the COMMAND environment variable to your script:

   COMMAND=python /app/run_inference.py --input s3://my-bucket/data --output s3://my-bucket/results

3. The container runs to completion and exits with code 0 on success.
4. Check logs to confirm success; destroy the container.

Cost estimate: cpu-8-ram-32 at $0.40/hr × 2 hours = $0.80 for a 2-hour batch job.

Pattern: Persistent inference server

Run a model server that stays up to handle requests from Actors:

1. Create a gpu-a100-40gb container with vllm/vllm-openai:latest.
2. Set environment variables:

   PORT=8000
   MODEL=mistralai/Mistral-7B-v0.1
   HF_TOKEN=hf_your_token

3. Copy the endpoint URL and store it as a namespace secret.
4. Reference it in your Actor YAML via {{ secrets.VLLM_ENDPOINT }}.
5. Leave the container running for as long as you need inference.

Pattern: Development shell

Start an interactive shell for debugging or data exploration:

1. Create a cpu-8-ram-32 or gpu-a100-40gb container with ubuntu:22.04.
2. Use the endpoint URL with a WebSocket terminal (if your image includes ttyd or similar) or SSH via a tunnel script in the entrypoint.
3. Install dependencies and run ad-hoc scripts.
4. Stop the container (data preserved) when done; restart later.

Pattern: Scheduled batch job

For jobs that run on a schedule (e.g. nightly data processing):

1. Keep the container in stopped state between runs.
2. Use the CleverThis API or an external scheduler to start the container, wait for completion, and stop it.

Example using the CleverThis management API:

# Start
curl -X POST https://api.cleverthis.com/a2a \
  -H "Authorization: Bearer ct_mgmt_YOUR_KEY" \
  -d '{"method":"container.start","params":{"id":"ct_container_abc123"}}'

# Poll until stopped (container exits on completion)
while true; do
  STATUS=$(curl -s .../container/ct_container_abc123 | jq -r '.status')
  [ "$STATUS" = "stopped" ] && break
  sleep 10
done

Pattern: Custom tool server for Actors

Run a small FastAPI service that Actors invoke as a tool:

# server.py
from fastapi import FastAPI
app = FastAPI()

@app.post("/search")
async def search(query: str):
    results = my_search_function(query)
    return {"results": results}

# Dockerfile
FROM python:3.11-slim
COPY requirements.txt server.py .
RUN pip install -r requirements.txt
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8080"]

Deploy with cpu-2-ram-8 (cheapest), copy the endpoint URL, and reference it in an Actor YAML tool: http node.

Cost management

Container costs are per-hour, not per-request. A container running 24 hours at cpu-4-ram-16 costs 24 × $0.20 = $4.80, regardless of whether it received any traffic.

Best practice: Stop containers when not in use. For workloads that need a container intermittently (e.g. batch jobs that run nightly), set up an automation that starts the container, runs the job, and stops it when done.

To avoid surprise charges, you can set a workspace-level spend cap in Usage & Billing. Container charges count toward the same credit wallet as API token usage.

Cost comparison by spec

Spec	Daily cost (24 hr)	Monthly cost (30 days)	Typical use
cpu-2-ram-8	$2.40	$72	Light scripting, small APIs
cpu-4-ram-16	$4.80	$144	Data processing, medium APIs
cpu-8-ram-32	$9.60	$288	Heavy CPU workloads
gpu-a100-40gb	$120	$3,600	Model serving, training

For GPU containers: stop when not actively serving requests. A GPU container idle 22 hours per day and serving 2 hours costs the same as one serving 24 hours.

Relation to Compute Fleet

Containers managed from this page are hosted on CleverThis's managed infrastructure. For more control — running containers on your own servers or Kubernetes cluster — see Compute Fleet. The Compute Fleet lets you register your own hardware and dispatch container commands to it.

Key differences:

	Containers page	Compute Fleet
Hardware	CleverThis managed cloud	Your own hardware
Billing	Hourly at CleverThis rates	Your own infrastructure costs
Setup	Zero setup; create and run	Agent install required per node
GPU options	Fixed set of managed specs	Any GPU you own
Control	UI and API	UI, API, and direct machine access

Troubleshooting

Container stuck in creating

The image is taking a long time to pull, or the pull failed.

• Large images (10+ GB like GPU-ready vLLM) take 5–15 minutes on first pull.
• Subsequent starts of the same image are fast (image is cached).
• If stuck for more than 20 minutes, destroy and recreate — the image pull may have stalled silently.

Container exits immediately after starting

The container process crashed on startup. Check the Logs tab for error output. Common causes:

• Missing command: the entrypoint script doesn't exist at the expected path.
• Missing dependency: a Python package or system library is missing. Rebuild your Docker image with the dependency included.
• Port mismatch: if your service listens on a port other than 8080, set the PORT environment variable.
• OOM (out of memory): the container ran out of RAM. Upgrade to a larger spec or reduce the process's memory footprint.

Endpoint URL returns 502 or connection refused

The container is running but the service inside hasn't started yet, or it crashed.

1. Check the Logs tab to see if the service started successfully.
2. Verify the service is listening on the correct port (PORT env var).
3. Give large model servers (vLLM, TGI) 2–5 minutes to load model weights before expecting the endpoint to respond.

GPU not available inside container

The container's process cannot see the GPU even though a gpu-a100-40gb spec was selected.

• Verify with docker run --gpus all nvidia/cuda:12.2-base-ubuntu22.04 nvidia-smi on the host node — if this fails, the NVIDIA container toolkit is misconfigured.
• For images that need the CUDA runtime, use a CUDA base image (e.g. nvidia/cuda:12.2-runtime-ubuntu22.04) rather than a generic Python image.

Private image pull fails (authorization error)

Registry credentials may not be configured, or the credentials are expired.

1. Go to Settings → Registries and verify credentials for the registry.
2. For GitHub Container Registry, ensure the PAT has read:packages scope.
3. For AWS ECR, ensure the IAM user/role has ecr:GetAuthorizationToken and ecr:BatchGetImage permissions.
4. Re-save credentials and retry creating the container.