Skip to main content
CleverThis
Forgejo
Sign in

Compute Fleet

The Compute Fleet lets you register your own servers, bare-metal machines, or Kubernetes clusters with CleverThis and use them as compute backends for containers, model endpoints, and Ray workloads. It is managed under Settings → Compute.

Compute Fleet bridges your private infrastructure with the CleverThis control plane. Your machines connect outward via a persistent WebSocket tunnel — no inbound ports or VPN are required. Once a node is registered, you can launch containers, download models, and start inference endpoints on it from the CleverThis UI.

Clusters and nodes​

The Compute Fleet is organized into clusters. A cluster is a logical group of nodes that share configuration (orchestration mode, access policies). Nodes are individual machines registered within a cluster.

Compute Fleet
└── Cluster A (standalone)
    ├── Node 1 (GPU workstation)
    └── Node 2 (CPU server)
└── Cluster B (kubernetes)
    ├── Node 1 (control plane)
    ├── Node 2 (GPU worker)
    └── Node 3 (GPU worker)

A namespace can be bound to a compute cluster, directing its workloads (containers, model endpoints) to that cluster's nodes.

Creating a cluster​

  1. Go to Settings → Compute in the left sidebar.

  2. Click Create Cluster.

  3. Fill in the form:

    FieldDescription
    NameHuman-readable label (e.g. home-lab, prod-gpu-cluster)
    DescriptionOptional — helps identify the cluster at a glance
    Orchestration modeHow containers are managed on this cluster (see below)

  4. Click Create.

Orchestration modes​

ModeDescriptionWhen to use
standaloneContainers run directly via Docker on each node. CleverThis manages placement.Single-machine labs, GPU workstations, simple self-hosted setups
kubernetesContainers are scheduled as Kubernetes Pods. CleverThis sends manifests to the kube API.Existing K8s clusters — EKS, GKE, AKS, or self-managed
argocdGitOps mode: workloads are deployed via ArgoCD. CleverThis pushes YAML to a Git repo and ArgoCD syncs.Teams using GitOps workflows; audit trail of every deployment

You cannot change the orchestration mode after a cluster is created. Delete and recreate the cluster if you need a different mode.

Adding nodes to a cluster​

Each node must run the CleverThis compute agent, which establishes an outbound tunnel to the control plane.

Generate a join token​

  1. Select your cluster from the list.
  2. Open the Commands tab.
  3. Click Generate Join Token.
  4. A dialog shows a one-time join command. Copy it immediately — the token is shown only once and expires in 24 hours.

The join command looks like:

curl -fsSL https://cdn.cleverthis.com/agent/install.sh | \
  CLUSTER_TOKEN=ct_node_TOKEN sh

Run the join command on your node​

SSH into the target machine and run the join command. The script:

  1. Installs Docker (if not already present)
  2. Downloads the CleverThis compute agent
  3. Registers the agent with the control plane using the token
  4. Starts the agent as a background service that dials out to agent.cleverthis.com

After 10–30 seconds, the node appears in the Nodes tab of the cluster with status online.

Requirements for a node​

  • Linux (Ubuntu 20.04+ recommended; Debian, CentOS, Rocky also work)
  • Docker 20+ or containerd installed (or installed by the script)
  • Outbound TCP to agent.cleverthis.com:443 (HTTPS WebSocket)
  • For GPU workloads: NVIDIA driver 525+ and nvidia-container-toolkit

GPU node setup​

For GPU workloads (model serving, training), additional setup is required before joining the node:

1. Install NVIDIA drivers​

# Ubuntu 22.04
sudo apt-get install -y linux-headers-$(uname -r)
sudo apt-get install -y nvidia-driver-535   # or latest stable

# Verify
nvidia-smi

Driver version 525 or later is required. Older drivers do not support all CUDA features used by vLLM and Ray.

2. Install nvidia-container-toolkit​

The container toolkit allows Docker containers to access the GPU:

# Add the NVIDIA package repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit

# Configure Docker to use the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify containers can see the GPU
docker run --rm --runtime=nvidia --gpus all nvidia/cuda:12.2-base-ubuntu22.04 nvidia-smi

3. Join the cluster​

After GPU setup, generate a join token and run the standard join command. The compute agent detects available GPUs automatically and reports them to the control plane.

The Nodes tab shows the GPU model (e.g. NVIDIA A100-SXM4-40GB) and current VRAM utilization once the node is online.

Minimum hardware recommendations​

Use caseCPURAMGPUStorage
CPU agent (data processing, scripting)4 cores16 GB—50 GB
Small GPU inference (7B model, fp16)8 cores32 GB24 GB VRAM200 GB
Large GPU inference (70B model)16 cores128 GB80 GB VRAM (×2)500 GB
Ray head node (coordinator)4 cores16 GB—50 GB
Ray training worker (7B fine-tune)8 cores64 GB40 GB VRAM200 GB

Node management​

The Nodes tab inside a cluster lists all registered nodes with their status and current workload.

ColumnDescription
NameAuto-assigned from the machine hostname; rename after joining
Statusonline, offline, cordoned, or draining
CPUCurrent CPU utilization
MemoryCurrent RAM utilization
GPUGPU model and utilization (if present)
ContainersNumber of containers running on this node

Node states​

StateMeaning
onlineNode is connected and accepting new workloads
offlineAgent is not reachable (network failure, machine off, service stopped)
cordonedNode will not receive new workloads; existing ones continue
drainingWorkloads are being migrated off; node enters cordoned when empty

Cordoning a node​

Cordoning prevents new containers from being scheduled on a node while leaving existing containers running. Use this before maintenance (OS updates, driver upgrades).

Click Cordon on the node row. To resume scheduling, click Uncordon.

Draining a node​

Draining actively migrates all running containers to other nodes in the cluster, then cordons the node. Use this before shutting down a node or performing hardware work that requires stopping all processes.

Click Drain. CleverThis signals each container to stop, waits for graceful shutdown, and starts replacement containers on other nodes. Once empty, the node is automatically cordoned.

Removing a node​

Click Delete on the node row. This deregisters the node from the cluster. The compute agent on the machine continues running but can no longer receive commands; stop and remove it manually:

systemctl stop cleverthis-agent
systemctl disable cleverthis-agent

Managing the compute agent​

The CleverThis compute agent runs as a systemd service named cleverthis-agent.

Viewing agent status and logs​

# Check if the agent is running
systemctl status cleverthis-agent

# View recent logs
journalctl -u cleverthis-agent -n 100 --no-pager

# Follow live logs
journalctl -u cleverthis-agent -f

Useful log patterns to look for:

  • connected to agent.cleverthis.com — the tunnel is established
  • GPU detected: NVIDIA A100 — GPU was successfully enumerated
  • heartbeat sent — periodic health check (every 30 seconds)
  • command received: container.create — an incoming command from the control plane

Updating the agent​

The agent updates itself automatically when the control plane pushes a new version. Manual update (if needed):

curl -fsSL https://cdn.cleverthis.com/agent/install.sh | sh

Re-running the install script updates the agent in place and restarts the service without re-registering the node.

Restarting the agent​

If the node shows offline despite the machine being up:

systemctl restart cleverthis-agent

The agent reconnects within 5–10 seconds and the node returns to online.

Kubernetes mode setup​

In kubernetes mode, CleverThis sends container manifests directly to your cluster's Kubernetes API server. The compute agent on each K8s node acts as a bridge.

Prerequisites​

  • A running Kubernetes cluster (1.24+)
  • kubectl configured on each node you register
  • ClusterAdmin role for the service account used by the agent
  • The cluster API server reachable from the agent

Join procedure​

Join each Kubernetes node using the standard join command. The agent on K8s nodes automatically detects the presence of a kubeconfig and switches to K8s mode.

The agent uses the node's default kubeconfig (~/.kube/config or the KUBECONFIG environment variable). For production clusters, create a dedicated service account:

kubectl create serviceaccount cleverthis-agent -n kube-system
kubectl create clusterrolebinding cleverthis-agent-binding \
  --clusterrole=cluster-admin \
  --serviceaccount=kube-system:cleverthis-agent

Then set the CLEVERTHIS_KUBECONFIG environment variable in the agent's service unit to use the service account kubeconfig.

What CleverThis creates in Kubernetes​

When a container or model endpoint is deployed to a Kubernetes cluster, CleverThis creates the following resources:

ResourceDescription
DeploymentThe container workload with the specified image and resource requests
ServiceClusterIP service exposing the container's port
Ingress (optional)External access if an endpoint URL is requested

All resources are created in the cleverthis-workloads namespace (auto-created on first use). CleverThis manages these resources exclusively — do not modify them manually, as changes will be overwritten.

ArgoCD mode setup​

In argocd mode, CleverThis manages workloads by pushing YAML manifests to a Git repository, and ArgoCD syncs them to the cluster. This gives you a complete GitOps audit trail: every deployment change is a Git commit.

Prerequisites​

  • ArgoCD 2.6+ installed in your Kubernetes cluster
  • A Git repository that ArgoCD watches (the "app-of-apps" repo)
  • ArgoCD API access credentials (token or username/password)
  • The argocd CLI or API accessible from your compute node

Configuration​

After creating an ArgoCD-mode cluster and generating a join token, set the following environment variables before running the install script:

export ARGOCD_SERVER=argocd.your-domain.com
export ARGOCD_TOKEN=your-argocd-api-token
export ARGOCD_GIT_REPO=git@github.com:your-org/your-infra-repo.git
export ARGOCD_GIT_BRANCH=main

curl -fsSL https://cdn.cleverthis.com/agent/install.sh | \
  CLUSTER_TOKEN=ct_node_TOKEN sh

The agent stores these credentials locally and uses them to push new manifests and trigger ArgoCD syncs when the control plane sends deployment commands.

Deployment flow in ArgoCD mode​

Control plane sends "container.create" command
    │
    ▼
Compute agent generates Kubernetes YAML manifest
    │
    ▼
Agent commits YAML to the Git repo (creates/updates a file in manifests/)
    │
    ▼
ArgoCD detects the Git change and syncs the cluster
    │
    ▼
Container is running; agent reports status back to control plane

Each deployment creates a Git commit like:

cleverthis: deploy container vllm-llama3 (ct_container_abc123)

You can browse these commits to see the full history of what CleverThis deployed and when.

Sending commands to nodes​

The Commands tab lets you send management commands directly to one or all nodes in a cluster. This is useful for administrative tasks and debugging.

Available command types​

CommandDescription
container.createCreate and start a new container on the node
container.startStart a stopped container by name or ID
container.stopStop a running container
container.destroyPermanently remove a container and its data
model.downloadPre-fetch a model from HuggingFace Hub or a registry to the node's cache
endpoint.startStart an inference endpoint (backed by a downloaded model)
endpoint.stopStop a running inference endpoint

Command targeting​

Select a target from the Node dropdown:

  • All nodes — broadcasts the command to every online node in the cluster
  • Specific node — sends the command to one node

All commands are non-blocking — the UI shows the dispatch status, and the actual execution result is reported asynchronously in the node's event log.

Pre-fetching models​

Large models (7B+) take 10–30 minutes to download. Use model.download to pre-cache models before starting a Ray training job or inference endpoint:

  1. In the Commands tab, select Node → the node with the GPU.
  2. Set Command type to model.download.
  3. Enter the HuggingFace model ID (e.g. meta-llama/Meta-Llama-3-8B).
  4. Click Send. The download runs in the background.

Check progress in the node's event log. Once complete, the model is cached at /var/cache/cleverthis/models/ on the node and loads instantly for future jobs.

Networking​

Outbound tunnel​

The compute agent establishes a persistent outbound WebSocket connection to agent.cleverthis.com:443. All communication between the CleverThis control plane and your nodes flows through this tunnel.

Firewall requirements:

DirectionProtocolDestinationPortPurpose
OutboundTCP (TLS)agent.cleverthis.com443Control plane tunnel
OutboundTCP (TLS)cdn.cleverthis.com443Agent installer and updates
OutboundTCP (TLS)Docker Hub, ghcr.io, etc.443Container image pulls
OutboundTCP (TLS)huggingface.co443Model downloads (if using HF Hub)

No inbound ports need to be opened on your firewall. Nodes that cannot make outbound HTTPS connections will show offline.

Node-to-node communication​

For Ray clusters, worker nodes communicate with the head node over the cluster's internal network. Ensure the following ports are accessible between nodes within the same cluster:

PortProtocolPurpose
6379TCPRay GCS (Global Control Store / Redis)
8265TCPRay Dashboard and Job API
8000TCPRay Serve HTTP server
10001–19999TCPRay object store transfers

If nodes are on separate networks with firewall rules between them, open these ports between nodes in the same compute cluster.

Endpoint URLs​

When a container or model endpoint is started on a compute node, CleverThis assigns it a public HTTPS endpoint URL (e.g. https://endpoint-abc123.compute.cleverthis.com). Traffic is proxied through the control plane — the node does not need to accept inbound connections directly.

Relation to Ray​

Ray clusters are built on top of compute clusters. When you create a Ray cluster, you select which compute cluster provides the underlying nodes. The Ray head node and worker nodes are scheduled as containers on those compute nodes.

Relation to Containers​

Containers created from the Containers page run on CleverThis's own managed infrastructure. Containers that need to run on your hardware should be dispatched via Compute Fleet commands rather than the Containers page.

Security model​

Outbound-only architecture​

Nodes establish outbound-only connections to agent.cleverthis.com. Your firewall does not need to allow any inbound connections. All traffic is TLS-encrypted using certificates signed by CleverThis's certificate authority.

Token lifecycle​

The join token is single-use and expires in 24 hours. After a node is registered, it authenticates using a long-lived certificate issued during registration. If a node is compromised, delete it from the cluster to revoke its certificate immediately. The node loses control plane access within 60 seconds of deletion.

Secret isolation​

Your workspace API keys and model weights are never exposed to compute nodes by default. Only the containers and models you explicitly dispatch are sent to your nodes. No other tenant's workloads or data are ever placed on your registered nodes.

Command authorization​

All commands sent to nodes (container.create, model.download, etc.) are authorized by the workspace that owns the cluster. Only users with workspace Owner or Admin role can send commands to compute nodes.

Troubleshooting​

Node shows offline immediately after joining​

The agent joined but lost connection. Common causes:

  1. Firewall blocking outbound 443 — run curl -v https://agent.cleverthis.com on the node. If it times out, your network blocks outbound HTTPS WebSocket.
  2. Docker not running — the agent requires Docker. Run docker info to check.
  3. Agent service crashed — check journalctl -u cleverthis-agent -n 50 for errors.

Node shows offline after working correctly​

The tunnel dropped, often due to a network interruption, machine reboot, or sleep/wake cycle. Usually self-recovers within 60 seconds as the agent reconnects. If it stays offline:

systemctl restart cleverthis-agent

Node not appearing in Nodes tab after running join command​

  • Verify the join command completed without errors (look for registration successful in the output).
  • Check the join token has not expired (24-hour limit).
  • Ensure the machine has outbound access to agent.cleverthis.com.
  • Try generating a new join token and re-running the install script.

Command dispatched but nothing happened​

Commands are asynchronous. Check the node's event log in the Nodes tab. If the log shows the command was received but failed:

  • container.create failure — Docker is not running, or the image could not be pulled. Check docker images and network connectivity to the image registry.
  • model.download failure — HuggingFace Hub is not reachable, or the model ID is incorrect. Test with curl https://huggingface.co on the node.
  • endpoint.start failure — model weights are not yet downloaded (run model.download first), or insufficient GPU VRAM.

GPU not detected after joining​

The GPU appears in the Nodes tab as none even though NVIDIA hardware is present:

  1. Verify nvidia-smi works on the host.
  2. Verify nvidia-container-toolkit is installed and Docker is configured to use the NVIDIA runtime (docker info | grep -i nvidia).
  3. Restart the agent: systemctl restart cleverthis-agent.
  4. If still not detected, check agent logs for GPU enumeration failed.

Container stays in creating state on compute node​

The container image is taking time to pull, or the pull failed. On the compute node:

# Check if Docker is pulling the image
docker events --filter type=image --since 5m

# Check if image pull failed
journalctl -u cleverthis-agent -n 50 | grep -i error

If the registry requires authentication, configure Docker credentials on the node:

docker login ghcr.io   # or your private registry

Then restart the agent so it picks up the new credentials.