Cluster install values

This page documents the Helm values accepted by tau cluster install. The distribution chart bundles Kueue, KubeRay, the Tau core controller, GPU monitoring, the taugrid-core services chart, and a portable baseline queue into a single versioned release. Portal is enabled by default for the operator quickstart; Stellar, lifecycle recorder, and image prewarm remain disabled until the platform opts in.

Print this reference from your terminal:

tau cluster explain-values

The Helm release namespace is the only namespace setting for TauGrid system workloads and Services. tau cluster install defaults it to tau-system; --namespace <name> moves the Kueue, KubeRay, Tau controller, Portal, GPU monitoring, and other enabled first-party workloads together. The first-party charts follow their Helm release namespace, and the deprecated gpu-monitoring.namespace override must remain empty. Cluster-scoped resources remain cluster-scoped, and Kueue keeps its Kubernetes API aggregation binding in kube-system.

MultiKueue capability

The standard installation includes MultiKueue API/controller support and read-only prerequisite observation. Operators separately create worker credentials, AdmissionChecks, MultiKueueConfigs, MultiKueueClusters, dedicated queues, and profiles. TauCluster.status.conditions[MultiKueueReady] reports the health of those actual operator-owned prerequisites. See Multi-cluster execution before publishing a MultiKueue profile.

Components

Toggle sub-charts with components.<key>.enabled:

Baseline Queue

A portable Kueue queue bootstrapped on first install. These quotas bound concurrent Kueue admission; Kubernetes scheduling still enforces real capacity. Production operators should replace them with deliberate capacity policy.

CPU, memory, and GPU share one Kueue resource group so each GPU pod set receives one node flavor across all requested resources. taugrid-default-cpu has zero GPU quota. When GPU admission, topology, and tau-core-controller are enabled, the controller discovers Nodes with both tau.azure.com/gpu-class and allocatable nvidia.com/gpu, creates one topology-aware ResourceFlavor per observed GPU class, and appends its summed allocatable GPU capacity plus one ready tau.azure.com/torchrun-host-slot per GPU Node to the ClusterQueue. Each discovered flavor declares sku=gpu:NoSchedule as an admission taint so CPU-only workloads cannot consume its quota. Node creation, GPU-class label changes, and allocatable GPU or host-slot capacity changes trigger reconciliation; a new pool may initially contribute zero capacity and receive quota once kubelet reports its GPUs and the slot. GPU capacity grows monotonically: scaling a pool to zero does not remove the flavor or reduce its last known GPU quota, while slot quota tracks ready slot capacity. Explicitly configured flavors remain supported and are not pruned by controller discovery. Only GPU flavors carry topologyName; each workload’s policy.topology selects unconstrained, same-host, same-accelerator-domain, same-network-domain, or same-site. Raw Kubernetes manifests remain expert-controlled.

The baseline ClusterQueue opts into discovery with tau.azure.com/discover-gpu-flavors=true only while GPU admission, topology, and the controller are enabled. Custom ClusterQueues can use the same label; queues without it are never mutated by discovery and labeled queues must cover nvidia.com/gpu. Explicit chart-configured GPU flavors should set tau.azure.com/torchrun-host-slot quota to their GPU Node count. Omitting it is accepted for upgrade compatibility and renders zero slot quota, leaving multi-node torchrun pending until the quota is configured.

tau-core-controller assigns every Node a conservative topology identity and reconciles taugrid-gpu-topology with the hierarchy tau.azure.com/site → tau.azure.com/network-domain → tau.azure.com/accelerator-domain → kubernetes.io/hostname. Nodes without authoritative shared-site, fabric, or accelerator-island metadata receive deterministic singleton labels. tau.azure.com/region remains normalized metadata but is not a Kueue topology level.

The controller recognizes Azure from aks.azure.com/cloud=azure, the Azure provider ID, or managed AKS labels, and derives region from topology.kubernetes.io/region or aks.azure.com/region. Managed Azure Nodes share a regional site. Only Azure ND-series Nodes with an authoritative agent-pool label share a network domain, scoped by region, agent pool, and VM size. Other capacity uses singleton non-IB domains.

Externally joined Azure Flex nodes are detected from kubernetes.azure.com/managed=false or aks.azure.com/stretch-managed=true. Their provisioning template must stamp net.unbounded-cloud.io/site=<globally-unique-site>. Nodes share the top-level Flex site. Nodes may additionally stamp net.unbounded-cloud.io/network-domain=<provider-authoritative-fabric-id> to share a fabric. NVL72/GB200/GB300 nodes may stamp net.unbounded-cloud.io/accelerator-domain=<provider-authoritative-island-id> to share an accelerator island. Without those inputs, each Node receives singleton domains; TauGrid never infers accelerator sharing from GPU model or SKU alone. Zone is omitted because it is not consistently available across providers.

The topology name is fixed because the controller owns this object and Kueue makes ResourceFlavor.spec.topologyName immutable. Discovered flavors use deterministic names based on GPU class.

kubeadm:
  nodeLabels:
    net.unbounded-cloud.io/site: research-flex
    net.unbounded-cloud.io/network-domain: research-fabric-a

Portal

The following are defaults of the TauGrid umbrella distribution used by tau cluster install. The standalone taugrid-core chart keeps Portal disabled, so platforms that install that child chart directly must opt in explicitly. Portal follows the Helm release namespace selected by tau cluster install --namespace; the CLI default is tau-system.

These defaults make the Portal shell, Runs and run-detail views, Cluster Nodes view, and live Ray discovery available to an operator through the ClusterIP Service. See Configure Portal to separately configure Kusto-backed boards, the scoped computed Jobs board, KueueViz, an authenticated researcher endpoint, and a durable experiment store.

baselineQueue.gpu.flavors

Declare GPU-node taints (for example sku=gpu:NoSchedule) in each GPU flavor’s nodeTaints. This makes the flavor ineligible for CPU-only pods if generic CPU quota is exhausted. TauGrid injects sku=gpu and nvidia.com/gpu tolerations into GPU workloads, so those workloads remain eligible. Keep a matching taint out of the flavor’s tolerations: repeating it there would make Kueue automatically tolerate it for every pod and remove the CPU-isolation guard.

# taugrid-values.yaml
baselineQueue:
  gpu:
    flavors:
      - name: a100-pool
        nodeLabels:
          kubernetes.io/os: linux
          tau.azure.com/gpu-class: a100-80gb
        nodeTaints:
          - key: sku
            value: gpu
            effect: NoSchedule
        tolerations: []
        resources:
          - name: nvidia.com/gpu
            nominalQuota: "1"
          - name: tau.azure.com/torchrun-host-slot
            nominalQuota: "1"

Sub-Chart Pass-Through

The remaining top-level keys pass values directly to embedded sub-charts:

Example: GPU Cluster

TauGrid’s controller derives the node contract from node.kubernetes.io/instance-type and repairs drift:

Use tau-core-controller.tauCluster.extraNodeLabelRules for another reviewed VM size. Setting nodeLabelRules replaces the built-in catalog. CPU-only clusters and GPU pools scaled to zero remain ready when no catalog entry currently matches.

# taugrid-values.yaml: H200 cluster with GPU taints
baselineQueue:
  gpu:
    flavors:
      - name: h200-pool
        nodeLabels:
          kubernetes.io/os: linux
          kueue.azure.com/gpu-series: nd-h200-v5
          tau.azure.com/gpu-class: h200-141gb
        nodeTaints:
          - key: sku
            value: gpu
            effect: NoSchedule
        tolerations: []
        resources:
          - name: nvidia.com/gpu
            nominalQuota: "8"
          - name: tau.azure.com/torchrun-host-slot
            nominalQuota: "1"

See Also

• CLI reference: tau cluster
• Install TauGrid
• Source: charts/taugrid/values.yaml