Run config reference

This page documents the direct run config: the normal, hand-written tau.yaml that tau run --config reads to submit a workload. See direct run config vs. managed workflow manifest for how this differs from the SDK-generated, schema_version: 1 manifest that tau.train/tau.serve render. Both can use a tau.yaml filename; inspect the schema, not the filename. Use the managed workflow shape only when TauGrid needs to own workflow semantics such as staged train/eval lineage.

Normal research projects check in direct config:

name: train
engine: rayjob
entrypoint: train.py

runtime:
  image: <pinned-image>

compute:
  workers: 2
  gpus_per_worker: 8

storage:
  data_pvc: training-data
  output: /data/checkpoints/workflows/train

storage.data_pvc names an existing Bound PVC in the resolved workload namespace. TauGrid references and mounts that claim; the platform owner provisions and owns the PVC, StorageClass, CSI configuration, and backing storage, choosing the backend and managing its lifecycle. The compatible CSI node component must register on every node eligible for the selected workload profile. A Bound claim alone does not prove that pods can mount it.

TauGrid’s PVC fields are Kubernetes abstractions, not a promise that a cloud provider’s managed CSI addon supports every node type. In particular, AKS-managed Azure CSI addons are expected on normal AKS agent pools but are not supported by TauGrid as the storage implementation for Flex or other externally registered nodes. TauGrid helps operators deploy and validate a compatible deployment-managed driver for those nodes. See AKS storage drivers and Flex nodes.

When durable output, checkpoint recovery, and metrics/artifact offload are not needed, omit storage.data_pvc. TauGrid still provides the pod-local /mnt emptyDir scratch path through TAU_HOT_DIR; its contents disappear with the pod.

Jobs without workload profiles

When TauCluster.spec.queues.capacityTeamRef selects your workspace’s team and spec.workloadProfiles is empty, a connected team-linked workspace supports direct batch Jobs without a preset:

name: train
engine: job
entrypoint: train.py
runtime:
  image: <pinned-image>
compute:
  gpus: 1
  cpu_request: "1"
  memory_request: 1Gi

Run tau run --config tau.yaml from a repository connected to the workspace. compute.gpus is required; set it to 0 for a CPU Job. The workspace and cluster must be Ready with current observed generations, the cluster must use Reconcile mode, and routing must use the workspace’s managed LocalQueue and ClusterQueue. Jobs remain suspended until Kueue admits them against the shared team pool; GPU requests and limits equal the explicit count. CPU and memory retain their normal explicit-resource validation. No workload-profile identity is stamped. Placement defaults to unconstrained for the pool’s topology-aware flavors; an explicit policy.topology is preserved. CPU-only Jobs use the same GPU-node pool and tolerate its GPU-node taints without requesting a GPU.

This path does not accept policy.shape, replace an explicitly requested policy.profile, or bypass a nonempty profile catalog. RayJobs and managed workflows still require a ready workload profile. Client dry-run still reads live workspace and cluster policy; offline profile snapshots remain a separate client-dry-run-only contract.

Direct batch Jobs can stage an immutable reference directory from another image, skipping ConfigMap creation:

engine: job
storage:
  image_assets:
    - name: pinned-reference-assets
      image: <registry>/<repository>@sha256:<64-hex-digest>
      source_path: /opt/source-assets
      mount_path: /opt/reference

TauGrid renders each asset as a pinned init-container image that runs /bin/cp -a into an emptyDir, then mounts that volume read-only in the main container. Source images must provide /bin/cp. Names and paths are validated, TauGrid-reserved paths cannot be replaced, and mutable image tags are rejected. storage.image_assets is intentionally limited to direct engine: job configs; managed workflows and RayJob configs reject it.

Direct Jobs can also stage their complete source tree from an immutable OCI image:

name: source-backed-job
engine: job
entrypoint: experiments/train.py
run:
  ttl_seconds_after_finished: 3600
  source:
    image: <registry>/<repository>@sha256:<64-hex-digest>
    path: /workspace

TauGrid copies run.source.path into an emptyDir with an init container, mounts the per-pod working copy at /tau/source, and runs entrypoint relative to that directory, keeping source bytes out of environment variables and ConfigMaps entirely. The source image must provide /bin/sh, cp, and chmod; mutable tags, absolute entrypoints, RayJob dispatch, managed workflows, and combining run.source with run.working_dir are rejected before rendering. The init container normalizes working-copy permissions and sets the executable bit on the entrypoint before a potentially non-root workload container starts. Build and push the source image once, then reuse its digest in every run config. Kubernetes uses the workload’s configured private-registry authentication; do not put registry credentials or signed download URLs in the config.

Direct Jobs can set the main container’s initial working directory directly, skipping shell-specific cd logic in the entrypoint:

engine: job
entrypoint: train.py
runtime:
  image: <pinned-image>
  working_dir: /workspace/project

runtime.working_dir must be a clean absolute path inside the image. TauGrid passes it straight through as Kubernetes container.workingDir, relaying the configured path unparsed; the image owns creating that directory and providing its own local files. It is Job-only and cannot be combined with run.source. Ray’s run.working_dir remains a different, host-relative field that packages a local project into Ray runtime_env.

For example, build a source-only image whose /workspace contains the checked out tree, push it to the platform’s private registry, and resolve the immutable digest before generating run configs:

docker buildx build --push -t "$SOURCE_REPOSITORY:$GIT_SHA" -f Dockerfile.source .
SOURCE_DIGEST="$(docker buildx imagetools inspect \
  "$SOURCE_REPOSITORY:$GIT_SHA" --format '{{.Manifest.Digest}}')"
printf '%s@%s\n' "$SOURCE_REPOSITORY" "$SOURCE_DIGEST"

run.ttl_seconds_after_finished is an optional retention period from 1 through 2,147,483,647 seconds for a completed or failed direct Job. It maps to Kubernetes spec.ttlSecondsAfterFinished; omission keeps TauGrid’s 28800-second default. The TTL starts only after every regular container has finished running.

Literal environment values are limited to 64 KiB each and 128 KiB in aggregate before workload creation. TauGrid’s generated embedded-payload environment entries are also capped at 64 KiB. Use run.source or storage.image_assets for content instead of embedding archives in runtime.env.

Main field groups:

Validate the installed contract. --config always names an explicit file; tau run itself instead takes an optional positional TARGET (for example tau run train) that resolves tau/train.yaml; the two serve different purposes, so keep validating against an explicit path:

tau run validate --config tau.yaml
tau run schema -o json
tau run explain-config

For the full submit-to-evidence loop against a named target, see first run.

The installed CLI is the final schema authority: use tau run schema.

Workload priority

policy.priority_tier selects one of the TauGrid-owned priority pairs:

The compatibility field policy.priority is an alias. A run-level tier can override the priority pair supplied by a selected workload profile; the profile remains authoritative for queue, resources, topology, and applicability. Do not combine a tier with policy.workload_priority_class, policy.pod_priority_class, or policy.disable_default_priorities.

The exact-class fields remain available for operator-managed compatibility, but ordinary consumers should use the built-in tiers rather than creating priority classes. Kueue admission priority controls queue ordering and workload preemption. Pod priority controls Kubernetes scheduling and pod preemption after admission; neither proves that an admitted workload is currently running.

Restricted pod security

Set runtime.security.mode: restricted when the workload images can run as non-root:

runtime:
  image: <pinned-image>
  security:
    mode: restricted

TauGrid applies the Kubernetes Restricted Pod Security fields to the Job or RayJob pod and every generated main, sidecar, and init container. Missing container user and group IDs default to numeric 65532; explicit nonzero IDs are preserved. Rendering fails if a profile requests root, privileged mode, privilege escalation, or added capabilities, and the image must support the selected non-root identity.

Config identity

TauGrid records one hash of the validated direct config on submitted Jobs and RayJobs. The hash covers runtime, resources, environment, storage, and other run behavior, so resume can warn when the config changed. Script and packaged-source content use separate payload-digest annotations.