Developer concepts on TauGrid

TauGrid object model

Mon, 01 Jan 0001 00:00:00 +0000

Full definitions for these terms live in the glossary. This page only explains how they relate and who owns what.

Concept	Owns
Repository	Code and history in one Git worktree
Project	One or more targets and a workspace connection
Workspace	Destination policy: cluster, namespace, queue, priority, output root, identity
Target	One checked-in runnable config
Run	One execution plus its lifecycle handle
Workload	Rendered Job or RayJob execution intent
Service	Online RayService or Deployment lifecycle target
Experiment	A comparison set over runs, metrics, and artifacts

The ownership split is deliberate:

Run and workload lifecycle

Mon, 01 Jan 0001 00:00:00 +0000

A TauGrid run crosses independent control planes:

Resolution: TauGrid finds the project, workspace, and target.
Rendering: TauGrid validates intent and builds a Job or RayJob.
Admission: Kueue reserves quota and admits the workload.
Scheduling: Kubernetes places pods on matching nodes and devices.
Runtime: Ray or the Job controller starts the application.
Progress: The model or data process advances and writes evidence.
Completion: Results, checkpoints, metrics, and terminal state remain inspectable.

These states are not equivalent:

Workspaces

Mon, 01 Jan 0001 00:00:00 +0000

A workspace connects repository intent to one cluster namespace, LocalQueue, workload ServiceAccount, and authorization contract.

Single active workspace

TauGrid v0 activates one workspace per cluster. The CLI resolves that workspace from the checked-in connection descriptor or the cluster’s primary-workspace marker, so researchers can omit the workspace name for normal runs.

The active workspace must be Ready before TauGrid submits a workload. Readiness confirms the namespace, queue, workload identity configuration, and authorization contract described in the TauWorkspace reference.

Configuration resolution

Mon, 01 Jan 0001 00:00:00 +0000

The normal researcher contract is a checked-in direct run config:

name: train
engine: rayjob
entrypoint: train.py

compute:
 workers: 2
 gpus_per_worker: 8

storage:
 data_pvc: training-data

TauGrid combines:

Explicit project and target selection.
Repository or monorepo discovery.
The project’s workspace connection descriptor.
Platform-owned workspace defaults.
Checked-in workload intent.
Temporary explicit operator overrides.

Ambiguity is an error. Dry-run output must keep default sources visible.

tau run validate --config tau/train.yaml
tau run train --dry-run=client
tau run explain-config

See direct run config vs. managed workflow manifest for how Python SDK-generated manifests relate to this default format.

Experiment evidence and artifacts

Mon, 01 Jan 0001 00:00:00 +0000

Every run should save enough information for you to answer four questions:

What code and settings produced this result?
Did the model improve during the run?
Where are the checkpoint and output files?
Can I compare, reproduce, or resume this run later?

What TauGrid saves

Saved data	Examples	Where it is saved
Run details	Project, run ID, config, and input references	TauGrid experiment index
Metrics over time	Loss, accuracy, throughput, and GPU use	Metric files
Metric summary	First and latest step, minimum, maximum, and point count	TauGrid experiment index
Output files	Checkpoints, models, images, tables, profiles, and reports	The directory set by `storage.output`
Run history	Attempts, retries, resumes, and final status	TauGrid experiment index and Kubernetes workload

The output files stay on the storage chosen by the platform team. TauGrid keeps an index so you can find and compare them.