Core technologies on TauGrid

What is Ray?

Mon, 01 Jan 0001 00:00:00 +0000

Ray is an open-source framework for running Python work across processes and machines. Teams use it for distributed training, hyperparameter tuning, data processing, and model serving.

KubeRay brings Ray to Kubernetes. Its operator manages resources such as RayJob and RayService, creates the Ray head and worker pods, and follows the Ray workload lifecycle.

How TauGrid uses Ray

TauGrid turns a repository target into either a Kubernetes Job or a KubeRay RayJob. A Ray workload follows this path:

What is Kueue?

Mon, 01 Jan 0001 00:00:00 +0000

Kueue is a Kubernetes-native queue and admission system for batch and AI workloads. It decides when a workload may consume shared quota. After admission, the Kubernetes scheduler decides which nodes run the pods.

The main Kueue objects

A LocalQueue is the queue a workspace submits to.
A ClusterQueue combines quota across one or more LocalQueues.
A ResourceFlavor describes a class of capacity, such as CPU nodes or a particular GPU class.
A Workload records the resources a Job or RayJob requests and its admission state.

TauGrid resolves the workspace LocalQueue and adds it to the rendered workload. Kueue then evaluates quota, priority, queue order, and resource flavors.

What is adx-mon?

Mon, 01 Jan 0001 00:00:00 +0000

adx-mon is Azure Data Explorer Monitor, an observability pipeline for Kubernetes clusters. It collects metrics, logs, and GPU telemetry, batches the data, and sends it to Azure Data Explorer (ADX/Kusto).

Its main components collect node and cluster signals, ingest batches into ADX, manage ADX schema commands, and evaluate configured alerts. Platform teams choose the signals, databases, retention, identities, and access rules.

How TauGrid uses adx-mon

adx-mon is an optional platform integration. A TauGrid environment can use it to:

What is Stellar?

Mon, 01 Jan 0001 00:00:00 +0000

Stellar is TauGrid’s experiment tracking and comparison experience. It ships in the taugrid-portal binary and presents run details, scalar metric history, and side-by-side comparisons in a browser or terminal.

What Stellar helps answer

Which code and settings produced this run?
How did loss, accuracy, throughput, or another metric change over time?
Which run performed best?
Where are the model, checkpoint, and other output files?
Did the run finish, retry, or resume?

Where the data comes from

TauGrid saves complete metric files and a small experiment index close to the run. Stellar can read that saved evidence directly. A platform can also provide an ADX/Kusto view for searching and comparing runs across workspaces.