Represent how to deliver the dataset to a compute target.
dataset_consumption_config( name, dataset, mode = "direct", path_on_compute = NULL )
The name of the dataset in the run, which can be different to the registered name. The name will be registered as environment variable and can be used in data plane.
The dataset that will be consumed in the run.
Defines how the dataset should be delivered to the compute target. There are three modes:
'direct': consume the dataset as dataset. 'download': download the dataset and consume the dataset as downloaded path. 'mount': mount the dataset and consume the dataset as mount path.
The target path on the compute to make the data available at. The folder structure of the source data will be kept, however, we might add prefixes to this folder structure to avoid collision.
est <- estimator(source_directory = ".", entry_script = "train.R", inputs = list(dataset_consumption_config('mydataset', dataset, mode = 'download')), compute_target = compute_target)