Azure ML Environments are used to define the containers where your code will run. In the simplest case you can add custom Python libraries using pip, Conda or directly via the Azure ML Python SDK. If more customization is necessary you can use custom docker images.
This page provides examples creating environments:
- From pip
- From Conda
- Directly via the Azure ML Python SDK
- From custom Docker image
Create Environment from pip
Create Environment from Conda
CondaDependencies class to create a Python environment in directly with the Azure ML
To create an
Environment from a custom docker image:
For example Azure Container Registry addresses are of the form
Never check in passwords. In this example we provide the password via an environment variable.
To create an
Environment from a dockerfile:
user_managed_dependencies = True: You are responsible for installing all necessary Python libraries, typically in your docker image.
interpreter_path: Only used when
user_managed_dependencies=Trueand sets the Python interpreter path (e.g.
It is possible to have Azure ML manage your Python installation when providing a custom base image. For example, using pip
Note. In this case Python libraries installed in
Dockerfile will not be available.
We strongly recommend building your docker image from one of the Azure ML base images available here: AzureML-Containers GitHub Repo - like this:
These images come configured with all the requirements to run on Azure ML.
If user wants to build from scratch, here are a list of requirements and recommendations to keep in mind:
- Conda: Azure ML uses Conda to manage python environments by default. If you intend to allow Azure ML to manage the Python environment, Conda is required.
- libfuse: Required when using
- Openmpi: Required for distributed runs
- nvidia/cuda: (Recommended) For GPU-based training build image from nvidia/cuda
- Mellanox OFED user space drivers (Recommend) For SKUs with Infiniband
We suggest users to look at the dockerfiles of Azure ML base images as references.
Azure ML can use a custom image from a private registry as long as login information are provided.
Register an environment
env: Environment to your workspace
ws to reuse/share with your team.
Registered environments can be obtained directly from the workspace handle
This dictionary contains custom environments that have been registered to the workspace as well as a collection of curated environments maintained by Azure ML.
Save an environment to a local directory:
This will generate a directory with two (human-understandable and editable) files:
azureml_environment.json: Metadata including name, version, environment variables and Python and Docker configuration
conda_dependencies.yml: Standard conda dependencies YAML (for more details see Conda docs).
Load this environment later with
To set environment variables use the
environment_variables: Dict[str, str] attribute. Environment variables
are set on the process where the user script is executed.
When the conda dependencies are managed by Azure ML (
user_managed_dependencies=False, by default), Azure ML will check whether the same environment has already been materialized into a docker image in the Azure Container Registry associated with the Azure ML workspace. If it is a new environment, Azure ML will have a job preparation stage to build a new docker image for the new environment. You will see a image build log file in the logs and monitor the image build progress. The job won't start until the image is built and pushed to the container registry.
This image building process can take some time and delay your job start. To avoid unnecessary image building, consider:
- Register an environment that contains most packages you need and reuse when possible.
- If you only need a few extra packages on top of an existing environment,
- If the existing environment is a docker image, use a dockerfile from this docker image so you only need to add one layer to install a few extra packagers.
- Install extra python packages in your user script so the package installation happens in the script run as part of your code instead of asking Azure ML to treat them as part of a new environment. Consider using a setup script.
Due to intricacy of the python package dependencies and potential version conflict, we recommend use of custom docker image and dockerfiles (based on Azure ML base images) to manage your own python environment. This practice not only gives users full transparency of the environment, but also saves image building times at agile development stage.
If you have docker installed locally, you can build the docker image from Azure ML environment locally with option to push the image to workspace ACR directly. This is recommended when users are iterating on the dockerfile since local build can utilize cached layers.
It can be useful to invoke a
bootstrap.sh script for faster development. One typical example
would be to modify the Python installation at runtime to avoid frequent image rebuilding.
This can be done quite simply with commands. First set up your
To have this run ahead of your training script
train.py make use of the command:
See Running Code in the Cloud for more details on
In some cases you may wish to run certain parts of your
on certain ranks in a distributed setup. This can be achieved with a little care
This script will wait for local rank 0 (
$AZ_BATCHAI_TASK_INDEX) to create its
before the other processes continue.
Each Azure workspace comes with a keyvault (you can find this in the Azure Portal under the same resource group as your Workspace).
This can be used both to get and set secrets:
Of course you can also make use of other keyvaults you might have in Azure.
Be sure to add
azure-keyvault to your projects requirements in