Authentication in Azure Service Operator v2
Azure Service Operator supports three different styles of authentication today.
- Service Principal
- azure-workload-identity authentication (OIDC + Managed Identity or Service Principal)
- [Deprecated] aad-pod-identity authentication (Managed Identity)
Service Principal
Prerequisites
- An existing Azure Service Principal.
To use Service Principal authentication, specify an aso-controller-settings
secret with the AZURE_CLIENT_ID
and AZURE_CLIENT_SECRET
keys set.
For more information about Service Principals, see creating an Azure Service Principal using the Azure CLI.
The AZURE_CLIENT_ID
is sometimes also called the App ID. The AZURE_CLIENT_SECRET
is the “password” returned by the command in the previously linked documentation.
Use the following Bash script to set the environment variables for the aso-controller-settings
secret:
export AZURE_CLIENT_ID="00000000-0000-0000-0000-00000000000" # The client ID (sometimes called App Id) of the Service Principal.
export AZURE_CLIENT_SECRET="00000000-0000-0000-0000-00000000000" # The client secret of the Service Principal.
export AZURE_SUBSCRIPTION_ID="00000000-0000-0000-0000-00000000000" # The Azure Subscription ID the identity is in.
export AZURE_TENANT_ID="00000000-0000-0000-0000-00000000000" # The Azure AAD Tenant the identity/subscription is associated with.
Create the aso-controller-settings
secret:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: aso-controller-settings
namespace: azureserviceoperator-system
stringData:
AZURE_SUBSCRIPTION_ID: "$AZURE_SUBSCRIPTION_ID"
AZURE_TENANT_ID: "$AZURE_TENANT_ID"
AZURE_CLIENT_ID: "$AZURE_CLIENT_ID"
AZURE_CLIENT_SECRET: "$AZURE_CLIENT_SECRET"
EOF
Azure Workload Identity
Prerequisites
- An existing Azure Service Principal or Managed Identity. The setup is the same regardless of which you choose.
- The Azure CLI.
Use the following Bash script to set the environment variables required for the below commands:
export AZURE_CLIENT_ID="00000000-0000-0000-0000-00000000000" # The client ID (sometimes called App Id) of the Service Principal, or the Client ID of the Managed Identity with which you are using Workload Identity.
export AZURE_SUBSCRIPTION_ID="00000000-0000-0000-0000-00000000000" # The Azure Subscription ID the identity is in.
export AZURE_TENANT_ID="00000000-0000-0000-0000-00000000000" # The Azure AAD Tenant the identity/subscription is associated with.
export SERVICE_ACCOUNT_ISSUER="https://oidc.prod-aks.azure.com/00000000-0000-0000-0000-00000000000/" # The OIDC endpoint for your cluster in this example AKS
Get the Application Object ID of the identity
For Managed Identity: export APPLICATION_OBJECT_ID=$(az resource show --id /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/{my resource group name}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{my managed identity name} --query "properties.principalId" -o tsv | tr -d '[:space:]')
Configure trust
Establish trust between your OIDC issuer URL and the backing Service Principal or Managed Identity. See how it works for details.
Service Principal
export APPLICATION_OBJECT_ID="$(az ad app show --id ${AZURE_CLIENT_ID} --query id -otsv)"
cat <<EOF > params.json
{
"name": "aso-federated-credential",
"issuer": "${SERVICE_ACCOUNT_ISSUER}",
"subject": "system:serviceaccount:azureserviceoperator-system:azureserviceoperator-default",
"description": "Kubernetes service account federated credential",
"audiences": [
"api://AzureADTokenExchange"
]
}
EOF
az ad app federated-credential create --id ${APPLICATION_OBJECT_ID} --parameters @params.json
Managed Identity
Set the following additional environment variables:
export MI_RESOURCE_GROUP="my-rg" # The resource group containing the managed identity that will be used by ASO
export MI_NAME="my-mi" # The name of the managed identity that will be used by ASO
cat <<EOF > body.json
{
"name": "aso-federated-credential",
"type":"Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials",
"properties": {
"issuer":"${SERVICE_ACCOUNT_ISSUER}",
"subject":"system:serviceaccount:azureserviceoperator-system:azureserviceoperator-default",
"audiences": [
"api://AzureADTokenExchange"
]
}
}
EOF
az rest --method put --url /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourcegroups/${MI_RESOURCE_GROUP}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/${MI_NAME}/federatedIdentityCredentials/aso-federated-credential?api-version=2022-01-31-preview --body @body.json
Set ASO to use the Service Account token
Helm
When installing ASO via Helm, set useWorkloadIdentityAuth
to true
.
helm upgrade --install --devel aso2 aso2/azure-service-operator \
--create-namespace \
--namespace=azureserviceoperator-system \
--set azureSubscriptionID=$AZURE_SUBSCRIPTION_ID \
--set azureTenantID=$AZURE_TENANT_ID \
--set azureClientID=$AZURE_CLIENT_ID \
--set useWorkloadIdentityAuth=true
Kubectl
Updating an existing deployment
If you installed ASO manually, you can update the existing Secret
to use Workload Identity authentication.
Update the aso-controller-settings
secret to have string data USE_WORKLOAD_IDENTITY_AUTH: "true"
Ensure that the aso-controller-settings
secret has the key USE_WORKLOAD_IDENTITY_AUTH
set to true
and restart the ASO pod.
New Deployment
Create the aso-controller-settings
secret:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: aso-controller-settings
namespace: azureserviceoperator-system
stringData:
AZURE_SUBSCRIPTION_ID: "$AZURE_SUBSCRIPTION_ID"
AZURE_TENANT_ID: "$AZURE_TENANT_ID"
AZURE_CLIENT_ID: "$AZURE_CLIENT_ID"
USE_WORKLOAD_IDENTITY_AUTH: "true"
EOF
[Deprecated] Managed Identity (aad-pod-identity)
This authentication mechanism still works but is deprecated. See Azure Workload Identity for the new way
Prerequisites
- An existing Azure Managed Identity.
- aad-pod-identity installed into your cluster. If you are running ASO on an Azure Kubernetes Service (AKS) cluster, you can instead use the integrated aad-pod-identity.
First, set the following environment variables:
export IDENTITY_RESOURCE_GROUP="myrg" # The resource group containing the managed identity.
export IDENTITY_NAME="myidentity" # The name of the identity.
export AZURE_SUBSCRIPTION_ID="00000000-0000-0000-0000-00000000000" # The Azure Subscription ID the identity is in.
export AZURE_TENANT_ID="00000000-0000-0000-0000-00000000000" # The Azure AAD Tenant the identity/subscription is associated with.
Use the az cli
to get some more details about the identity to use:
export IDENTITY_CLIENT_ID="$(az identity show -g ${IDENTITY_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query clientId -otsv)"
export IDENTITY_RESOURCE_ID="$(az identity show -g ${IDENTITY_RESOURCE_GROUP} -n ${IDENTITY_NAME} --query id -otsv)"
Manual Deploy
Deploy an AzureIdentity
:
cat <<EOF | kubectl apply -f -
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentity
metadata:
name: aso-identity
namespace: azureserviceoperator-system
spec:
type: 0
resourceID: ${IDENTITY_RESOURCE_ID}
clientID: ${IDENTITY_CLIENT_ID}
EOF
Deploy an AzureIdentityBinding
to bind this identity to the Azure Service Operator manager pod:
cat <<EOF | kubectl apply -f -
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentityBinding
metadata:
name: aso-identity-binding
namespace: azureserviceoperator-system
spec:
azureIdentity: aso-identity
selector: aso-manager-binding
EOF
Deploy the aso-controller-settings
secret, configured to use the identity:
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: aso-controller-settings
namespace: azureserviceoperator-system
stringData:
AZURE_SUBSCRIPTION_ID: "$AZURE_SUBSCRIPTION_ID"
AZURE_TENANT_ID: "$AZURE_TENANT_ID"
AZURE_CLIENT_ID: "$IDENTITY_CLIENT_ID"
EOF
Helm Chart Deploy
helm repo add aso2 https://raw.githubusercontent.com/Azure/azure-service-operator/main/v2/charts
helm repo update
helm upgrade --install --devel aso2 aso2/azure-service-operator \
--create-namespace \
--namespace=azureserviceoperator-system \
--set azureSubscriptionID=$AZURE_SUBSCRIPTION_ID \
--set aadPodIdentity.enable=true \
--set aadPodIdentity.azureManagedIdentityResourceId=${IDENTITY_RESOURCE_ID} \
--set azureClientID=${IDENTITY_CLIENT_ID}
Using a credential for ASO with reduced permissions
Most examples of installing ASO suggest using an identity that has Contributor access to the Subscription. Such broadly scoped access is not required to run the operator, it’s just the easiest way to set things up and so is often used in examples.
Here we discuss a few approaches to restricting the access that the ASO identity has.
Contributor access at a reduced scope
In some scenarios you may know that ASO will be operating on only certain resource groups. For example, if you have two
teams working in a cluster with ASO, “Marketing” and “Product”. If you know ahead of time that the only resource groups these
teams will use are marketing-dev
, marketing-prod
, product-dev
and product-prod
, you can pre-create the following resources:
- The 4 resource groups.
- An identity for use by ASO, assigned Contributor permission to each of the 4 resource groups.
Now install ASO following the instructions above. Once ASO is installed, adopt the existing resource groups in ASO by applying a YAML in the cluster
for each of the 4 resource groups. Now users of ASO are free to create resources in those resource groups as normal. If they attempt
to use ASO to create a new resource group foo
they will be rejected by Azure as the ASO identity doesn’t have permission to create
arbitrary resource groups in the Subscription.
Reduced access at Subscription scope
In other scenarios, you may want to give out reduced access at the Subscription scope. This can be done by determining what resources in ASO you will be using and creating a custom role scoped to just those permissions.
For example if you want to use ASO to manage ResourceGroups, Redis and MySQL flexible servers, but not anything else, you could
create a role similar to aso-operator.json
:
{
"Name": "ASO Operator",
"IsCustom": true,
"Description": "Role with access to perform only the operations which we allow ASO to perform",
"Actions": [
"Microsoft.Resources/subscriptions/resourceGroups/*",
"Microsoft.Cache/*",
"Microsoft.DBforMySQL/*"
],
"NotActions": [
],
"AssignableScopes": [
"/subscriptions/{subscriptionId1}",
"/subscriptions/{subscriptionId2}"
]
}
Then use the az cli to create that custom role definition: az role definition create --role-definition ~/aso-operator.json
See list of resource provider operations for a comprehensive list of operations.
Note: We strongly recommend giving full permissions to the resource types ASO will be managing. ASO needs read
, write
, and delete
permissions to a resource
to fully manage it. In some cases, it also needs action
permissions. It’s recommended to give *
permissions for a given resource type which ASO
will be managing.
Once the role has been created, assign it to the ASO identity and install the operator as normal. It will be able to manage any resource of the types you allowed. Any other resources will fail to be created/updated/deleted with a permissions error.