managedClusters
Summary
Details
Deploy AKS cluster across availability zones
Impact: High Category: High Availability
APRL GUID: 4f63619f-5001-439c-bacb-8de891287727
Description:
Azure Availability Zones ensure high availability by offering independent locations within regions, equipped with their own power, cooling, and networking to ensure applications and data are protected from datacenter-level failures.
Potential Benefits:
Enhanced fault tolerance for AKS
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Returns AKS clusters that do not have any availability zones enabled or only use a single zone
resources
| where type =~ "Microsoft.ContainerService/managedClusters"
| where location in~ ("australiaeast", "brazilsouth", "canadacentral", "centralindia", "centralus", "eastasia", "eastus", "eastus2", "francecentral", "germanywestcentral", "israelcentral", "italynorth", "japaneast", "japanwest", "koreacentral", "mexicocentral", "newzealandnorth", "northeurope", "norwayeast", "polandcentral", "qatarcentral", "southafricanorth", "southcentralus", "southeastasia", "spaincentral", "swedencentral", "switzerlandnorth", "uaenorth", "uksouth", "westeurope", "westus2", "westus3", "usgovvirginia", "chinanorth3")
| project id, name, tags, location, pools = properties.agentPoolProfiles
| mv-expand pool = pools
| extend
numOfAvailabilityZones = iif(isnull(pool.availabilityZones), 0, array_length(pool.availabilityZones))
| where numOfAvailabilityZones < 2
| project
recommendationId = "4f63619f-5001-439c-bacb-8de891287727",
id,
name,
tags,
param1 = strcat("NodePoolName: ", pool.name),
param2 = strcat("Mode: ", pool.mode),
param3 = strcat("AvailabilityZones: ", iif(numOfAvailabilityZones == 0, "None", strcat("Zone ", strcat_array(pool.availabilityZones, ", ")))),
param4 = strcat("Location: ", location)
Isolate system and application pods
Impact: High Category: High Availability
APRL GUID: 5ee083cd-6ac3-4a83-8913-9549dd36cf56
Description:
AKS assigns the kubernetes.azure.com/mode: system label to nodes in system node pools signaling the preference for system pods should be scheduled there. The CriticalAddonsOnly=true:NoSchedule taint can be added to your system nodes to prohibit application pods from being scheduled on them.
Potential Benefits:
Enhanced reliability via pod isolation
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Returns each AKS cluster with nodepools that do not have system pods labelled with CriticalAddonsOnly
resources
| where type == "microsoft.containerservice/managedclusters"
| mv-expand agentPoolProfile = properties.agentPoolProfiles
| where agentPoolProfile.mode =~ 'System' // system node pools
| extend taint = tostring(parse_json(agentPoolProfile.nodeTaints))
| extend hasCriticalAddonsTaint = agentPoolProfile.kubeletConfig has 'CriticalAddonsOnly'
| extend hasNodeLabel = agentPoolProfile.customNodeLabels has 'CriticalAddonsOnly'
| extend hasCriticalAddonsOnly = hasCriticalAddonsTaint or hasNodeLabel or isempty(taint)
| extend nodePool = tostring(parse_json(agentPoolProfile.name))
| where hasCriticalAddonsOnly
| project
recommendationId="5ee083cd-6ac3-4a83-8913-9549dd36cf56",
id,
name,
tags,
param1=strcat("nodepoolName: ", nodePool)
Disable local accounts
Impact: High Category: Security
APRL GUID: ca324d71-54b0-4a3e-b9e4-10e767daa9fc
Description:
Local Kubernetes accounts in AKS, being non-auditable and legacy, are discouraged. Microsoft Entra's integration offers centralized management, multi-factor authentication, RBAC for detailed access, and a secure, scalable authentication system compatible with Azure and external identity providers.
Potential Benefits:
Enhanced security and access control
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Returns a list of AKS clusters not using AAD enabled
resources
| where type == "microsoft.containerservice/managedclusters"
| extend aadProfile = tostring (parse_json(properties.aadProfile))
| extend disablelocalAdmin = tostring(parse_json(properties.disableLocalAccounts))
| extend RBAC = tostring(parse_json(properties.enableRBAC))
| where RBAC == "false"
| project recommendationId="ca324d71-54b0-4a3e-b9e4-10e767daa9fc", name, id, tags, param1=strcat("aadProfile: ", aadProfile), param2=strcat("disablelocalAdmin: ",disablelocalAdmin), param3=strcat("RBAC: ", RBAC)
Configure Azure CNI networking for dynamic allocation of IPs
Impact: Medium Category: Scalability
APRL GUID: c22db132-399b-4e7c-995d-577a60881be8
Description:
Azure CNI enhances cluster IP and network management, allowing dynamic IP allocation, scalable subnets, direct pod-VNET connectivity, and supports diverse network policies for pods and nodes with Azure Network Policies and Calico, optimizing network efficiency and security
Potential Benefits:
Dynamic IP allocation, scalable subnets, direct VNET access
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Check AKS Clusters using kubenet network profile
resources
| where type == "microsoft.containerservice/managedclusters"
| extend networkProfile = tostring (parse_json(properties.networkProfile.networkPlugin))
| where networkProfile =="kubenet"
| project recommendationId="c22db132-399b-4e7c-995d-577a60881be8", name, id, tags, param1=strcat("networkProfile :",networkProfile)
Enable the cluster auto-scaler on an existing cluster
Impact: High Category: Scalability
APRL GUID: 902c82ff-4910-4b61-942d-0d6ef7f39b67
Description:
The cluster auto-scaler in AKS adjusts node counts based on pod resource needs and available capacity, enabling scaling as per demand to prevent outages.
Potential Benefits:
Optimizes scaling and prevents outages
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Find AKS clusters with auto-scaling disabled
Resources
| where type == "microsoft.containerservice/managedclusters"
| extend autoScaling = tostring (parse_json(properties.agentPoolProfiles.[0].enableAutoScaling))
| where autoScaling == "false"
| project recommendationId="902c82ff-4910-4b61-942d-0d6ef7f39b67", name, id, tags, param1=strcat("autoScaling :", autoScaling)
Back up Azure Kubernetes Service
Impact: Low Category: Disaster Recovery
APRL GUID: 269a9f1a-6675-460a-831e-b05a887a8c4b
Description:
AKS, popular for stateful apps needing backups, can now use Azure Backup to secure clusters and attached volumes through an installed Backup Extension, enabling backup and restore operations via a Backup Vault.
Potential Benefits:
Ensures data safety for AKS
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// Azure Resource Graph Query
// Find AKS clusters that do not have backup enabled
resources
| where type =~ 'Microsoft.ContainerService/managedClusters'
| extend lname = tolower(name)
| join kind=leftouter(recoveryservicesresources
| where type =~ 'microsoft.dataprotection/backupvaults/backupinstances'
| extend lname = tolower(tostring(split(properties.dataSourceInfo.resourceID, '/')[8]))
| extend protectionState = properties.currentProtectionState
| project lname, protectionState) on lname
| where protectionState != 'ProtectionConfigured'
| extend param1 = iif(isnull(protectionState), 'Protection Not Configured', strcat('Protection State: ', protectionState))
| project recommendationId = "269a9f1a-6675-460a-831e-b05a887a8c4b", name, id, tags, param1
Use zone-redundant storage for persistent volumes when running multi-zone AKS
Impact: Medium Category: High Availability
APRL GUID: d3111036-355d-431b-ab49-8ddad042800b
Description:
ZRS ensures data replication across three zones, protecting against zonal outages. It's available for Azure Disks, Container Storage, Files, and Blob by setting the SKU to ZRS in storage classes, enhancing multi-zone AKS clusters from v1.29.
Potential Benefits:
Increases data durability and availability
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// cannot-be-validated-with-arg
Upgrade Persistent Volumes using in-tree drivers to Azure CSI drivers
Impact: High Category: Governance
APRL GUID: b002c030-72e6-4a37-8217-1cb276c43169
Description:
From Kubernetes 1.26, Azure Disk and Azure File in-tree drivers are deprecated in favor of CSI drivers. Existing deployments remain operational but untested; users should switch to CSI drivers for new features and SKUs.
Potential Benefits:
Ensures future compatibility
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// cannot-be-validated-with-arg
Implement Resource Quota to ensure that Kubernetes resources do not exceed hard resource limits
Impact: Low Category: Scalability
APRL GUID: 9a1c17e5-c9a0-43db-b920-adaf54d1bcb7
Description:
A ResourceQuota object sets limits on resource use per namespace, controlling the number and type of objects created, and the total compute resources available.
Potential Benefits:
Limits AKS resource usage per namespace
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// cannot-be-validated-with-arg
Attach Virtual Nodes (ACI) to the AKS cluster
Impact: Low Category: Scalability
APRL GUID: b4639ca7-6308-429a-8b98-92f0bf9bf813
Description:
To rapidly scale AKS workloads, utilize virtual nodes for quick pod provisioning, unlike Kubernetes auto-scaler. For clusters with availability zones, ensure one nodepool per AZ due to persistent volumes not working across AZs, preventing auto-scaler pod creation failures if lacking access.
Potential Benefits:
Faster scaling with virtual nodes
Learn More:
ARG Query:
Click the Azure Resource Graph tab to view the query
// cannot-be-validated-with-arg