Alert Threshold Override
The Alert Threshold Override feature, available with release 2024-09-05, allows both Greenfield and Brownfield customers to override alert threshold for specific resources during or after the deployment of AMBA-ALZ. Thanks to this new feature, it’s now possible to use a tag with specific name and value, to override the default alert threshold for specific resources. The new value will be used, only for the tagged resources, in place of the global one coming from the parameter file.
This feature is only available for metrics and log-search alerts, since Activity Log based alerts do not use threshold and, as such, cannot benefits from this new enhancement. Using the feature is easy: customers need to create a resource tag with a specific name and assign a value of their choice. Once this release is deployed, tags can be created either before or after the execution of remediation task. However, the feature behavior differs between Metric and Log-search alerts.
For metric alerts, if tags are configured before the remediation tasks execution, corresponding alerts (which are resource-specific) will be created using different thresholds for the same resource type:
If the tags are configured after the remediation task have completed, given the tag being part of the compliance criteria, the resource will be marked as not compliant, as such customers will just need to remediate the corresponding policy initiative(s) as documented at Remediate Policies to reconfigure exiting alerts with the new threshold.
Considering the different nature of log-search alerts where resource information is retrieved at query runtime, it does not make any difference if the tags are configured before or after the remediation task execution. The log-search alert query is created with a placeholder containing the threshold passed by the parameter file and with a logic to look at the resource-specific override tag, thanks to the ability to Correlate data in Azure Data Explorer and Azure Resource Graph with data in a Log Analytics workspace. If the specific override tag name is present, the query will use the tag value as new threshold, otherwise it will use the default one passed through the parameter file:
To work correctly, this feature needs to look at specific tag names. Unfortunately it is not possible to allow for more flexibility in tag name in this case. Tag names have been defined, according to the following naming convention:
Mapping between resource type friendly name and resource provider namespace (together with the recommended abbreviation) can be found at Abbreviation recommendations for Azure resources
***_amba-<metricName/counterName>-threshold-override_***
There might be cases where for the same resource, the same metric is used more than one. In this scenario, we implemented a differentiator value inserted right after the metric name, making the naming convention resampling the following format:
***_amba-<metricName/counterName>-<differentiator>-threshold-override_***
The following table contains the mapping between the alert name and the corresponding tag value to be created:
Resource Type | Alert Name | Alert Type | Override Tag name |
---|---|---|---|
Machine - Azure Arc | subscription().displayName -HybridVMHighDataDiskReadLatencyAlert | Log search | _amba-ReadLatencyMs-Data-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMLowDataDiskSpaceAlert | Log search | _amba-FreeSpacePercentage-Data-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighDataDiskWriteLatencyAlert | Log search | _amba-WriteLatencyMs-Data-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMDisconnectedAlert | Log search | _amba-Disconnected-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHeartBeatAlert | Log search | _amba-Heartbeat-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighNetworkInAlert | Log search | _amba-ReadBytesPerSecond-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighNetworkOutAlert | Log search | _amba-WriteBytesPerSecond-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighOSDiskReadLatencyAlert | Log search | _amba-ReadLatencyMs-OS-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMLowOSDiskSpaceAlert | Log search | _amba-FreeSpacePercentage-OS-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighOSDiskWriteLatencyAlert | Log search | _amba-WriteLatencyMs-OS-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMHighCPUAlert | Log search | _amba-UtilizationPercentage-threshold-override_ |
Machine - Azure Arc | subscription().displayName -HybridVMLowMemoryAlert | Log search | _amba-AvailableMemoryPercentage-threshold-override_ |
Virtual machine | subscription().displayName -VMHighDataDiskReadLatencyAlert | Log search | _amba-ReadLatencyMs-Data-threshold-override_ |
Virtual machine | subscription().displayName -VMLowDataDiskSpaceAlert | Log search | _amba-FreeSpacePercentage-Data-threshold-override_ |
Virtual machine | subscription().displayName -VMHighDataDiskWriteLatencyAlert | Log search | _amba-WriteLatencyMs-Data-threshold-override_ |
Virtual machine | subscription().displayName -VMHeartBeatAlert | Log search | _amba-Heartbeat-threshold-override_ |
Virtual machine | subscription().displayName -VMHighNetworkInAlert | Log search | _amba-ReadBytesPerSecond-threshold-override_ |
Virtual machine | subscription().displayName -VMHighNetworkOutAlert | Log search | _amba-WriteBytesPerSecond-threshold-override_ |
Virtual machine | subscription().displayName -VMHighOSDiskReadLatencyAlert | Log search | _amba-ReadLatencyMs-OS-threshold-override_ |
Virtual machine | subscription().displayName -VMLowOSDiskSpaceAlert | Log search | _amba-FreeSpacePercentage-OS-threshold-override_ |
Virtual machine | subscription().displayName -VMHighOSDiskWriteLatencyAlert | Log search | _amba-WriteLatencyMs-OS-threshold-override_ |
Virtual machine | subscription().displayName -VMHighCPUAlert | Log search | _amba-UtilizationPercentage-threshold-override_ |
Virtual machine | subscription().displayName -VMLowMemoryAlert | Log search | _amba-AvailableMemoryPercentage-threshold-override_ |
Log Analytics workspace | resourceName -DailyCapLimitReachedAlert | Log search | Not available since threshold will always be 0 |
Resource Type | Alert Name | Alert Type | Override Tag name |
---|---|---|---|
Virtual machine | resourceName -AvailableMemoryAlert | Metrics | _amba-AvailableMemoryBytes-threshold-override_ |
Automation Account | resourceName -TotalJob | Metrics | _amba-TotalJob-threshold-override_ |
Front Door and CDN profile | resourceName -OriginHealthPercentage | Metrics | _amba-OriginHealthPercentage-threshold-override_ |
Front Door and CDN profile | resourceName -OriginLatencyAlert | Metrics | Not available since it uses dynamic thresholds |
Front Door and CDN profile | resourceName -Percentage4XXAlert | Metrics | Not available since it uses dynamic thresholds |
Front Door and CDN profile | resourceName -Percentage5XXAlert | Metrics | Not available since it uses dynamic thresholds |
Key vault | ActivityKeyVaultDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Key vault | resourceName -Availability | Metrics | _amba-Availability-threshold-override_ |
Key vault | resourceName -CapacityAlert | Metrics | _amba-SaturationShoebox-threshold-override_ |
Key vault | resourceName -LatencyAlert | Metrics | _amba-ServiceApiLatency-threshold-override_ |
Key vault | resourceName -RequestsAlert | Metrics | Not available since it uses dynamic thresholds |
Azure Key Vault Managed HSM | ActivityManagedHSMDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Azure Key Vault Managed HSM | resourceName -Availability | Metrics | _amba-Availability-threshold-override_ |
Azure Key Vault Managed HSM | resourceName -LatencyAlert | Metrics | _amba-ServiceApiLatency-threshold-override_ |
Application gateway | resourceName -agApplicationGatewayTotalTime | Metrics | Not available since it uses dynamic thresholds |
Application gateway | resourceName -agBackendLastByteResponseTime | Metrics | Not available since it uses dynamic thresholds |
Application gateway | resourceName -agCapacityUnits | Metrics | _amba-CapacityUnits-threshold-override_ |
Application gateway | resourceName -agComputeUnits | Metrics | _amba-ComputeUnits-threshold-override_ |
Application gateway | resourceName -agCpuUtilization | Metrics | _amba-CpuUtilization-threshold-override_ |
Application gateway | resourceName -agFailedRequests | Metrics | Not available since it uses dynamic thresholds |
Application gateway | resourceName -agResponseStatus | Metrics | Not available since it uses dynamic thresholds |
Application gateway | resourceName -agUnhealthyHostCount | Metrics | _amba-UnhealthyHostCount-threshold-override_ |
Firewall | ActivityAzureFirewallDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Firewall | resourceName -FirewallHealth | Metrics | _amba-FirewallHealth-threshold-override_ |
Firewall | resourceName -SNATPortUtilization | Metrics | _amba-SNATPortUtilization-threshold-override_ |
ExpressRoute circuit | resourceName -ArpAvailability | Metrics | _amba-ArpAvailability-threshold-override_ |
ExpressRoute circuit | resourceName -BgpAvailability | Metrics | _amba-BgpAvailability-threshold-override_ |
ExpressRoute circuit | resourceName -QosDropBitsInPerSecond | Metrics | Not available since it uses dynamic thresholds |
ExpressRoute circuit | resourceName -QosDropBitsOutPerSecond | Metrics | Not available since it uses dynamic thresholds |
ExpressRoute gateway | resourceName -GatewayERBitsInAlert | Metrics | _amba-ERGatewayConnectionBitsInPerSecond-threshold-override_ |
ExpressRoute gateway | resourceName -GatewayERBitsOutAlert | Metrics | _amba-ERGatewayConnectionBitsOutPerSecond-threshold-override_ |
ExpressRoute gateway | resourceName -GatewayERCPUAlert | Metrics | _amba-ExpressRouteGatewayCpuUtilization-threshold-override_ |
ExpressRoute port | resourceName -DirectERBitsInAlert | Metrics | _amba-PortBitsInPerSecond-threshold-override_ |
ExpressRoute port | resourceName -DirectERBitsOutAlert | Metrics | _amba-PortBitsOutPerSecond-threshold-override_ |
ExpressRoute port | resourceName -DirectERLineProtocolAlert | Metrics | _amba-LineProtocol-threshold-override_ |
ExpressRoute port | resourceName -DirectERRxLightLevelHighAlert | Metrics | _amba-RxLightLevel-High-threshold-override_ |
ExpressRoute port | resourceName -DirectERRxLightLevelLowAlert | Metrics | _amba-RxLightLevel-Low-threshold-override_ |
ExpressRoute port | resourceName -DirectERTxLightLevelHighAlert | Metrics | _amba-TxLightLevel-High-threshold-override_ |
ExpressRoute port | resourceName -DirectERTxLightLevelLowAlert | Metrics | _amba-TxLightLevel-Low-threshold-override_ |
Front Door | resourceName -BackendHealthPercentage | Metrics | _amba-BackendHealthPercentage-threshold-override_ |
Front Door | resourceName -BackendRequestLatencyAlert | Metrics | Not available since it uses dynamic thresholds |
Load balancer | resourceName -ALBDataPathAvailability | Metrics | _amba-VipAvailability-threshold-override_ |
Load balancer | resourceName -ALBGlobalBackendAvailability | Metrics | _amba-GlobalBackendAvailability-threshold-override_ |
Load balancer | resourceName -ALBHealthProbeStatus | Metrics | _amba-DipAvailability-threshold-override_ |
Load balancer | resourceName -ALBUsedSNATPorts | Metrics | _amba-UsedSNATPorts-threshold-override_ |
Network security group | ActivityNSGDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Private DNS zone | resourceName -CapacityUtilizationAlert | Metrics | _amba-VirtualNetworkLinkCapacityUtilization-threshold-override_ |
Private DNS zone | resourceName -QueryVolumeAlert | Metrics | _amba-QueryVolume-threshold-override_ |
Private DNS zone | resourceName -RecordSet_Capacity_Utilization | Metrics | _amba-RecordSetCapacityUtilization-threshold-override_ |
Private DNS zone | resourceName -RequestsAlert | Metrics | _amba-VirtualNetworkWithRegistrationCapacityUtilization-threshold-override_ |
Public IP address | resourceName -BytesInDDOSAlert | Metrics | _amba-bytesinddos-threshold-override_ |
Public IP address | resourceName -DDOS_Attack | Metrics | _amba-ifunderddosattack-threshold-override_ |
Public IP address | resourceName -PacketsInDDosAlert | Metrics | _amba-PacketsInDDoS-threshold-override_ |
Public IP address | resourceName -VIPAvailabityAlert | Metrics | _amba-VipAvailability-threshold-override_ |
Route table | ActivityUDRUpdate | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Traffic Manager profile | resourceName -EndpointHealthAlert | Metrics | _amba-EndpointHealth-threshold-override_ |
Virtual network gateway | resourceName -TunnelBandwidthAlert | Metrics | _amba-TunnelAverageBandwidth-threshold-override_ |
Virtual network gateway | resourceName -TunnelEgressAlert | Metrics | _amba-TunnelEgressBytes-threshold-override_ |
Virtual network gateway | resourceName -TunnelEgressPacketDropCountAlert | Metrics | Not available since it uses dynamic thresholds |
Virtual network gateway | resourceName -TunnelEgressPacketDropTSMismatchAlert | Metrics | Not available since it uses dynamic thresholds |
Virtual network gateway | resourceName -GatewayERBitsAlert | Metrics | _amba-ExpressRouteGatewayBitsPerSecond-threshold-override_ |
Virtual network gateway | resourceName -GatewayERCPUAlert | Metrics | _amba-ExpressRouteGatewayCpuUtilization-threshold-override_ |
Virtual network gateway | resourceName -TunnelIngressAlert | Metrics | _amba-TunnelIngressBytes-threshold-override_ |
Virtual network gateway | resourceName -TunnelIngressPacketDropCountAlert | Metrics | Not available since it uses dynamic thresholds |
Virtual network gateway | resourceName -TunnelIngressPacketDropTSMismatchAlert | Metrics | Not available since it uses dynamic thresholds |
Virtual network | resourceName -DDOSAttackAlert | Metrics | _amba-ifunderddosattack-threshold-override_ |
VPN Gateway | ActivityVPNGatewayDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
VPN Gateway | resourceName -GatewayBandwidthAlert | Metrics | _amba-tunnelaveragebandwidth-threshold-override_ |
VPN Gateway | resourceName -BGPPeerStatusAlert | Metrics | _amba-bgppeerstatus-threshold-override_ |
VPN Gateway | resourceName -TunnelEgressAlert | Metrics | _amba-tunnelegressbytes-threshold-override_ |
VPN Gateway | resourceName -TunnelEgressPacketDropCountAlert | Metrics | Not available since it uses dynamic thresholds |
VPN Gateway | resourceName -TunnelEgressPacketDropTSMismatchAlert | Metrics | Not available since it uses dynamic thresholds |
VPN Gateway | resourceName -TunnelIngressAlert | Metrics | _amba-tunnelingressbytes-threshold-override_ |
VPN Gateway | resourceName -TunnelIngressPacketDropCount | Metrics | Not available since it uses dynamic thresholds |
VPN Gateway | resourceName -TunnelIngressPacketDropTSMismatchAlert | Metrics | Not available since it uses dynamic thresholds |
Log Analytics workspace | ActivityLAWorkspaceDelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Log Analytics workspace | ActivityLAWorkspaceRegenKey | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Subscription | ResourceHealthUnhealthyAlert | Resource health | Not available since Activity Log based alerts do not have thresholds |
Subscription | ServiceHealthHealth | Service health | Not available since Activity Log based alerts do not have thresholds |
Subscription | ServiceHealthIncident | Service health | Not available since Activity Log based alerts do not have thresholds |
Subscription | ServiceHealthMaintenance | Service health | Not available since Activity Log based alerts do not have thresholds |
Subscription | ServiceSecurityIncident | Service health | Not available since Activity Log based alerts do not have thresholds |
Storage account | ActivitySADelete | Activity Log | Not available since Activity Log based alerts do not have thresholds |
Storage account | resourceName -AvailabilityAlert | Metrics | _amba-Availability-threshold-override_ |
App Service plan | resourceName -CpuPercentage | Metrics | _amba-CpuPercentage-threshold-override_ |
App Service plan | resourceName -DiskQueueLengthAlert | Metrics | Not available since it uses dynamic thresholds |
App Service plan | resourceName -HttpQueueLengthAlert | Metrics | Not available since it uses dynamic thresholds |
App Service plan | resourceName -MemoryPercentage | Metrics | _amba-MemoryPercentage-threshold-override_ |