Azure Monitor Baseline Alerts
Download AlertsGlossaryGitHubGitHub IssuesToggle Dark/Light/Auto modeToggle Dark/Light/Auto modeToggle Dark/Light/Auto modeBack to homepage

Alert Threshold Override

Overview

The Alert Threshold Override feature, available with release 2024-09-05, allows both Greenfield and Brownfield customers to override alert threshold for specific resources during or after the deployment of AMBA-ALZ. Thanks to this new feature, it’s now possible to use a tag with specific name and value, to override the default alert threshold for specific resources. The new value will be used, only for the tagged resources, in place of the global one coming from the parameter file.

How this feature works

This feature is only available for metrics and log-search alerts, since Activity Log based alerts do not use threshold and, as such, cannot benefits from this new enhancement. Using the feature is easy: customers need to create a resource tag with a specific name and assign a value of their choice. Once this release is deployed, tags can be created either before or after the execution of remediation task. However, the feature behavior differs between Metric and Log-search alerts.

Metrics alerts

For metric alerts, if tags are configured before the remediation tasks execution, corresponding alerts (which are resource-specific) will be created using different thresholds for the same resource type:

Metric Alerts - Override threshold at work

If the tags are configured after the remediation task have completed, given the tag being part of the compliance criteria, the resource will be marked as not compliant, as such customers will just need to remediate the corresponding policy initiative(s) as documented at Remediate Policies to reconfigure exiting alerts with the new threshold.

Log-search alerts

Considering the different nature of log-search alerts where resource information is retrieved at query runtime, it does not make any difference if the tags are configured before or after the remediation task execution. The log-search alert query is created with a placeholder containing the threshold passed by the parameter file and with a logic to look at the resource-specific override tag, thanks to the ability to Correlate data in Azure Data Explorer and Azure Resource Graph with data in a Log Analytics workspace. If the specific override tag name is present, the query will use the tag value as new threshold, otherwise it will use the default one passed through the parameter file:

Log-search Alerts - Override threshold at work

Which tag does customers need to create

To work correctly, this feature needs to look at specific tag names. Unfortunately it is not possible to allow for more flexibility in tag name in this case. Tag names have been defined, according to the following naming convention:

Mapping between resource type friendly name and resource provider namespace (together with the recommended abbreviation) can be found at Abbreviation recommendations for Azure resources

***_amba-<metricName/counterName>-threshold-Override_***

There might be cases where for the same resource, the same metric is used more than one. In this scenario, we implemented a differentiator value inserted right after the metric name, making the naming convention resampling the following format:

***_amba-<metricName/counterName>-<differentiator>-threshold-Override_***

The following table contains the mapping between the alert name and the corresponding tag value to be created:


Log-search alerts table

Resource TypeAlert NameAlert TypeOverride Tag name
Machine - Azure Arcsubscription().displayName-HybridVMHighDataDiskReadLatencyAlertLog search_amba-ReadLatencyMs-Data-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMLowDataDiskSpaceAlertLog search_amba-FreeSpacePercentage-Data-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighDataDiskWriteLatencyAlertLog search_amba-WriteLatencyMs-Data-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMDisconnectedAlertLog search_amba-Disconnected-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHeartBeatAlertLog search_amba-Heartbeat-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighNetworkInAlertLog search_amba-ReadBytesPerSecond-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighNetworkOutAlertLog search_amba-WriteBytesPerSecond-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighOSDiskReadLatencyAlertLog search_amba-ReadLatencyMs-OS-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMLowOSDiskSpaceAlertLog search_amba-FreeSpacePercentage-OS-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighOSDiskWriteLatencyAlertLog search_amba-WriteLatencyMs-OS-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMHighCPUAlertLog search_amba-UtilizationPercentage-threshold-Override_
Machine - Azure Arcsubscription().displayName-HybridVMLowMemoryAlertLog search_amba-AvailableMemoryPercentage-threshold-Override_
Virtual machinesubscription().displayName-VMHighDataDiskReadLatencyAlertLog search_amba-ReadLatencyMs-Data-threshold-Override_
Virtual machinesubscription().displayName-VMLowDataDiskSpaceAlertLog search_amba-FreeSpacePercentage-Data-threshold-Override_
Virtual machinesubscription().displayName-VMHighDataDiskWriteLatencyAlertLog search_amba-WriteLatencyMs-Data-threshold-Override_
Virtual machinesubscription().displayName-VMHeartBeatAlertLog search_amba-Heartbeat-threshold-Override_
Virtual machinesubscription().displayName-VMHighNetworkInAlertLog search_amba-ReadBytesPerSecond-threshold-Override_
Virtual machinesubscription().displayName-VMHighNetworkOutAlertLog search_amba-WriteBytesPerSecond-threshold-Override_
Virtual machinesubscription().displayName-VMHighOSDiskReadLatencyAlertLog search_amba-ReadLatencyMs-OS-threshold-Override_
Virtual machinesubscription().displayName-VMLowOSDiskSpaceAlertLog search_amba-FreeSpacePercentage-OS-threshold-Override_
Virtual machinesubscription().displayName-VMHighOSDiskWriteLatencyAlertLog search_amba-WriteLatencyMs-OS-threshold-Override_
Virtual machinesubscription().displayName-VMHighCPUAlertLog search_amba-UtilizationPercentage-threshold-Override_
Virtual machinesubscription().displayName-VMLowMemoryAlertLog search_amba-AvailableMemoryPercentage-threshold-Override_
Log Analytics workspaceresourceName-DailyCapLimitReachedAlertLog searchNot available since threshold will always be 0

Metric alerts table

Resource TypeAlert NameAlert TypeOverride Tag name
Virtual machineresourceName-AvailableMemoryAlertMetrics_amba-AvailableMemoryBytes-threshold-Override_
Automation AccountresourceName-TotalJobMetrics_amba-TotalJob-threshold-Override_
Front Door and CDN profileresourceName-OriginHealthPercentageMetrics_amba-OriginHealthPercentage-threshold-Override_
Front Door and CDN profileresourceName-OriginLatencyAlertMetricsNot available since it uses dynamic thresholds
Front Door and CDN profileresourceName-Percentage4XXAlertMetricsNot available since it uses dynamic thresholds
Front Door and CDN profileresourceName-Percentage5XXAlertMetricsNot available since it uses dynamic thresholds
Key vaultActivityKeyVaultDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
Key vaultresourceName-AvailabilityMetrics_amba-Availability-threshold-Override_
Key vaultresourceName-CapacityAlertMetrics_amba-SaturationShoebox-threshold-Override_
Key vaultresourceName-LatencyAlertMetrics_amba-ServiceApiLatency-threshold-Override_
Key vaultresourceName-RequestsAlertMetricsNot available since it uses dynamic thresholds
Azure Key Vault Managed HSMActivityManagedHSMDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
Azure Key Vault Managed HSMresourceName-AvailabilityMetrics_amba-Availability-threshold-Override_
Azure Key Vault Managed HSMresourceName-LatencyAlertMetrics_amba-ServiceApiLatency-threshold-Override_
Application gatewayresourceName-agApplicationGatewayTotalTimeMetricsNot available since it uses dynamic thresholds
Application gatewayresourceName-agBackendLastByteResponseTimeMetricsNot available since it uses dynamic thresholds
Application gatewayresourceName-agCapacityUnitsMetrics_amba-CapacityUnits-threshold-Override_
Application gatewayresourceName-agComputeUnitsMetrics_amba-ComputeUnits-threshold-Override_
Application gatewayresourceName-agCpuUtilizationMetrics_amba-CpuUtilization-threshold-Override_
Application gatewayresourceName-agFailedRequestsMetricsNot available since it uses dynamic thresholds
Application gatewayresourceName-agResponseStatusMetricsNot available since it uses dynamic thresholds
Application gatewayresourceName-agUnhealthyHostCountMetrics_amba-UnhealthyHostCount-threshold-Override_
FirewallActivityAzureFirewallDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
FirewallresourceName-FirewallHealthMetrics_amba-FirewallHealth-threshold-Override_
FirewallresourceName-SNATPortUtilizationMetrics_amba-SNATPortUtilization-threshold-Override_
ExpressRoute circuitresourceName-ArpAvailabilityMetrics_amba-ArpAvailability-threshold-Override_
ExpressRoute circuitresourceName-BgpAvailabilityMetrics_amba-BgpAvailability-threshold-Override_
ExpressRoute circuitresourceName-QosDropBitsInPerSecondMetricsNot available since it uses dynamic thresholds
ExpressRoute circuitresourceName-QosDropBitsOutPerSecondMetricsNot available since it uses dynamic thresholds
ExpressRoute gatewayresourceName-GatewayERBitsInAlertMetrics_amba-ERGatewayConnectionBitsInPerSecond-threshold-Override_
ExpressRoute gatewayresourceName-GatewayERBitsOutAlertMetrics_amba-ERGatewayConnectionBitsOutPerSecond-threshold-Override_
ExpressRoute gatewayresourceName-GatewayERCPUAlertMetrics_amba-ExpressRouteGatewayCpuUtilization-threshold-Override_
ExpressRoute portresourceName-DirectERBitsInAlertMetrics_amba-PortBitsInPerSecond-threshold-Override_
ExpressRoute portresourceName-DirectERBitsOutAlertMetrics_amba-PortBitsOutPerSecond-threshold-Override_
ExpressRoute portresourceName-DirectERLineProtocolAlertMetrics_amba-LineProtocol-threshold-Override_
ExpressRoute portresourceName-DirectERRxLightLevelHighAlertMetrics_amba-RxLightLevel-High-threshold-Override_
ExpressRoute portresourceName-DirectERRxLightLevelLowAlertMetrics_amba-RxLightLevel-Low-threshold-Override_
ExpressRoute portresourceName-DirectERTxLightLevelHighAlertMetrics_amba-TxLightLevel-High-threshold-Override_
ExpressRoute portresourceName-DirectERTxLightLevelLowAlertMetrics_amba-TxLightLevel-Low-threshold-Override_
Front DoorresourceName-BackendHealthPercentageMetrics_amba-BackendHealthPercentage-threshold-Override_
Front DoorresourceName-BackendRequestLatencyAlertMetricsNot available since it uses dynamic thresholds
Load balancerresourceName-ALBDataPathAvailabilityMetrics_amba-VipAvailability-threshold-Override_
Load balancerresourceName-ALBGlobalBackendAvailabilityMetrics_amba-GlobalBackendAvailability-threshold-Override_
Load balancerresourceName-ALBHealthProbeStatusMetrics_amba-DipAvailability-threshold-Override_
Load balancerresourceName-ALBUsedSNATPortsMetrics_amba-UsedSNATPorts-threshold-Override_
Network security groupActivityNSGDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
Private DNS zoneresourceName-CapacityUtilizationAlertMetrics_amba-VirtualNetworkLinkCapacityUtilization-threshold-Override_
Private DNS zoneresourceName-QueryVolumeAlertMetrics_amba-QueryVolume-threshold-Override_
Private DNS zoneresourceName-RecordSet_Capacity_UtilizationMetrics_amba-RecordSetCapacityUtilization-threshold-Override_
Private DNS zoneresourceName-RequestsAlertMetrics_amba-VirtualNetworkWithRegistrationCapacityUtilization-threshold-Override_
Public IP addressresourceName-BytesInDDOSAlertMetrics_amba-bytesinddos-threshold-Override_
Public IP addressresourceName-DDOS_AttackMetrics_amba-ifunderddosattack-threshold-Override_
Public IP addressresourceName-PacketsInDDosAlertMetrics_amba-PacketsInDDoS-threshold-Override_
Public IP addressresourceName-VIPAvailabityAlertMetrics_amba-VipAvailability-threshold-Override_
Route tableActivityUDRUpdateActivity LogNot available since Activity Log based alerts do not have thresholds
Traffic Manager profileresourceName-EndpointHealthAlertMetrics_amba-EndpointHealth-threshold-Override_
Virtual network gatewayresourceName-TunnelBandwidthAlertMetrics_amba-TunnelAverageBandwidth-threshold-Override_
Virtual network gatewayresourceName-TunnelEgressAlertMetrics_amba-TunnelEgressBytes-threshold-Override_
Virtual network gatewayresourceName-TunnelEgressPacketDropCountAlertMetricsNot available since it uses dynamic thresholds
Virtual network gatewayresourceName-TunnelEgressPacketDropTSMismatchAlertMetricsNot available since it uses dynamic thresholds
Virtual network gatewayresourceName-GatewayERBitsAlertMetrics_amba-ExpressRouteGatewayBitsPerSecond-threshold-Override_
Virtual network gatewayresourceName-GatewayERCPUAlertMetrics_amba-ExpressRouteGatewayCpuUtilization-threshold-Override_
Virtual network gatewayresourceName-TunnelIngressAlertMetrics_amba-TunnelIngressBytes-threshold-Override_
Virtual network gatewayresourceName-TunnelIngressPacketDropCountAlertMetricsNot available since it uses dynamic thresholds
Virtual network gatewayresourceName-TunnelIngressPacketDropTSMismatchAlertMetricsNot available since it uses dynamic thresholds
Virtual networkresourceName-DDOSAttackAlertMetrics_amba-ifunderddosattack-threshold-Override_
VPN GatewayActivityVPNGatewayDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
VPN GatewayresourceName-GatewayBandwidthAlertMetrics_amba-tunnelaveragebandwidth-threshold-Override_
VPN GatewayresourceName-BGPPeerStatusAlertMetrics_amba-bgppeerstatus-threshold-Override_
VPN GatewayresourceName-TunnelEgressAlertMetrics_amba-tunnelegressbytes-threshold-Override_
VPN GatewayresourceName-TunnelEgressPacketDropCountAlertMetricsNot available since it uses dynamic thresholds
VPN GatewayresourceName-TunnelEgressPacketDropTSMismatchAlertMetricsNot available since it uses dynamic thresholds
VPN GatewayresourceName-TunnelIngressAlertMetrics_amba-tunnelingressbytes-threshold-Override_
VPN GatewayresourceName-TunnelIngressPacketDropCountMetricsNot available since it uses dynamic thresholds
VPN GatewayresourceName-TunnelIngressPacketDropTSMismatchAlertMetricsNot available since it uses dynamic thresholds
Log Analytics workspaceActivityLAWorkspaceDeleteActivity LogNot available since Activity Log based alerts do not have thresholds
Log Analytics workspaceActivityLAWorkspaceRegenKeyActivity LogNot available since Activity Log based alerts do not have thresholds
SubscriptionResourceHealthUnhealthyAlertResource healthNot available since Activity Log based alerts do not have thresholds
SubscriptionServiceHealthHealthService healthNot available since Activity Log based alerts do not have thresholds
SubscriptionServiceHealthIncidentService healthNot available since Activity Log based alerts do not have thresholds
SubscriptionServiceHealthMaintenanceService healthNot available since Activity Log based alerts do not have thresholds
SubscriptionServiceSecurityIncidentService healthNot available since Activity Log based alerts do not have thresholds
Storage accountActivitySADeleteActivity LogNot available since Activity Log based alerts do not have thresholds
Storage accountresourceName-AvailabilityAlertMetrics_amba-Availability-threshold-Override_
App Service planresourceName-CpuPercentageMetrics_amba-CpuPercentage-threshold-Override_
App Service planresourceName-DiskQueueLengthAlertMetricsNot available since it uses dynamic thresholds
App Service planresourceName-HttpQueueLengthAlertMetricsNot available since it uses dynamic thresholds
App Service planresourceName-MemoryPercentageMetrics_amba-MemoryPercentage-threshold-Override_