Log Query Alert
Azure 서비스의 로그를 수집하고 있는 경우, 대상 로그에 관해 사용자 지정 쿼리를 이용하여 알람 규칙을 만드는 것이 가능하다.
예를 들어서, Virtual machine
의 로그를 수집하고 있다면 디스크의 빈 용량을 확인하는 쿼리를 사용하여 해당 값을 주기적으로 확인하고 몇 % 이하의 경우 알람을 발생시키는 것이 가능하다.
Tip. Azure Portal 에서 수동으로 Custom log search
를 지정하여 알람 규칙을 만드는 경우는 아래를 참고
Perf
| where ObjectName == "LogicalDisk"
| where CounterName == "% Free Space"
| where Computer == "AZRWIN-1"
| where InstanceName == "C:"
| summarize arg_max(TimeGenerated, *) by InstanceName
| project CounterValue
관련 포스트
준비 하기
https://github.com/jcy9033/azure-cli/tree/master/azureMonitor/logQueryAlert
참고용 Azure CLI 명령어 소개
Microsoft Learn az monitor scheduled-query
link
az monitor scheduled-query create --condition
--name
--resource-group
--scopes
[--action-groups]
[--auto-mitigate {false, true}]
[--check-ws-alerts-storage {false, true}]
[--condition-query]
[--custom-properties]
[--description]
[--disabled {false, true}]
[--evaluation-frequency]
[--location]
[--mad]
[--severity]
[--skip-query-validation {false, true}]
[--tags]
[--target-resource-type]
[--window-size]
CSV 구성 소개
- AlertRuleName
- Subscription
- ResourceGroup
- Severity
Subscription_UserResourceGroup_User- ResourceName
- LogQuery
:Perf | where CounterName == '% Committed Bytes In Use' | where Computer == 'AZRWIN-1' | summarize arg_max(TimeGenerated, *) by InstanceName | project CounterValue
Tip. ” ” (Double quotation) 대신 ‘ ‘ (Single quotation)을 사용하지 않으면 에러가 발생하므로 주의. - Measure
: 측정값의 기준 ex) CounterValue - Aggregation
- WindowSize
- Dimensions
- Operator
- Threshold
- EvaluationFrequency
- AutoMitigate
- Location
- ActionGroupName
Script 구성 소개
CSV 파일 경로 및 가져 오기
# Please enter the file path for the CSV
$csvPath = "CSV File Path"
$csvData = Import-Csv $csvPath -Encoding UTF8
Subscription 확인 하기
몇 번째 행을 처리 중인지 확인하기 위해서 Create No.$i
를 추가했다.
$i = 1
$csvData | ForEach-Object {
#----------------------------------------- Subscription check
Write-Host "----------# [Create No.$i : $(Get-Date)]"
$TargetSubscription = $_.Subscription
$ActiveSubscription = az account show --query name -o tsv
if ($ActiveSubscription -eq $TargetSubscription) {
Write-Host "[Info] Already on the target subscription: $TargetSubscription"
}
else {
az account set --subscription $TargetSubscription
Write-Host "[Info] Switched to the target subscription: $TargetSubscription"
}
Action Group 값 가져 오기
#----------------------------------------- Action Group
$ActionGroupName = $_.ActionGroupName
$ActionGroupList = $ActionGroupName -split ';'
Write-Host "[Info] Action group list: $ActionGroupList"
$ActionGroups = @()
foreach ($ActionGroup in $ActionGroupList) {
$ActionGroupJson = az graph query -q "Resources | where type == 'microsoft.insights/actiongroups' and name has '$ActionGroup' | project id" | ConvertFrom-Json
if ($ActionGroupJson.data -and $ActionGroupJson.data.Count -gt 0) {
$ActionGroupId = $ActionGroupJson.data[0].id
$ActionGroups += "$ActionGroupId"
}
else {
Write-Host "[Warning] No valid action group ID found for $ActionGroup"
}
}
Write-Host "[Info] Action groups parameter: $ActionGroups"
Log Query 변수에 저장하기
#----------------------------------------- Log query
$LogQuery = $_.LogQuery
Write-Host "[Info] Condition Log Query: $LogQuery"
Condition String 만들기
#----------------------------------------- Conditions
$Condition = "$($_.Aggregation) $($_.Measure) from 'LogQuery' $($_.Operator) $($_.Threshold)"
Write-Host "[Info] Scheduled query alert condition: $Condition"
Resource id 값 가져오기
#----------------------------------------- Resource parameters
$ResourceName = $_.ResourceName
$Scopes = (az resource list --query "[?name=='$ResourceName'].id" --output tsv).Trim()
Write-Host "[Info] Resource scopes: $Scopes"
Log Query Alert 만들기
#----------------------------------------- Create Scheduled query alert
az monitor scheduled-query create --name $_.AlertRuleName --resource-group $_.ResourceGroup --scopes $Scopes `
--severity $_.Severity `
--condition "$Condition" `
--condition-query LogQuery="$LogQuery" `
--evaluation-frequency $_.EvaluationFrequency `
--auto-mitigate $_.AutoMitigate `
--window-size $_.WindowSize `
--action-group $ActionGroups
$i++
}
Write-Host "----------# [All done: $(Get-Date)]"
Script 실행 결과
PS C:\Users\cchi9\OneDrive\Vscode\azure-cli\azureMonitor\logQueryAlert> .\main.ps1
----------# [Create No.1 : 01/06/2024 14:17:17]
[Info] Already on the target subscription: Azure subscription 2
[Info] Action group list: ag-1 ag-2 ag-3
[Info] Action groups parameter: /subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-1 /subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-2 /subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-3
[Info] Condition Log Query: Perf | where CounterName == '% Committed Bytes In Use' | where Computer == 'AZRWIN-1' | summarize arg_max(TimeGenerated, *) by InstanceName | project CounterValue
[Info] Scheduled query alert condition: avg CounterValue from 'LogQuery' > 90
[Info] Resource scopes: /subscriptions/0b5f5005-c30c-4a28-89c1-9457d0cd5e0f/resourceGroups/system-1/providers/Microsoft.Compute/virtualMachines/AZRWIN-1
{
"actions": {
"actionGroups": [
"/subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-1",
"/subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-2",
"/subscriptions/611a7ed8-17fa-480a-901d-d7084803c376/resourceGroups/core-1/providers/microsoft.insights/actiongroups/ag-3"
],
"customProperties": {}
},
"autoMitigate": true,
"checkWorkspaceAlertsStorageConfigured": false,
"createdWithApiVersion": "2021-08-01",
"criteria": {
"allOf": [
{
"dimensions": [],
"failingPeriods": {
"minFailingPeriodsToAlert": 1,
"numberOfEvaluationPeriods": 1
},
"metricMeasureColumn": "CounterValue",
"metricName": null,
"operator": "GreaterThan",
"query": "Perf | where CounterName == '% Committed Bytes In Use' | where Computer == 'AZRWIN-1' | summarize arg_max(TimeGenerated, *) by InstanceName | project CounterValue",
"resourceIdColumn": null,
"threshold": 90.0,
"timeAggregation": "Average"
}
]
},
"description": null,
"displayName": null,
"enabled": true,
"etag": null,
"evaluationFrequency": "0:05:00",
"id": "/subscriptions/0b5f5005-c30c-4a28-89c1-9457d0cd5e0f/resourceGroups/system-1/providers/microsoft.insights/scheduledqueryrules/log1-1",
"isLegacyLogAnalyticsRule": null,
"isWorkspaceAlertsStorageConfigured": null,
"kind": null,
"location": "japaneast",
"muteActionsDuration": null,
"name": "log1-1",
"overrideQueryTimeRange": null,
"resourceGroup": "system-1",
"scopes": [
"/subscriptions/0b5f5005-c30c-4a28-89c1-9457d0cd5e0f/resourceGroups/system-1/providers/Microsoft.Compute/virtualMachines/AZRWIN-1"
],
"severity": 1,
"skipQueryValidation": false,
"systemData": {
"createdAt": "2024-01-06T05:17:23.125963+00:00",
"createdBy": "xxxxx",
"createdByType": "User",
"lastModifiedAt": "2024-01-06T05:17:23.125963+00:00",
"lastModifiedBy": "xxxxx",
"lastModifiedByType": "User"
},
"tags": null,
"targetResourceTypes": null,
"type": "Microsoft.Insights/scheduledQueryRules",
"windowSize": "0:05:00"
}
----------# [Create No.2 : 01/06/2024 14:17:29]
[Info] Already on the target subscription: Azure subscription 2
[Info] Action group list: ag-1 ag-2 ag-3
...Skip
----------# [All done: 01/06/2024 14:17:49]