Azure Monitor Agent DCRs: Collect Windows Events, Counters and Syslog

To collect Windows event logs, performance counters and Syslog with Azure Monitor Agent, create a data collection rule (DCR) in the same region as your Log Analytics workspace, add a Windows Event Logs, Performance Counters or Linux Syslog data source with an Azure Monitor Logs destination, and associate the rule with your machines. When you add machines to a DCR in the Azure portal, the agent (AzureMonitorWindowsAgent or AzureMonitorLinuxAgent) is installed automatically and the association is created. Data lands in the Event, Perf and Syslog tables; allow up to five minutes after you create the rule.

Who this is for and what you will have at the end

This guide is for administrators who need guest operating system logs from Azure VMs and Azure Arc-enabled servers in a Log Analytics workspace, either as a first deployment or as the replacement for the retired Log Analytics agent. It assumes you have a workspace and can create resources in the subscription that holds it.

At the end you will have:

• A Windows DCR that collects filtered event logs and a set of performance counters.
• A Linux DCR that collects Syslog facilities at the severity you choose.
• The same rules as JSON files you can keep in source control and deploy with the Azure CLI or PowerShell.
• Associations for Azure VMs and Arc-enabled servers, plus an Azure Policy assignment that covers new machines.
• Queries to prove the data is arriving, and a troubleshooting path when it isn't.

How the pieces fit together

A DCR can hold up to 10 data sources. Combining sources in one DCR or splitting them across several is functionally the same; choose the layout that makes the rules easiest to manage.

Prerequisites

• A Log Analytics workspace where you have at least contributor rights, and permission to create DCRs. The built-in Monitoring Contributor role covers creating and editing DCRs and associations; deploying them as templates also needs Microsoft.Resources/deployments/*.
• Region alignment. Create the DCR in the same region as the destination workspace.
• Machines. Azure VMs need a managed identity; the portal enables a system-assigned identity by default when you add a VM to a DCR. Machines outside Azure must be connected to Azure Arc first; see onboarding servers to Azure Arc at scale.
• Network access on TCP 443 to the endpoints below, with HTTPS inspection disabled. On Azure virtual networks, allow the AzureMonitor and AzureResourceManager service tags.

The agent supports a direct proxy, a Log Analytics gateway (on Azure VMs) and private links. Arc-enabled servers don't support the Log Analytics gateway for the agent.

Design decisions before you build

• Separate Windows and Linux rules. Microsoft warns that mixing Windows and Linux counter names in one DCR can collect the same metric twice, for example \LogicalDisk(*)\Disk Transfers/sec and Logical Disk(*)\Disk Transfers/sec. One DCR per operating system avoids that.
• One workspace per data source. You can add several workspaces as destinations, but each receives a full copy and you pay for each.
• Security events. Choose either the Event table through a DCR or the SecurityEvent table through the Microsoft Sentinel connector, not both.
• Legacy agent. If the old Log Analytics agent is still installed on a machine, both agents may write the same data to the same table until you remove it.

Step 1: Create the Windows DCR in the portal

1. In the Azure portal, open Monitor > Data Collection Rules > Create.
2. On Basics, enter a rule name, subscription, resource group and the Region of your workspace. Choose the Type of telemetry for VM guest data; Help me choose shows which data sources each option allows and whether they need a DCE or managed identity. In the classic experience, set Platform Type to Windows instead.
3. On Resources, select Add resources and pick Azure VMs and Arc-enabled servers. You can also add them later. Virtual machine scale sets with flexible orchestration can't be added directly; add the individual VMs or use Azure Policy.
4. On Collect and deliver, select Add new dataflow for each data source described in the next two steps.
5. Select Review + create.

Windows event logs with XPath filters

Choose Windows Event Logs as the data source type. Basic lets you pick logs and severities. Custom lets you enter XPath queries in the form LogName!XPathQuery, which is where most of the cost control happens, because filtering at the agent means the data is never ingested.

Test each query on a Windows machine before adding it. The log name goes to -LogName, the rest to -FilterXPath:

$XPath = '*[System[(Level=1 or Level=2 or Level=3) and (EventID != 6)]]'
Get-WinEvent -LogName 'System' -FilterXPath $XPath -MaxEvents 5

If you get events, the query is valid. No events were found that match the specified selection criteria means the syntax is probably fine but nothing matches locally. The specified query is invalid means fix the syntax. XPath 1.0 limitations apply: band, position and timediff work, while starts-with and contains aren't supported. Analytic and Debug channels can't be collected, and event logs must be on a local disk, not a network share.

Add a destination of type Azure Monitor Logs and select the workspace.

Performance counters

Choose Performance Counters. Basic offers predefined objects and a sample rate; a lower sample rate means more frequent collection. Custom accepts counter paths in the format \PerfObject(ParentInstance/ObjectInstance#InstanceIndex)\Counter. If a counter name contains an ampersand, write it as &, for example \Memory\Free & Zero Page List Bytes.

Send counters to Azure Monitor Logs for the Perf table. You can also add Azure Monitor Metrics (preview); note that Arc-enabled server metrics aren't visible in Metrics Explorer and must be read through the Metrics REST API, and that the agent's proxy settings don't apply to that destination.

Step 2: Create the Linux Syslog DCR

Create a second DCR the same way (with Platform Type set to Linux in the classic experience), add your Linux machines, and add a Linux Syslog data source. For each facility, choose a Minimum log level or NONE. Everything at that severity and above is collected, in this order from lowest to highest: Debug, Info, Notice, Warning, Error, Critical, Alert, Emergency.

The checkboxes on this page are cleared every time you reopen an existing DCR, so they don't tell you what the rule currently collects. Check the facilityNames property in the DCR's JSON view instead.

When a DCR with Syslog reaches a machine, the agent writes a forwarding configuration for the local daemon and restarts it:

• rsyslog: /etc/rsyslog.d/10-azuremonitoragent-omfwd.conf, which forwards to 127.0.0.1 on TCP port 28330. On SELinux systems that use Unix sockets, the agent writes /etc/rsyslog.d/10-azuremonitoragent.conf instead.
• syslog-ng: /etc/syslog-ng/conf.d/azuremonitoragent-tcp.conf.

Two behaviors catch people out. On rsyslog, the agent adds its rule to the default ruleset only, so inputs bound to other rulesets aren't forwarded. And semicolons in Syslog messages are removed during ingestion into Log Analytics; replace them at the source if you need them.

Step 3: Manage the same rules as JSON

Portal-built rules are fine to start with, but JSON definitions are easier to review and redeploy. This Windows rule combines the event and counter data sources; replace the workspace resource ID and the location.

{
  "location": "westeurope",
  "properties": {
    "dataSources": {
      "windowsEventLogs": [
        {
          "name": "eventLogsDataSource",
          "streams": [ "Microsoft-Event" ],
          "xPathQueries": [
            "System!*[System[(Level=1 or Level=2 or Level=3) and (EventID != 6)]]",
            "Application!*[System[(Level = 1 or Level = 2 or Level = 3)]]"
          ]
        }
      ],
      "performanceCounters": [
        {
          "name": "perfCounters60",
          "streams": [ "Microsoft-Perf" ],
          "samplingFrequencyInSeconds": 60,
          "counterSpecifiers": [
            "\\Processor(_Total)\\% Processor Time",
            "\\Memory\\Committed Bytes",
            "\\LogicalDisk(_Total)\\Free Megabytes",
            "\\PhysicalDisk(_Total)\\Avg. Disk Queue Length"
          ]
        }
      ]
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.OperationalInsights/workspaces/law-prod-weu",
          "name": "centralWorkspace"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [ "Microsoft-Event" ],
        "destinations": [ "centralWorkspace" ],
        "transformKql": "source",
        "outputStream": "Microsoft-Event"
      },
      {
        "streams": [ "Microsoft-Perf" ],
        "destinations": [ "centralWorkspace" ],
        "transformKql": "source",
        "outputStream": "Microsoft-Perf"
      }
    ]
  }
}

The Linux rule uses the syslog data source and the Microsoft-Syslog stream:

{
  "location": "westeurope",
  "properties": {
    "dataSources": {
      "syslog": [
        {
          "name": "syslogAuth",
          "streams": [ "Microsoft-Syslog" ],
          "facilityNames": [ "auth", "authpriv" ],
          "logLevels": [ "Info", "Notice", "Warning", "Error", "Critical", "Alert", "Emergency" ]
        },
        {
          "name": "syslogSystem",
          "streams": [ "Microsoft-Syslog" ],
          "facilityNames": [ "daemon", "kern", "syslog" ],
          "logLevels": [ "Warning", "Error", "Critical", "Alert", "Emergency" ]
        }
      ]
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.OperationalInsights/workspaces/law-prod-weu",
          "name": "centralWorkspace"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [ "Microsoft-Syslog" ],
        "destinations": [ "centralWorkspace" ],
        "transformKql": "source",
        "outputStream": "Microsoft-Syslog"
      }
    ]
  }
}

Create or update the rules from the files:

az monitor data-collection rule create --location westeurope --resource-group rg-monitor \
  --name dcr-windows-baseline --rule-file ./dcr-windows-baseline.json
 
az monitor data-collection rule create --location westeurope --resource-group rg-monitor \
  --name dcr-linux-syslog --rule-file ./dcr-linux-syslog.json

New-AzDataCollectionRule -Name 'dcr-windows-baseline' -ResourceGroupName 'rg-monitor' -JsonFilePath '.\dcr-windows-baseline.json'

If the command returns a vague error, deploy the same JSON through the REST API or an ARM template; Microsoft notes those methods return more detailed compile errors. Also keep in mind that editing a DCR in the portal overwrites JSON-only features such as transformations the portal doesn't support. Once you manage a rule as JSON, keep managing it as JSON.

Step 4: Install the agent and associate machines

Creating a DCR from the CLI or PowerShell doesn't install anything. Install the extension, then create the association. Unlike the portal, the CLI doesn't enable a managed identity for you, so if the VM has none yet, enable a system-assigned identity first with az vm identity assign --name vm-app01 --resource-group rg-app. For an Azure VM using its system-assigned identity:

az vm extension set --name AzureMonitorWindowsAgent --publisher Microsoft.Azure.Monitor \
  --ids "/subscriptions/<subscription-id>/resourceGroups/rg-app/providers/Microsoft.Compute/virtualMachines/vm-app01" \
  --enable-auto-upgrade true

For an Arc-enabled Linux server:

az connectedmachine extension create --name AzureMonitorLinuxAgent --publisher Microsoft.Azure.Monitor \
  --type AzureMonitorLinuxAgent --machine-name srv-lnx01 --resource-group rg-arc-servers \
  --location westeurope --enable-auto-upgrade true

Then associate each machine with the right DCR:

az monitor data-collection rule association create --name "vm-app01-windows-baseline" \
  --rule-id "/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.Insights/dataCollectionRules/dcr-windows-baseline" \
  --resource "/subscriptions/<subscription-id>/resourceGroups/rg-app/providers/Microsoft.Compute/virtualMachines/vm-app01"

New-AzDataCollectionRuleAssociation -AssociationName 'srv-lnx01-syslog' `
  -TargetResourceId '/subscriptions/<subscription-id>/resourceGroups/rg-arc-servers/providers/Microsoft.HybridCompute/machines/srv-lnx01' `
  -DataCollectionRuleId '/subscriptions/<subscription-id>/resourceGroups/rg-monitor/providers/Microsoft.Insights/dataCollectionRules/dcr-linux-syslog'

Don't clone a machine that already has the agent installed; Microsoft doesn't support it. Install the agent after cloning, ideally through policy.

Step 5: Cover new machines with Azure Policy

For fleets, let Azure Policy install the agent and create associations. Open the DCR in the portal, select Policies, and choose Assign Policy or Assign Initiative. Built-in options include the initiative Configure Windows machines to run Azure Monitor Agent and associate them to a Data Collection Rule and the policy Configure Windows Machines to be associated with a Data Collection Rule or a Data Collection Endpoint. The Policy/Initiative definition list shows every definition that accepts a DCR as a parameter, so pick the one that matches the rule's operating system. Choose the subscription and resource group scope and keep Policy Enforcement enabled.

The assignment only affects new or updated machines until you create a remediation task, so create one immediately for the existing machines.

Verification

Allow up to five minutes after creating or changing a DCR. First confirm the agent is healthy; it sends a heartbeat every minute:

Heartbeat
| where Category == "Azure Monitor Agent"
| where TimeGenerated > ago(15m)
| summarize LastHeartbeat = max(TimeGenerated) by Computer, OSType
| order by LastHeartbeat desc

Then check each table:

Event
| where TimeGenerated > ago(1h)
| summarize Events = count() by Computer, EventLog, EventLevelName
| order by Events desc

Perf
| where TimeGenerated > ago(30m)
| summarize Samples = count(), AvgValue = avg(CounterValue) by Computer, ObjectName, CounterName

Syslog
| where TimeGenerated > ago(1h)
| summarize Messages = count() by Computer, Facility, SeverityLevel

On the Azure side, list the rules associated with a machine to spot duplicates:

Get-AzDataCollectionRuleAssociation -ResourceUri '/subscriptions/<subscription-id>/resourceGroups/rg-app/providers/Microsoft.Compute/virtualMachines/vm-app01'

DCR metrics such as Logs Rows Received per Min, Logs Rows Dropped per Min and Logs Transformation Errors per Min show whether data reaches Azure Monitor and whether a transformation discards it.

Troubleshooting

No heartbeat from a Windows machine. Check that AzureMonitorWindowsAgent shows Provisioning succeeded under Extensions + applications. That status only means the package installed, so also check that MonAgentCore.exe is running. Extension logs are in C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Monitor.AzureMonitorWindowsAgent, and core agent logs under C:\WindowsAzure\Resources\AMADataStore.<machine-name>\Configuration.

Agent runs but collects nothing. Look for C:\WindowsAzure\Resources\AMADataStore.<machine-name>\mcs\mcsconfig.latest.xml. If it's missing, the machine isn't associated with a DCR, the managed identity isn't enabled, or the agent can't reach the Instance Metadata Service; IMDS errors appear in ...\Tables\MAEventTable.tsf. An empty or missing mcs\configchunks folder means the agent couldn't download its DCRs from the configuration service, which usually points to the handler.control.monitor.azure.com endpoints being blocked.

Events missing for one log. Open mcs\mcsconfig.lkg.xml and check that a Subscription node with your XPath query exists. If it doesn't, the DCR JSON has no windowsEventLogs section or the query was rejected. For counters, look for CounterSet nodes in the same file.

Data in the wrong region or not at all. A DCR in a different region from the workspace won't deliver. Recreate it in the workspace's region.

Linux messages missing. Confirm the forwarding file exists under /etc/rsyslog.d/ or /etc/syslog-ng/conf.d/, that the source isn't bound to a non-default rsyslog ruleset, and restart the daemon after any manual edit.

Duplicate rows. Look for two DCRs with the same data source on one machine, Security events collected both by a DCR and by Sentinel, or the legacy agent still running.

Checklist

• Workspace and DCRs in the same region; Windows and Linux rules kept separate.
• XPath queries tested with Get-WinEvent and kept under the 20-expression limit.
• Counters chosen deliberately, with Linux and Windows counter names never mixed.
• Syslog facilities and levels confirmed in the JSON facilityNames property.
• Rules stored as JSON and deployed with az monitor data-collection rule create or New-AzDataCollectionRule.
• Agent installed with automatic upgrade; associations created; policy plus remediation task for new machines.
• Heartbeat, Event, Perf and Syslog queries returning data.

Once logs are flowing from your hybrid servers, the same Arc-enabled machines can be patched centrally; see Azure Update Manager scheduled patching.

References

• Collect log data from virtual machines with Azure Monitor
• Collect Windows events with Azure Monitor Agent
• Collect performance counters with Azure Monitor Agent
• Collect Syslog events with Azure Monitor Agent
• Sample data collection rules
• Create data collection rules using JSON
• Manage data collection rule associations
• Install and manage the Azure Monitor Agent
• Azure Monitor Agent network configuration
• Troubleshoot the Azure Monitor agent on Windows virtual machines