In this blog, I’m going to focus on a common yet very important use case: transformation (workspace and DCR) transformation.

With transformations in Azure Monitor, you can filter or modify incoming data before it’s sent to a Log Analytics workspace. Transformations are performed in Azure Monitor in the data ingestion pipeline after the data source delivers the data and before it’s sent to the destination. The data source might perform its own filtering before sending data but then rely on the transformation for further manipulation before it’s sent to the destination.

The following diagram illustrates the transformation process for incoming data and shows a sample query that might be used.

Before we begin, it’s important to understand why to use transformation and the supported tables for the same.

Our focus in this blog is going to be around workspace transformation and DCR transformation. Let’s delve into the use cases for both:

Workspace transformation

It’s recommended to use workspace transformation for platform logs (collected via diagnostic settings) and logs collected using MMA (legacy agent).

The workspace transformation data collection rule (DCR) is a special DCR that’s applied directly to a Log Analytics workspace. The purpose of this DCR is to perform transformations on data that does not yet use a DCR for its data collection, and thus has no means to define a transformation.

Prerequisites for creating workspace transformation:

A Log Analytics workspace where you have at least contributor rights.
Permissions to create DCR objects in the workspace.
A table that already has some data.
The table can’t be linked to the workspace transformation DCR.

I recommend going through our public documentation for step by step guidance to create workspace transformation.

In my use case, I’ve created a transformation rule on DeviceEvents table to keep only those records where ActionType contains “Antivirus”

Once the transformation is configured from the Azure Portal, it creates a workspace transformation DCR (if not present) & appends the ARM template of the DCR with the transformation logic.

To validate this transformation logic, we can browse to the respective log analytics workspace > Logs and run a KQL query to check if DeviceEvents table and the ActionTypes it’s capturing. In my use case, it’s retaining only those records where ActionType contains “Antivirus” as shown below:

DCR Transformation

This transformation technique is best suited for the logs collected via AMA. It’s recommended to update “transformKql” with relevant KQL logic under data flow.

I recommend going through our public documentation which talks about structure of transformation. Also, check our public documentation for step by step guidance on creating the transformation logic.

In my use case, I’ve created a DCR transformation logic to drop those records from Syslog table where ProcessName contains “Kernel”.

I already have a Linux VM which is sending Syslog to my log analytic workspace via AMA.

Let’s browse to the DCR associated with this VM and then browse to Automation > Export Template > Deploy > Edit Template as shown below:

The template should have all the required facility names defined for Syslog (under data sources). Once it’s validated, check the “dataFlows” section and update the “transformKql” logic and “outputSteam” as shown below:

Here we’ve defined a steam for Microsoft-Syslog to discard the rows if ProcessName contains “kernel”. Once the ARM template is updated, Save the changes and hit Review and Create so that changes could take effect.

Check our public documentation on the structure of single destination and multiple destination DCR examples.

To validate this transformation logic, go to the respective log analytics workspace and run a KQL query to check if there are any rows in Syslog table where ProcessName contains “kernel”

Transformation may incur some additional charges, check our public documentation on the same.

​Microsoft Tech Community – Latest Blogs –Read More