Month: September 2024
Troubleshooting Custom Attribute Errors in Microsoft Purview: Tips for Importing a Glossary
Hello,
I’m new to Purview and could use some help. I’ve been assigned to create a glossary for my team and started by compiling all the terms and definitions into a spreadsheet. I also created an import template in Purview, adding two custom columns (attributes) using the custom template feature.
I aligned my spreadsheet with Purview’s sample CSV file and attempted to upload it. However, I encountered an error stating that the custom attribute columns were supposed to be empty (screenshot attached). While I know I can manually modify these attributes through the front end, I have over 500 terms to process.
Any guidance on what I might be doing wrong would be greatly appreciated!
Hello,I’m new to Purview and could use some help. I’ve been assigned to create a glossary for my team and started by compiling all the terms and definitions into a spreadsheet. I also created an import template in Purview, adding two custom columns (attributes) using the custom template feature.I aligned my spreadsheet with Purview’s sample CSV file and attempted to upload it. However, I encountered an error stating that the custom attribute columns were supposed to be empty (screenshot attached). While I know I can manually modify these attributes through the front end, I have over 500 terms to process.Any guidance on what I might be doing wrong would be greatly appreciated! Read More
Create relationship between calendar and fact using Date range
Hi Team,
I have two tables as below:
The fact table has date range in a imbalanced way as below:
For example, row 1 and 2 contains full days in a month(April & May).
But for row 3 contains only 1st 20 days and row 4 contains next 10 days.
This is how we get the data and we can do nothing about it.
We want to create a relationship from this fact table to calendar table in order to find the average cost per month.
If we find the average as it is from above fact table, it gives average as 200 which is wrong :
Correct one is if we add the 3rd and 4th row values because it counts to full april month(01/04/2023 – 30/04/2023) then it counts to 500. average is 266.67. But data doesn’t come as we wish.
As the fact table contains dates in two columns(date range), we first convert this 2 column of dates into a single list of date column in power query(dataflow). Then connect that date column to calendar date and find the average cost per month.
Now what happens is that, as our original data(Fact) contains millions of rows, this operation of converting two column of dates into a single column(each day) causes lot of performance issues. This causes data flow refresh issues, report to slow down etc.
Is there any other ways we can create relationship between calendar table and fact table, without converting range of dates into day granularity in fact table but still pick up the cost for those dates?
Is it possible to create a dax measure that create a relationship between these two tables ?
Please suggest.
FYR, PFA sample file here Test.pbix
Thanks in advance!
Hi Team, I have two tables as below: The fact table has date range in a imbalanced way as below: For example, row 1 and 2 contains full days in a month(April & May).But for row 3 contains only 1st 20 days and row 4 contains next 10 days.This is how we get the data and we can do nothing about it. We want to create a relationship from this fact table to calendar table in order to find the average cost per month.If we find the average as it is from above fact table, it gives average as 200 which is wrong : Correct one is if we add the 3rd and 4th row values because it counts to full april month(01/04/2023 – 30/04/2023) then it counts to 500. average is 266.67. But data doesn’t come as we wish. As the fact table contains dates in two columns(date range), we first convert this 2 column of dates into a single list of date column in power query(dataflow). Then connect that date column to calendar date and find the average cost per month. Now what happens is that, as our original data(Fact) contains millions of rows, this operation of converting two column of dates into a single column(each day) causes lot of performance issues. This causes data flow refresh issues, report to slow down etc. Is there any other ways we can create relationship between calendar table and fact table, without converting range of dates into day granularity in fact table but still pick up the cost for those dates? Is it possible to create a dax measure that create a relationship between these two tables ? Please suggest.FYR, PFA sample file here Test.pbix Thanks in advance!@SergeiBaklan Read More
How to become a Azure Solutions Architect
I am currently about to graduate high school and apply for a Bachelor’s degree in Information Technology majoring in Computer Science. May I ask for advice on how to start my journey to become a solutions architect? Such as learning pathways, skills to develop, entry-level jobs that I can get hand-on experience with, and needed certifications.
Furthermore, I would like to know what it is like to become a Solutions Architect. Such as what is a typical day to day life as a solutions architect. Do you have the option to work remotely/hybrid? What is the work environment like?
I am currently about to graduate high school and apply for a Bachelor’s degree in Information Technology majoring in Computer Science. May I ask for advice on how to start my journey to become a solutions architect? Such as learning pathways, skills to develop, entry-level jobs that I can get hand-on experience with, and needed certifications.Furthermore, I would like to know what it is like to become a Solutions Architect. Such as what is a typical day to day life as a solutions architect. Do you have the option to work remotely/hybrid? What is the work environment like? Read More
CentrePal, Moveworks, and Saturam offer transactable partner solutions in Azure Marketplace
Microsoft partners like CentrePal, Moveworks, and Saturam deliver transact-capable offers, which allow you to purchase directly from Azure Marketplace. Learn about these offers below:
CentrePal Contact Center for Microsoft Teams: CentrePal empowers customer service and helpdesk teams to deliver exceptional customer service from Microsoft Teams. With CentrePal, agents can easily manage customer interactions across voice and chat, gain insights, and more – all from the familiar Teams environment.
Moveworks – The Agentic Copilot for All Employees: Designed to complement Copilot for Microsoft 365, the Moveworks Copilot is powered by a pioneering reasoning engine that uses public and proprietary language models to understand employee requests, then build and execute multi-step plans to resolve them.
Qualdo-DRX: Qualdo-DRX delivers around-the-clock Microsoft Azure data observability. Monitor more than 70 data reliability and quality metrics, proactively identify data anomalies, drive informed decision-making, and receive automatic alerts when data quality issues arise.
Microsoft Tech Community – Latest Blogs –Read More
Error: “You’ve entered too many argument for this function”
Hi – i’m receiving this error when completing this excel formula. I can’t determine why though. Can you please assist? The goal of this formula is to determine if the cells are blank or filled in and return “completed” “partially completed” or “not completed” And if column J is closed then return closed.
Also, if column Y has the below two text than X can remain blank. if they dont contain that text, column X needs to be filled in.
=IF(J2=”Closed”,”Closed”,IF(OR(Y2=”7. > 24 months”,Y2=”7. > 24 months”),””,IF(AND(COUNTBLANK(K2:W2)=0,COUNTBLANK(AA2:AD2)=0),”Completed”,”Partially Completed”),”Not Completed”)
Hi – i’m receiving this error when completing this excel formula. I can’t determine why though. Can you please assist? The goal of this formula is to determine if the cells are blank or filled in and return “completed” “partially completed” or “not completed” And if column J is closed then return closed.Also, if column Y has the below two text than X can remain blank. if they dont contain that text, column X needs to be filled in. =IF(J2=”Closed”,”Closed”,IF(OR(Y2=”7. > 24 months”,Y2=”7. > 24 months”),””,IF(AND(COUNTBLANK(K2:W2)=0,COUNTBLANK(AA2:AD2)=0),”Completed”,”Partially Completed”),”Not Completed”) Read More
Detecting AiTM Phishing via 3rd-Party Network events in Unified Security Operations Platform
Microsoft Security has been evolving from individual security products – such as endpoint, email, identity, and app – to XDR (Extended Detection and Response) solution, and it also offers a cloud-native SIEM solution, Microsoft Sentinel. Despite having these two strong security backbones, we have made tremendous progress by unifying the SIEM and XDR experience into a single platform called the Unified Security Operations Platform. Thanks to that, this platform provides comprehensive visibility, investigation, and response capabilities across endpoints, hybrid identities, emails, collaboration tools, cloud apps, cloud workloads, and data.
Regarding Advanced Hunting capability, there are now no boundaries for threat hunting. Security analysts can access various tables across Microsoft Defender XDR and Microsoft Sentinel. However, with the introduction of the unified hunting experience, “SecurityAlert” table, which previously contained data from Microsoft security solutions and was available in Microsoft Sentinel, is no longer present in Advanced Hunting. Instead, Advanced Hunting now utilizes “AlertInfo“ and “AlertEvidence” tables.
Hunting Adversary-in-the-Middle (AiTM) attacks is a great example of gaining additional insights through advanced hunting. AiTM attacks use sophisticated tactics, including the creation of fraudulent sites that intercept user login credentials. This allows attackers to hijack sign-in sessions and bypass authentication protections. Even users with Multifactor Authentication (MFA) enabled can fall victim to this method. A Unified Security Operations Platform not only provides out-of-the-box (OOTB) detection alerts but also includes attack disruption capabilities to stop ongoing attacks, thanks to its correlation mechanisms and various signals from Microsoft Defender XDR. Although a Unified Security Operations Platform provides a significant number of AiTM detections, you may still want visibility into how third-party network activity and network detections correlate with first-party logs, such as Entra ID sign-in events and AiTM-related URL click actions. Therefore, at this time, we would like to provide two queries.
Hunting AiTM Phishing Events in a Unified Security Operations Platform
These two KQL queries utilize four data tables. The first is CommonSecurityLog table, which stores 3rd-party network logs from sources such as ‘Palo Alto Networks,’ ‘Fortinet,’ ‘Check Point,’ and ‘Zscaler,’ available through Microsoft Sentinel. The second is SigninLogs table, which contains Microsoft Entra ID sign-in event data from Microsoft Sentinel. Finally, AlertEvidence and AlertInfo tables contain detection data from Microsoft Defender XDR. By using join operations and binding common keys across four different data tables within a single platform, we can detect suspicious AiTM-related activities in Advanced Hunting, a Unified Security Operations Platforms.
Prerequisite :
1) These KQL queries should be executed within the Unified Security Operations Platform environment, rather than in Microsoft Defender XDR or Microsoft Sentinel portal. For details on integration, please refer to Connect Microsoft Sentinel to Microsoft Defender XDR – Microsoft Defender XDR | Microsoft Learn
2) To run these queries, Azure Monitor, CommonSecurityLog table (‘Palo Alto Networks,’ ‘Fortinet,’ ‘Check Point,’ and ‘Zscaler’) must is required. For details on the table, please refer to Azure Monitor Logs reference – CommonSecurityLog | Microsoft Learn
Phishing Link Clicks in Network Traffic
Description: This rule is designed to identify successful phishing link clicks by users and the subsequent network activity from non-Microsoft network devices.
How it works: It identifies phishing-related alerts in Microsoft Defender XDR and matches them with 3rd party network device logs such as Firewalls instead non Microsoft devices. It aims to detect successful phishing link clicks followed by suspicious network activity.
// Define a list of alert titles that we are interested in
let Alert_List = dynamic([“Phishing link click observed in Network Traffic”,
“Phish delivered due to an IP allow policy”,
“A potentially malicious URL click was detected”,
“High Risk Sign-in Observed in Network Traffic”,
“A user clicked through to a potentially malicious URL”,
“Suspicious network connection to AitM phishing site”,
“Messages containing malicious entity not removed after delivery”,
“Email messages containing malicious URL removed after delivery”,
“Email reported by user as malware or phish”,
“Phish delivered due to an ETR override”,
“Phish not zapped because ZAP is disabled”]);
// Filter AlertInfo for relevant alerts within the past 10 days from Defender for Office 365
AlertInfo
| where TimeGenerated > ago(5d)
| where DetectionSource == “Microsoft Defender for Office 365”
| where Title has_any (Alert_List)
// Join with AlertEvidence to get additional evidence details
| join kind=inner (
AlertEvidence
| where TimeGenerated > ago(5d)
| where DetectionSource == “Microsoft Defender for Office 365”
| where EntityType in (“Url”, “User”)
) on AlertId
// Parse the JSON field AdditionalFields to extract entities
| extend Entities = parse_json(AdditionalFields)
| mv-apply Entity = Entities on (
where Entity.Type in (‘account’, ‘url’)
| extend
// Assign entity properties based on type
EntityUPN = iff(Entities.Type == ‘account’, strcat(Entities.Name, “@”, Entities.UPNSuffix), Entities.UserPrincipalName),
“”,
EntityUrl = iff(Entities.Type == ‘url’, tostring(Entities.Url), “”)
)
// Extract the domain from the URL if it’s not empty
| extend DomainFromUrl = iff(isnotempty(EntityUrl), tostring(parse_url(EntityUrl).Host), “”)
// Summarize to create sets of UPNs and URLs grouped by AlertId and TimeGenerated
| summarize UserPrincipalNames = make_set(EntityUPN), Urls = make_set(EntityUrl) by AlertId, TimeGenerated
// Expand the sets to have individual rows for each UPN and URL
| mv-expand Urls
| mv-expand UserPrincipalNames
// Filter out empty URLs and UPNs
| where isnotempty(Urls)
| where isnotempty(UserPrincipalNames)
// Parse URL into its components
| extend Url = tostring(Urls)
| extend Domain = tostring(parse_url(Url).Host), Path = tostring(parse_url(Url).Path)
// Project relevant columns
| project AlertTime = TimeGenerated, AlertId, UserPrincipalName = UserPrincipalNames, Url, Domain, Path, tostring(parse_url(Url))
// Join with CommonSecurityLog for related network activity
| join kind=inner (
CommonSecurityLog
//| where TimeGenerated > ago(5d)
| where DeviceAction != “Block”
| where DeviceProduct has_any (“FortiGate”, “PAN”, “VPN”, “FireWall”, “NSSWeblog”, “URL”)
| where isnotempty(RequestURL)
| where isnotempty(SourceUserName)
| extend RequestURL = tostring(tolower(RequestURL))
| project
LogTime = TimeGenerated,
DeviceVendor,
DeviceProduct,
Activity,
DestinationHostName,
DestinationIP,
Domain = tostring(parse_url(RequestURL).Host),
RequestPath = tostring(parse_url(RequestURL).Path),
MaliciousIP,
UserName = tostring(split(SourceUserName, “@”)[0]),
UPNSuffix = tostring(split(SourceUserName, “@”)[1]),
SourceUserName,
IndicatorThreatType,
ThreatSeverity,
AdditionalExtensions,
ThreatConfidence
) on Domain // Join on Domain for matching records
| where RequestPath has Path
Correlating M365D Alerts with Non-Microsoft Network Device Activity
Description: This rule correlates Microsoft Defender XDR phishing-related alerts with sign-in activities on non-Microsoft network devices, especially when users connect to phishing URLs.
How it works: It correlates Microsoft 365 Defender alerts with network logs from devices like Palo Alto Networks, Fortinet, Check Point, and Zscaler. It focuses on cases where users connect to phishing URLs from these devices and subsequently make successful sign-in attempts.
// Define a list of alert titles that we are interested in
let Alert_List = dynamic([“Phishing link click observed in Network Traffic”,
“Phish delivered due to an IP allow policy”,
“A potentially malicious URL click was detected”,
“High Risk Sign-in Observed in Network Traffic”,
“A user clicked through to a potentially malicious URL”,
“Suspicious network connection to AitM phishing site”,
“Messages containing malicious entity not removed after delivery”,
“Email messages containing malicious URL removed after delivery”,
“Email reported by user as malware or phish”,
“Phish delivered due to an ETR override”,
“Phish not zapped because ZAP is disabled”]);
// Filter AlertInfo for relevant alerts within the past 10 days from Defender for Office 365
AlertInfo
| where TimeGenerated > ago(5d)
| where DetectionSource == “Microsoft Defender for Office 365”
| where Title has_any (Alert_List)
// Join with AlertEvidence to get additional evidence details
| join kind=inner (
AlertEvidence
| where TimeGenerated > ago(5d)
| where DetectionSource == “Microsoft Defender for Office 365”
| where EntityType in (“Url”, “User”)
) on AlertId
// Parse the JSON field AdditionalFields to extract entities
| extend Entities = parse_json(AdditionalFields)
| mv-apply Entity = Entities on (
where Entity.Type in (‘account’, ‘url’)
| extend
// Assign entity properties based on type
EntityUPN = iff(Entities.Type == ‘account’, strcat(Entities.Name, “@”, Entities.UPNSuffix), Entities.UserPrincipalName),
“”,
EntityUrl = iff(Entities.Type == ‘url’, tostring(Entities.Url), “”)
)
// Extract the domain from the URL if it’s not empty
| extend DomainFromUrl = iff(isnotempty(EntityUrl), tostring(parse_url(EntityUrl).Host), “”)
// Summarize to create sets of UPNs and URLs grouped by AlertId and TimeGenerated
| summarize UserPrincipalNames = make_set(EntityUPN), Urls = make_set(EntityUrl) by AlertId, TimeGenerated
// Expand the sets to have individual rows for each UPN and URL
| mv-expand Urls
| mv-expand UserPrincipalNames
// Filter out empty URLs and UPNs
| where isnotempty(Urls)
| where isnotempty(UserPrincipalNames)
// Parse URL into its components
| extend Url = tostring(Urls)
| extend Domain = tostring(parse_url(Url).Host), Path = tostring(parse_url(Url).Path)
| extend AlertTime= TimeGenerated
// matching with 3rd party network logs and 3p Alerts
| join kind= inner (CommonSecurityLog
| where TimeGenerated > ago(5d)
| where DeviceVendor has_any (“Palo Alto Networks”, “Fortinet”, “Check Point”, “Zscaler”)
| where DeviceProduct startswith “FortiGate” or DeviceProduct startswith “PAN” or DeviceProduct startswith “VPN” or DeviceProduct startswith “FireWall” or DeviceProduct startswith “NSSWeblog” or DeviceProduct startswith “URL”
| where DeviceAction != “Block”
| where isnotempty(RequestURL)
| project
3plogTime=TimeGenerated,
DeviceVendor,
DeviceProduct,
Activity,
DestinationHostName,
DestinationIP,
RequestURL=tostring(tolower(RequestURL)),
MaliciousIP,
SourceUserName=tostring(tolower(SourceUserName)),
IndicatorThreatType,
ThreatSeverity,
ThreatConfidence,
SourceUserID,
SourceHostName)
on $left.Url == $right.RequestURL
// matching successful Login from suspicious IP
| join kind=inner (SigninLogs
//filtering the Successful Login
| where TimeGenerated > ago(5d)
| where ResultType == 0
| project
IPAddress,
SourceSystem,
SigniningTime= TimeGenerated,
OperationName,
ResultType,
ResultDescription,
AlternateSignInName,
AppDisplayName,
AuthenticationRequirement,
ClientAppUsed,
RiskState,
RiskLevelDuringSignIn,
UserPrincipalName=tostring(tolower(UserPrincipalName)),
Name = tostring(split(UserPrincipalName, “@”)[0]),
UPNSuffix =tostring(split(UserPrincipalName, “@”)[1]))
on $left.DestinationIP == $right.IPAddress and $left.SourceUserName == $right.UserPrincipalName
| where SigniningTime between ((AlertTime – 6h) .. (AlertTime + 6h)) and 3plogTime between ((AlertTime – 6h) .. (AlertTime + 6h))
Microsoft Tech Community – Latest Blogs –Read More
The power of Data Collection Rules: Detect Disabling Windows Defender Real-Time Protection
This is Part 3 of our Blog series on how to collect events using DCRs for advanced use cases:
Part 2: Monitoring PowerShell through Azure Monitor Agent (AMA) and Microsoft Sentinel
Although Defender for Endpoint (MDE) consists of various protections against tampering and of alerts to detect it, adversaries are constantly trying to find a way to successfully circumvent and terminate MDE.
One product that really helps you to protect against all kinds of tampering, is Defender’s Tamper protection. To make the most out of your product suite, ensure that you have Tamper Protection configured and enforced in your environment: Make sure Tamper Protection is turned on – Microsoft Community Hub.
But nevertheless, there are scenarios in which organizations cannot enforce Tamper Protection for all devices and attackers being attackers always try their best to bypass defences and come up with new creative approaches to evade detection mechanisms and defences. Therefore, it might be a good idea to monitor Defender related event logs in parallel and to respond if you spot such malicious behaviour.
In this article we will review what event IDs you can collect to detect potential malicious behaviour that can affect the device protection in your organization.
First, let’s have a look at the relevant event IDs and their definitions. To get more detail on each event, you can also refer to this article Review event logs and error codes to troubleshoot issues with Microsoft Defender Antivirus.
The following event IDs can be found in the Security event log and are relevant to detect if MDE was disabled:
3002 – Real-time protection encountered an error and failed.
5001 – Real-time protection is disabled.
5008 – The antimalware engine encountered an error and failed.
5010 – Scanning for malware and other potentially unwanted software is disabled.
5012 – Scanning for viruses is disabled.
Since these events also contain regular configuration changes, they should be reviewed on a regular basis and only be acted upon if a certain configuration change was initiated without your knowledge.Now, we will see how we can collect these specific logs in Microsoft Sentinel and the Unified Security Operations Platform (if you have not onboarded Microsoft Sentinel to the Microsoft Defender portal, see how to do it here: Connect Microsoft Sentinel to Microsoft Defender XDR – Microsoft Defender XDR | Microsoft Learn).
Please note that we will be mentioning several event Ids and drilling down on why we recommend them. If you wish to collect them all, you can go directly to the Summary section at the bottom of the article and copy the final xPath query.
Step 1: Configure the data collection Rule (DCR) to collect the required events
Just like we did in the previous article (link), using the Azure Monitor agent (AMA), you can select the events you would like to collect from your servers.
To create your DCR, as this time we are collecting non-Security events, under Microsoft Sentinel, select Configuration and then Data connectors:
Then, select Windows Forwarded Events. If you cannot find this connector, please make sure you download the solution from the Content hub (under Content management) in the first place.
From here, select create DCR, add your servers, and under Collect select Custom. Paste the following xPath queries:
Microsoft-Windows-Windows Defender/Operational!*[System[(EventID=3002) or (EventID=5001) or (EventID=5008) or (EventID=5010) or (EventID=5012)]]
Step 2: Detecting instances in which critical parts of MDE were disabled or shut down
In Advanced Hunting, please use the following query to find instances in which critical parts of MDE were disabled or shut down:
WindowsEvent
| where EventID in (3002, 5001, 5008, 5010, 5012)
| extend [“EventID description”] = case(
EventID == 3002, “Real-time protection encountered an error and failed”,
EventID == 5001, “Real-time protection is disabled”,
EventID == 5008, “The antimalware engine encountered an error and failed”,
EventID == 5010, “Scanning for malware and other potentially unwanted software is disabled”,
EventID == 5012, “Scanning for viruses is disabled”,
“Unknown EventID”
)
| extend ProductName = tostring(EventData[“Product Name”]), ProductVersion = tostring(EventData[“Product Version”])
| project TimeGenerated, Computer, SystemUserId, EventID, [‘EventID description’], ProductName, ProductVersion
Now, you may want to create a detection —consider creating a near-real time (NRT) analytic— to trigger an alert when this happens. In that case, please go to Analytics (under Data connectors), and create your detection. Please remember to map the entities, as this is critical for the correlation engine to work, among other things.
Step 3: Detecting configuration changes
Additionally, it is also a good practice to monitor configuration changes, as this might indicate that an attacker is tampering with it. If a configuration change was not initiated by someone who is authorized to, consider this behaviour suspicious.
The following events can be also found in the Security event log and indicate a configuration change:
5004 – The real-time protection configuration changed.
5007 – The antimalware platform configuration changed.
To collect these events, please add the following XPath query to your existing DCR, which you can edit from the connector used in Step 1:
Microsoft-Windows-Windows Defender/Operational!*[System[(EventID=5004) or (EventID=5007)]]
Use this query to detect a configuration change:
WindowsEvent
| where EventID == 5013
| extend [“EventID description”] = “Tamper protection blocked a change to Microsoft Defender Antivirus”
| extend ProductName = tostring(EventData[“Product Name”]), ProductVersion = tostring(EventData[“Product Version”]), OldValue = tostring(EventData[“Old Value”]), NewValue = tostring(EventData[“New Value”])
| project TimeGenerated, Computer, SystemUserId, EventID, [‘EventID description’], ProductName, ProductVersion, OldValue, NewValue
However, you may not want to trigger incidents when these events happen, as configuration changes may not be malicious. In this case, you have two options:
Create an analytic rule that triggers an alert, but not an incident. This is an interesting approach for customers using the Unified Security Operations Platform (onboarding Microsoft Sentinel to the Microsoft Defender portal). This was, if the correlation engine finds anything suspicious, you may get an incident.
Create a hunting query (under Threat Management, select Hunting).
Step 4: Detect tampering attempts
There is also another event that you might find useful to keep track of: event id 5013. If you have Tamper Protection enabled on your devices and if any attempts were made to change Defender’s configuration, that were blocked by the Tamper Protection feature, you can find it in the Defender event log under event 5013.
To collect these events, please add the following XPath query to your existing DCR, which you can edit from the connector used in Step 1:
Microsoft-Windows-Windows Defender/Operational!*[System[(EventID=5013)]]
Use this query to detect a configuration change:
WindowsEvent
| where EventID == 5013
| extend [“EventID description”] = “Tamper protection blocked a change to Microsoft Defender Antivirus”
| extend ProductName = tostring(EventData[“Product Name”]), ProductVersion = tostring(EventData[“Product Version”])
| project TimeGenerated, Computer, SystemUserId, EventID, [‘EventID description’], ProductName, ProductVersion
It might be useful to keep track of this event to early spot attempted manipulation attempts in your environment.
Summary
If you wish to collect all events mentioned in this article, you can use this xPath query under the connector Windows Forwarded Events (as this will make the events to go the WindowsEvent table):
Microsoft-Windows-Windows Defender/Operational!*[System[(EventID=3002) or (EventID=5001) or (EventID=5004) or (EventID=5007) or (EventID=5008) or (EventID=5010) or (EventID=5012) or (EventID=5013)]]
This concludes Part 3 of our 3 Part blog series on how to collect events using DCRs for advanced use cases_
Part 1: The power of Data Collection Rules: Collecting events for advanced use cases in Microsoft USOP – Mic…
Part 2: Monitoring PowerShell through Azure Monitor Agent (AMA) and Microsoft Sentinel
We welcome your feedback and questions on this or any of the other parts of this blog article series and look forward to hearing from you.
Special thanks to our reviewers Ashwin Patil and Yaniv Carmel from the Security Research team.
Miriam Wiesner (@miriamxyra) – Senior Security Research PM for Microsoft Defender XDR Incidents | Maria de Sousa-Valadas Castaño – Senior Product Manager Unified SecOps Platform | Shirley Kochavi – Senior Product Manager Unified SecOps Platform
Microsoft Tech Community – Latest Blogs –Read More
See what’s possible with Copilot in Excel (part 6)
Unlock the power of problem-solving and creativity with Copilot in Excel, where you can seamlessly detect palindromes, generate map links, craft riddles, plan sailing itineraries, and work with your photography hobby—all within your spreadsheets.
Monday, 16-Sep – Detecting Palindromes using Copilot in Excel
Tuesday, 17-Sep – Generating links to maps using Copilot in Excel
Wednesday, 18-Sep – Copilot in Excel riddles
Thursday, 19-Sep – Sailing itinerary with Copilot in Excel
Friday, 20-Sep – Using Copilot in Excel for my photography hobby
These posts are pinned within the Tech Community Forum each week. You can catch up on the other Copilot series by reading the recap blogs here >.
Stay tuned for next week’s series!
Microsoft Tech Community – Latest Blogs –Read More
SharePoint Allowing File Upload with Invalid Filename Starting with ‘$’
Dear Microsoft Team,
I am encountering an issue with SharePoint that contradicts the official documentation. According to the SharePoint documentation, filenames starting with the character $ are not allowed when uploading files. However, in our SharePoint environment, I have been able to upload files with names beginning with $ without any error or restriction.
Dear Microsoft Team,I am encountering an issue with SharePoint that contradicts the official documentation. According to the SharePoint documentation, filenames starting with the character $ are not allowed when uploading files. However, in our SharePoint environment, I have been able to upload files with names beginning with $ without any error or restriction. Read More
Teams Panels – check-in issues
Hey, anyone experience issues with Teams Panels check-in feature?
We have seen that the “Check-In” button on the panel is there but suddenly is dissapears. When that happens the rooms is not released and the email “Booking declined: No one checked in, so the room was automatically removed from your meeting to make it available for others.” is never sent to the organizer.
Hey, anyone experience issues with Teams Panels check-in feature? We have seen that the “Check-In” button on the panel is there but suddenly is dissapears. When that happens the rooms is not released and the email “Booking declined: No one checked in, so the room was automatically removed from your meeting to make it available for others.” is never sent to the organizer. Read More
la mise a jour suivante :Cumulative Update for Windows 11 Insider Preview (10.0.26120.1843) (KB50431
Bonjour,
Imposiible de telecharger et d’installer la mise a jour
Bonjour,Imposiible de telecharger et d’installer la mise a jour Read More
unable to connect AVD
i am getting below error when i am trying to connect host through AVD, i was able to access it before and now its not working Please suggest
i am getting below error when i am trying to connect host through AVD, i was able to access it before and now its not working Please suggest Read More
Date Format in EXCEL for the Web
“I added a data validation rule into a column of a worksheet created and successfully tested in EXCEL for Windows. Only Dates after 1. August 2023 should be accepted. The Region of the Worksheet and my Computer is set to Germany.
When I enter a date e.g. 2.3.24 ( = 2 March 2023) into Excel for the Web, I get the message, that only dates after 08/01/2023 are accepted.
It appears that Excel for the Web expects the date in the format mm/dd/yyyy, but I would like to enter it in the format dd.mm.yyyy.
How can I adjust this behavior in Excel for the Web?”
”I added a data validation rule into a column of a worksheet created and successfully tested in EXCEL for Windows. Only Dates after 1. August 2023 should be accepted. The Region of the Worksheet and my Computer is set to Germany.When I enter a date e.g. 2.3.24 ( = 2 March 2023) into Excel for the Web, I get the message, that only dates after 08/01/2023 are accepted.It appears that Excel for the Web expects the date in the format mm/dd/yyyy, but I would like to enter it in the format dd.mm.yyyy.How can I adjust this behavior in Excel for the Web?” Read More
How to Create an xPath Filter for a Data Collection Rule
In the world of data collection, efficiency is key. Just as my miniature schnauzer buddy, Raven, has a knack for sniffing out the most interesting scents while ignoring the mundane, an xPath filter can be used to streamline data collection by focusing only on the most relevant information. This document will guide you through the process of writing an xPath filter for a data collection rule, ensuring that your data ingress is as efficient and effective as Raven’s nose.
Imagine Raven on a walk, surrounded by countless scents. If she were to investigate every single one, she’d waste a lot of time and energy. Instead, she selectively sniffs out the most intriguing smells, saving her effort for what truly matters. Similarly, an xPath filter helps you avoid the unnecessary data, allowing you to zero in on the specific pieces of information you need. By the end of this guide, you’ll be able to create xPath filters that reduce data ingress and improve the overall efficiency of your data collection processes.
Windows Event Log capture is a cornerstone of effective security monitoring. It provides a detailed record of events across a multitude of logs, including but not limited to system, application, security, and Application and Service event logs. By capturing these logs, organizations can monitor for suspicious activities, detect potential security breaches, and ensure compliance with regulatory requirements. The detailed information contained in these logs helps security teams identify patterns and anomalies that may indicate malicious behavior, enabling a proctive approach to threat detection and response.
One of the primary benefits of Windows Event Log capture is its ability to provide real-time visibility into the activities occurring within an IT environment. This visibility is crucial for identifying and responding to security incidents promptly. For instance, logs can reveal unauthorized access attempts, changes to critical system files, or unusual network traffic patterns. By analyzing these logs, security teams can quickly pinpoint the source of an issue and take appropriate action to mitigate the risk.
However, the sheer volume of data generated by Windows Event Logs can be overwhelming. This is where xPath filters come into play. xPath, or XML Path Language, is a powerful tool for querying and filtering XML data. When applied to Windows Event Logs, xPath filters can help security teams focus on the most relevant events, reducing the noise and making it easier to identify significant security incidents. By using xPath filters, organizations can create customized queries that extract specific information from the logs, such as failed login attempts, changes to user privileges, specific error codes, or even events related to specific applications like Microsoft Exchange or SQL Server.
The use of xPath filters not only enhances the efficiency of log analysis but also improves the accuracy of threat detection. By narrowing down the data to only the most pertinent events, security teams can reduce false positives and concentrate their efforts on genuine threats. This targeted approach ensures that critical security incidents are not overlooked amidst a sea of irrelevant data. Additionally, xPath filters can be tailored to meet the unique needs of an organization, allowing for a highly customized and effective log monitoring solution.
In essence, Windows Event Log capture is an invaluable tool for maintaining the security and integrity of an IT environment. By leveraging xPath filters, organizations can optimize their log analysis processes, ensuring that they can quickly and accurately identify and respond to security threats. Just as Raven, my miniature schnauzer, efficiently sniffs out the most interesting scents, xPath filters help security teams focus on the most critical events, enhancing their ability to protect their systems and data.
This document provides guidance on creating an xPath filter for a Data Collection Rule (DCR) for the Azure Monitoring Agent (AMA). A DCR is used to create a filter for user-defined Event IDs from Windows event logs. The xPath filter, which is created by the user, can be applied to both Windows – Azure, Hybrid and Azure Arc devices. The collected events are then sent to an Azure Monitor, Log Analytics Workspace table named “EVENT”.
To get started the user will need to define a list of Windows Event Logs to capture and the Event IDs within those log(s). There is the option to collect all of the Event IDs but that is not recommended unless specifically needed. All data collected and stored consume storage space which cost money for data ingressed and stored.
For the examples within this document, I will collect the following Event IDs from the Event Logs:
Security
1102, 4624
System
111, 113, 117
PowerShell
Root PowerShell
400
Operational PowerShell
4103, 4104
To review where these are located within the Event logs, there are screen grabs to help find them.
Start up the Windows Event Viewer
Drill in the Event Viewer to find each Event Log
To create an xPath filter, you can have the Event Log Viewer assist in the build.
Right click on an Event Log log and select “Filter Current Log”
This will bring up the following Window (There are two tabs available):
On the “Filter” tab, enter the Event IDs, separating each with a comma.
Select the “XML” tab
To convert the Event Log Filter into an xPath form for the DCR, the following steps should be performed.
Looking at the first screen capture above (Encased in red):
<Query Id=”0” Path=”Security”>
Copy the name within the quotes
Security
Append “!” on to the previous results
Security!
Looking at the second screen capture above (Encased in red):
<Select Path=”Security”>*[System[(EventID=1102 or EventID=4624)]]</Select>
Copy everything between the two greater than and less than symbols > < on to the previous results
Security!*[System[(EventID=1102 or EventID=4624)]]
The result is now an xPath filter that can be used with your DCR (To be created later).
Note 1: Additional EventIDs can be added to the filter by simply inserting “ or EventID=9999” before the close parenthesis “)”within the xPath filter.
Note 2: If all you would like to capture ALL events from a log then only the first 2 bullets above would need to be completed and then a “*” would need to be appended to include all events.
Security!*
Repeat the process above for each Event Log you would like to capture/filter. From the examples to be used the complete list of xPath filters can be found below.
Save the xPath definitions in a text file that will be used in the DCR creation process, coming up next.
Security!*[System[(EventID=1102 or EventID=4624)]]
System!*[System[(EventID=111 or EventID=113 or EventID=117)]]
Windows PowerShell!*[System[(EventID=400)]]
Microsoft-Windows-PowerShell/Operational!*[System[(EventID=4103 or EventID=4104)]]
Creating a new DCR to capture the data
Browse to https://portal.azure.com > “Monitor”
Settings > Data Collection Rules
Create a Data Collection Rule
Select “+ Create”
On the “Basic” tab, enter “Rule Name”, “Subscription”, “Resource Group”, “Region” and Windows
Using a “Data Collection Endpoint” is optional for this demo
On the “Resources” tab, select “Add resources”
Browse to the resources to apply this DCR against
On the “Collect and deliver” tab
Select “+ Add data source”
On the sub-tab “Data source”, one by one, copy and paste each xPath filter. Hit “Add” after each paste.
After all the xPath’ s have been added they should now be displayed below the entry line as you can see in the example below.
On the sub-tab “Destination”, enter “Destination Type”, “Subscription”, and “Destination Details”.
The “Destinations Details”, Log Analytics Workspace, is where the Event ID details will be sent within Azure.
Enter any Tags required
Review the “Review and create”, if all ok, select “Create”
The Data Collection Rules blade should now reflect the new rule.
When using a “Custom” xPath filter, the “Basic” tab reflects like nothing has been defined. Ensure you review the “Custom” / “Data source” tab to see the filters.
After you have created this DCR (xPath filter) the devices that will have this defined against will soon start to send their EventID activities to the “Event” table within the Log Analytics Workspace.
To ensure that a device has had the DCR applied against it, the following PowerShell commands can be done to review the definition on the device.
From a PowerShell command prompt run the below:
Note: The xml output will be stored at c:temp. Ensure that path exists before running the script below.
Connect-AzAccount
$subscriptionId = “————————————-“
Set-AzContext -SubscriptionId $subscriptionId
$resourceGroupName = “RG-Security”
$dcrName = “CustomerSpecificRuleSet”
# Get the Data Collection Rule and output to a file
Get-AzDataCollectionRule -ResourceGroupName $resourceGroupName -Name $dcrName -SubscriptionId $subscriptionId | ConvertTo-Xml -As String -Depth 3 | Out-File -FilePath “C:tempdcr.xml”
Write-Host “Data Collection Rule details have been written to C:tempdcr.xml”
Looking in the dcr.xml finds the following:
Once the DCR has been applied and events start to be pushed you can now find the events in the “Event” table.
Query the Event Table
In Azure Monitor > Logs the data can now be queried.
A simple query:
Event
| project TimeGenerated, Source, EventLog, EventID
If you notice the event log name and any EventIDs associated are populated within the query. The example only outputs 4 columns, but this was done to provide for display simplicity. Review the schema for a complete set of columns available.
In conclusion, just as Raven efficiently sniffs out the most interesting scents on her walks, an xPath filter can help you streamline your data collection by focusing on the most relevant information. By applying the techniques outlined in this document, you can reduce data ingress and ensure that your data collection processes are both efficient and effective.
Remember, the key to a successful xPath filter is precision and selectivity, much like Raven’s keen sense of smell. With these skills, you’ll be able to navigate through vast amounts of data and extract only what is necessary, saving time and resources. Happy filtering!
Microsoft Tech Community – Latest Blogs –Read More
Looking for old windows beta builds which is not archived and non leaked ones for internal builds
I wanted these builds for historical and research purporses so it would be great if I get get access to the iso’s
I wanted these builds for historical and research purporses so it would be great if I get get access to the iso’s Read More
Strange Glitches Found in Windows 11
Occasionally, when I click on an icon in the taskbar, it unexpectedly opens the icon adjacent to it. To address this issue, I find it necessary to restart explorer.exe.
Furthermore, there is an additional problem where a newly opened window frequently and unpredictably reverts to a previously opened one.
These persistent issues have persisted for several months, despite multiple clean installations of Windows 11.
Occasionally, when I click on an icon in the taskbar, it unexpectedly opens the icon adjacent to it. To address this issue, I find it necessary to restart explorer.exe. Furthermore, there is an additional problem where a newly opened window frequently and unpredictably reverts to a previously opened one. These persistent issues have persisted for several months, despite multiple clean installations of Windows 11. Read More
Continuous Help Support
In the recent updates, a significant issue has arisen where multiple programs unexpectedly open their help pages, causing interruptions in functionality until the help window is closed. This sometimes results in delays lasting up to 10 minutes, prompting me to take a break and brew a cup of coffee while waiting for the help program to cease its persistent opening and closing cycles.
In response, I have taken the step to disable updates and have discarded the last two updates in the hope that this action may resolve the problem, although only time will tell. The situation became so untenable that I had to uninstall Office and opt for Apache Open Office as a replacement, only to encounter similar help window issues.
I will keep you updated on whether this adjustment proves effective in addressing the persistent problem.
In the recent updates, a significant issue has arisen where multiple programs unexpectedly open their help pages, causing interruptions in functionality until the help window is closed. This sometimes results in delays lasting up to 10 minutes, prompting me to take a break and brew a cup of coffee while waiting for the help program to cease its persistent opening and closing cycles. In response, I have taken the step to disable updates and have discarded the last two updates in the hope that this action may resolve the problem, although only time will tell. The situation became so untenable that I had to uninstall Office and opt for Apache Open Office as a replacement, only to encounter similar help window issues. I will keep you updated on whether this adjustment proves effective in addressing the persistent problem. Read More
Title: Taskbar search function flashes before I can access it
I attempted to conduct a search, but it abruptly stops and disappears, leaving me unable to proceed. This issue seems to occur intermittently, resolving itself only to reoccur later on.
I attempted to conduct a search, but it abruptly stops and disappears, leaving me unable to proceed. This issue seems to occur intermittently, resolving itself only to reoccur later on. Read More
Title: Surprisingly Impressed by Windows After Years of Mac Usage – Thread by stormboy, Nov 3, 2023
I recently received my Dell Latitude 3310 laptop on Thursday evening to use for my severe weather chase team and ham radio activities. I had been a primarily Apple user since switching to Mac shortly after the release of Windows Vista. Considering my past negative experiences with Microsoft Windows, I never expected to be impressed with any version of Windows again. However, after using the laptop for a day and a half, I must admit that Windows 11 Pro has left me pleasantly surprised.
Upon receiving the laptop, it was pre-installed with version 21H2, and it prompted me to update to version 22H2. I anticipated a frustrating experience with the update process, expecting interruptions and potential issues. To my delight, Windows 11 Pro handled the update smoothly. It informed me when the download was complete and asked to schedule a restart for the installation, respecting my ongoing tasks.
Despite my concerns, the installation process was quick and trouble-free. I was worried about potential issues post-update, but to my relief, everything worked seamlessly when I resumed using the laptop that evening. While I have not extensively used it yet, it seems that Microsoft has finally created an operating system that just works. I am genuinely pleased with my experience with Windows 11 Pro so far.
I recently received my Dell Latitude 3310 laptop on Thursday evening to use for my severe weather chase team and ham radio activities. I had been a primarily Apple user since switching to Mac shortly after the release of Windows Vista. Considering my past negative experiences with Microsoft Windows, I never expected to be impressed with any version of Windows again. However, after using the laptop for a day and a half, I must admit that Windows 11 Pro has left me pleasantly surprised. Upon receiving the laptop, it was pre-installed with version 21H2, and it prompted me to update to version 22H2. I anticipated a frustrating experience with the update process, expecting interruptions and potential issues. To my delight, Windows 11 Pro handled the update smoothly. It informed me when the download was complete and asked to schedule a restart for the installation, respecting my ongoing tasks. Despite my concerns, the installation process was quick and trouble-free. I was worried about potential issues post-update, but to my relief, everything worked seamlessly when I resumed using the laptop that evening. While I have not extensively used it yet, it seems that Microsoft has finally created an operating system that just works. I am genuinely pleased with my experience with Windows 11 Pro so far. Read More
Title: Embracing Sleep in Just a Few Minutes
I have tried various configurations of the Windows power system to adjust the sleep timeout period to no avail. Despite starting with a 20-minute timeout, my attempts to increase it have somehow led to a decrease in the period. The display, which is an older AOC monitor, does not offer any relevant settings for this. Can you provide assistance on how to effectively extend the timeout period?
I have tried various configurations of the Windows power system to adjust the sleep timeout period to no avail. Despite starting with a 20-minute timeout, my attempts to increase it have somehow led to a decrease in the period. The display, which is an older AOC monitor, does not offer any relevant settings for this. Can you provide assistance on how to effectively extend the timeout period? Read More