Introduction

In an increasingly interconnected world, the need for robust cybersecurity measures has never been more critical. As businesses and organizations migrate to the cloud, they must address not only the conventional threats but also more sophisticated ones like Distributed Denial of Service (DDoS) attacks. Azure, Microsoft’s cloud computing platform, offers powerful tools to protect your applications and data. In this blog post, we will explore how to leverage Azure DDoS Protection in combination with Azure Web Application Firewall (WAF) rate limiting to enhance your security posture.

Understanding DDoS Attacks

Distributed Denial of Service attacks are a malicious attempt to disrupt the normal functioning of a network, service, or website by overwhelming it with a flood of internet traffic. These attacks can paralyze online services, causing severe downtime and financial losses. Azure DDoS Protection is a service designed to mitigate such attacks and ensure the availability of your applications hosted on Azure.

Combining Azure DDoS Protection with WAF Rate Limiting

While Azure DDoS Protection can mitigate many types of attacks, it’s often beneficial to combine it with a Web Application Firewall for comprehensive security. Azure WAF provides protection at the application layer, inspecting HTTP/HTTPS traffic and identifying and blocking malicious requests. One of the key features of Azure WAF is rate limiting, which allows you to control the number of incoming requests from a single IP address or Geo location. By setting appropriate rate limiting rules, you can mitigate application-layer DDoS attacks.

In this article, we will delve into DDoS protection logs, exploring how to harness this valuable data to configure rate limiting on the Application Gateway WAF. By doing so, we fortify our defenses at various layers, ensuring a holistic approach to DDoS protection.

Note: Rate limiting for Application gateway WAF is currently in GA, you can find more information here  Azure Web Application Firewall (WAF) rate limiting | Microsoft Learn 

Example Attack scenario

In this scenario, we outline a two-phase DDoS (Distributed Denial of Service) attack for illustration purposes. The attacker initiates with a Layer 4 TCP SYN flood attack by a bot network. This targets the network infrastructure with a flood of TCP (Transmission Control Protocol) SYN packets, primarily targeting Layer 4, the transport layer. The objective is to overwhelm network resources, including bandwidth and processing capacity, disrupting access for legitimate users. Azure DDoS Protection detects and mitigates this Layer 4 attack.

Subsequently, attackers transition to Phase 2, launching a Layer 7 (L7) DDoS attack with the same bot network. Here, the focus shifts to Layer 7, the application layer. In this scenario, they deploy a Layer 7 flood attack, exploiting application-level vulnerabilities in the target application. The goal remains consistent: disrupting the target’s application by leveraging Layer 7 weaknesses. Real-world DDoS attacks may employ various vectors, depending on application vulnerabilities. Azure DDoS Protection, combined with complementary security measures like Web Application Firewall (WAF) rate limiting, forms a robust defense against these attacks, ensuring service continuity and protection against evolving DDoS tactics.”

Note: In our testing environment, we’re using spoofed Layer 4 DDoS attacks instead of those carried out by a bot network. In actual real-world situations, the attack vectors can vary widely, adapting to the specific vulnerabilities and targets. In this scenario, we assume that the attackers use the same source IPs since they are focused on launching attacks in quick succession and do not expect the target to respond quickly enough. This scenario serves as a simplified representation to highlight the importance of multi-layered defenses and the role of Azure DDoS Protection and WAF rate limit in mitigating DDoS attacks.

Prerequisites

Set up an Application Gateway with the WAF V2 SKU and select the latest WAF engine by choosing CRS 3.2 as the default rule set.
Associate a public IP address with your application gateway and activate Azure DDoS Protection (Network or IP SKU).
Ensure that logging is enabled for your public IP resource and on your Application Gateway.

Setting up DDoS protection

Ensure that Azure DDoS Protection is activated for your application gateway’s public IP. You can do this by navigating to the public IP address resource and verifying that DDoS protection is correctly configured.

To enable logging for your public IP address, access your public IP resource. Within the Diagnostic settings, create a new diagnostic configuration. Ensure that you select the DDoS logs categories and specify your preferred destination log analytics workspace.

Investigating and understanding Azure DDoS protection logs

Navigate to your log analytics workspace logs and run the following query to confirm that your public IP endpoint was under active DDoS mitigation:

AzureDiagnostics

| where Category == “DDoSProtectionNotifications”

Note: Azure DDoS protection logs are generated only during active DDoS mitigation.

As shown below, there’s a log type called “MitigationStarted,” confirming the occurrence of a DDoS attack. The message field provides details about the targeted public IP.

Next, let’s determine the source IPs responsible for this DDoS attack. Run the following query:

AzureDiagnostics

| where Category == “DDoSMitigationFlowLogs”

| where Message <> “Packet was forwarded to service”

| project Message, SourceIPAddress = tostring(sourcePublicIpAddress_s)

| summarize LogCount = count() by Message, SourceIPAddress

| order by LogCount desc

This query filters Azure Diagnostics logs for “DDoSMitigationFlowLogs,” extracts log messages and source IP addresses, and summarizes how many times each unique combination of message and source IP address appears in the logs. The results are sorted in descending order of log counts.

In the query results below, we see that the highest log counts contain the message ‘protocol violation invalid TCP syn,’ indicating that this traffic was identified as malicious by the DDoS mitigation system

Another method for identifying malicious source IPs in DDoS attacks is by utilizing the Sentinel DDoS Protection solution. This solution includes two analytical rules, triggering incidents when specific thresholds are reached. I’ve implemented the PPS threshold, which led to the incident described below.

As illustrated, the entities displayed represent the source IPs detected during this DDoS mitigation, aligning with the source IPs previously identified through the logs query.

For additional details on the Sentinel DDoS Protection solution, see here Azure DDoS Solution for Microsoft Sentinel – Microsoft Community Hub

Configuring rate limit on Application gateway WAF

Now that we have pinpointed the malicious source IPs behind the DDoS attacks, we can employ this data to set up rate limiting in our Web Application Firewall (WAF). Rate limiting is configured through custom rules, and you have the flexibility to attach the policy either globally to your Application Gateway or on a per-site/URI basis. For instance, if your Application Gateway serves four distinct sites and you wish to tailor the WAF configuration for each site, you can attach different policies to individual listeners to accommodate site-specific WAF settings. For further details. For more information, see here Configure per-site WAF policies using PowerShell – Azure Web Application Firewall | Microsoft Learn

Within the custom rules section, create a new rule and select ‘rate limit’ as the rule type. Here, you have the flexibility to choose the rate limit duration, ranging from 1 to 5 minutes, as well as the rate limit request threshold, which defines the maximum number of requests permitted within the specified rate limit duration. Given that we have identified the source IPs, choose ‘client address’ as the group rate limit traffic option. In the ‘conditions’ section, choose the match type ‘IP address,’ and then add the identified malicious IP addresses.

Note: While it is possible to configure a complete block on the identified IP addresses, it’s worth noting that attackers occasionally compromise legitimate users’ machines to launch DDoS attacks. Therefore, we opt for rate limiting to avoid outright blocking, allowing for a more nuanced approach to security.

The optimal rate limit setting depends on your specific environment and traffic patterns. One useful metric to guide you is the ‘WAF Total Requests’ found under your Application Gateway instance metrics. By selecting this metric and extending the timeline to at least 30 days, you can gather more comprehensive data to make an informed decision. Another method of rate limiting you can utilize with this information is to group by ‘None’ instead of ‘ClientAddr’ or ‘GeoLocation’. This approach groups all traffic together and counts it against the threshold of the rate limit rule you set up. Since the metric shows total WAF requests, you can use this group-by option to set the threshold against all traffic without maintaining counters for each client IP address or geography. Keep in mind that this is a powerful setting, and you should be careful when configuring it, as it could block legitimate traffic to your resources.

As an alternative approach, you have the option to set up rate limiting based on geo-location, which clusters traffic based on the geographical origin of their source IP addresses. By using the Azure DDoS Protection mitigation logs, you can pinpoint the countries from which the attacks originate and subsequently fine-tune your rate limiting rules accordingly. To find the post-mitigation logs, run the query below:

AzureDiagnostics

| where Category == “DDoSMitigationReports”

| where ReportType_s == “Post mitigation”

By leveraging the Post-Mitigation Report logs, you gain valuable insights into the countries of origin for the source IPs, along with other useful details such as top source ASNs (Autonomous System Numbers), top continents, drop reasons, and protocols. This information can be used in configuring rate limiting based on geographic locations, utilizing the top source countries data.

Investigating WAF metrics and logs

Navigate to your Application Gateway metrics tab and add these two metrics, “WAF Total Requests” and “ WAF Custom Rule Matches”,  to get a view on total requests inspected by WAF and the custom rules hit. As you can see below there’s an increase in matched custom rules due to rate limiting

To confirm that rate limiting is actively working, we can investigate WAF logs by running the following query:

AzureDiagnostics

|where Category == “ApplicationGatewayFirewallLog”

|where priority_d ==  30  //Replace 30 with your rate limiting custom rule priority

Benefits from combining Azure DDoS Protection with Azure Web Application Firewall rate limiting

Comprehensive Protection: You have multi-layered security, addressing both network-level and application-level threats.
Customization: You can fine-tune your rate limiting rules to suit your application’s unique requirements.
Visibility: Azure provides detailed traffic telemetry and analytics, allowing you to gain insights into potential threats.

Rate limiting on Azure Front Door WAF

The concepts explained for Application Gateway rate limiting in this post are also applicable to Azure Front Door WAF rate limiting. Azure Front Door (AFD) offers rate limiting capabilities as part of its Web Application Firewall (WAF) features. This allows you to control the number of requests a user can make to your application within a set time frame, effectively protecting against Layer 7 DDoS attacks. The rate limiting is configured through custom WAF rules, where you can specify the threshold for the number of web requests allowed from each socket IP address within a period of one or five minutes. Additionally, you can set up multiple rate limits for different paths within your application to ensure comprehensive protection.

This approach ensures that the rate limiting strategies discussed for Application Gateway in this blog post are equally applicable and effective when implemented on Azure Front Door WAF, offering a robust solution for your application’s security needs

Conclusion

Protecting your applications and data from DDoS attacks is a top priority in today’s digital landscape. Azure DDoS Protection, combined with Azure Web Application Firewall rate limiting, offers a powerful defense strategy. By implementing these services on either Application Gateway or Azure Front Door, you can protect your resources, maintain high availability, and provide a secure online experience for your users.

Resources

Rate Limiting Feature for Azure WAF on Application Gateway now in Preview. – Microsoft Community Hub

Application DDoS protection – Azure Web Application Firewall | Microsoft Learn

Azure DDoS Solution for Microsoft Sentinel – Microsoft Community Hub

Create rate limiting custom rules for Application Gateway WAF v2 (preview) – Azure Web Application Firewall | Microsoft Learn

Configure Azure DDoS Protection diagnostic logging through portal | Microsoft Learn

​Microsoft Tech Community – Latest Blogs –Read More 

In our previous blog, we explored how Power BI can complement Azure Workbook for consuming and visualizing data from Microsoft Defender for Cloud (MDC). In this second installment of our series, we dive into a common limitation faced when working with Azure Resource Graph (ARG) data – the 1000-record limit – and how Power BI can effectively address this constraint to enhance your data analysis and security insights.

The 1000-Record Limit: A Bottleneck in Data Analysis

When querying Azure Resource Graph (ARG) programmatically or using tools like Azure Workbook, users often face a limitation where the results are truncated to 1000 records. This limitation can be problematic for environments with extensive data, such as those with numerous subscriptions or complex resource configurations. Notably, this limit does not apply when accessing data through the Azure Portal’s built-in Azure Resource Graph Explorer, where users can query and view larger datasets without restriction. This difference can create a significant bottleneck for organizations relying on programmatic access to ARG data for comprehensive analysis.

Power BI and ARG Data Connector: Breaking Through the Limit

One of the key advantages of using Power BI’s ARG data connector is its ability to bypass the 1000-record limit imposed by Azure Workbook and other similar tools. By leveraging Power BI’s capabilities, users can access and visualize a comprehensive dataset without the constraints that typically come with ARG queries.

The Power BI ARG data connector provides a robust solution by enabling the extraction of larger datasets, which allows for more detailed and insightful analysis. This feature is particularly useful for organizations with extensive resource configurations and security plans, as it facilitates a deeper understanding of their security posture.

Case Study: Porting the MDC Security Plans Coverage Workbook to Power BI

To illustrate the benefits of using Power BI to overcome the 1000-record limit, let’s walk through a practical example. We’ll use a Power BI report that ports the existing MDC workbook on Security Plans Coverage. This report showcases how Power BI can provide a more granular view of security plans across an organization.

Setting Up the Power BI Report

To make it easier for you to get started with enhancing your MDC insights, we have developed an actual Power BI report, which you can access via our GitHub repository at https://aka.ms/AArnksi. Here’s how you can set it up and start using it:

Download the Report: Navigate to the GitHub repository and download the Power BI report template file. This report has been pre-configured to connect to Azure Resource Graph and includes various visualizations to help you analyze your security plans coverage.

Connect to Azure Resource Graph:

Open the downloaded Power BI report template file in Power BI Desktop.
When prompted, authenticate with your Azure credentials to establish a connection to your Azure Resource Graph.
Ensure that the necessary permissions are in place for accessing the required ARG data.

Import Data:

The Power BI report is set up to query ARG data and import the full dataset, bypassing the 1000-record limit. You can modify the queries if needed to suit your specific requirements.
Review the imported data to ensure completeness and accuracy.

Create and Customize Visualizations:

The report includes various pre-built visualizations such as tables, charts, and maps to comprehensively represent the Security Plans Coverage.
You can customize these visualizations or add new ones to tailor the report to your organization’s specific needs. This allows you to highlight key metrics and insights relevant to your security posture.

Next sample images of the report:

By using this pre-configured Power BI report, you can quickly overcome the 1000-record limit and gain deeper insights into your security plans across multiple subscriptions and resources. The detailed visualizations provide a clear and actionable view of your security coverage, enabling you to make informed decisions and enhance your organization’s security posture.

Enhancing Security Insights

Detailed Coverage Analysis: Use Power BI to display a detailed view of security plans across multiple subscriptions and resources. This allows for a more thorough analysis compared to the truncated data typically seen in Azure Workbook. For instance, you can drill down into specific subscription or multi cloud connector to identify gaps in security coverage.

Custom Reporting: Tailor your report to include custom metrics and KPIs that are specific to your organization’s security requirements, providing actionable insights that drive informed decision-making. This customization ensures that the report aligns with your unique security policies and compliance standards.

Sharing and Collaboration

Publish and Share: Once your report is complete, publish it to the Power BI service for sharing with stakeholders. This ensures that key decision-makers have access to the comprehensive data needed for strategic planning. By making the report accessible to a broader audience, you can facilitate informed discussions and decision-making processes.

Collaborate: Utilize Power BI’s collaboration features to gather feedback and make data-driven adjustments to your security plans. This collaborative approach helps in continuously improving the security posture based on real-time insights and stakeholder input.

Conclusion

By leveraging Power BI’s ARG data connector, organizations can overcome the limitations of the 1000-record constraint and gain deeper insights into their security posture. This approach not only enhances the visibility of security plans coverage but also empowers teams with the data they need to ensure robust security management.

Stay tuned for the next installment in our blog series, where we’ll continue to explore advanced techniques and best practices for integrating Power BI with Microsoft Defender for Cloud.

Microsoft Defender for Cloud Additional Resources

Download the new Microsoft CNAPP eBook at aka.ms/MSCNAPP
Become a Defender for Cloud Ninja by taking the assessment at aka.ms/MDCNinja

Reviewers

Yuri Diogenes, Principal PM Manager, CxE Defender for Cloud

Tal Rosler, Senior PM lead, Microsoft Defender for Cloud

​Microsoft Tech Community – Latest Blogs –Read More