Hello!

I would like to get some help regarding a Pivot Table error I came across.

So I have the following columns given:

I would like to calculate the DIH column by a calculated field/measure. The formula should be :

– ln(In Stock)/ln(1-Withdraw Rate) = x

– ln(1864)/ln(1-0,15…) = 46

When I type it inside my Pivot Table the results are way different than they should be.

I should get a result of 46 [months] instead I get 2,83.

Now the question is how can I solve this?

Thank you a lot in advance 

​Hello!I would like to get some help regarding a Pivot Table error I came across.So I have the following columns given:   I would like to calculate the DIH column by a calculated field/measure. The formula should be :- ln(In Stock)/ln(1-Withdraw Rate) = x – ln(1864)/ln(1-0,15…) = 46 When I type it inside my Pivot Table the results are way different than they should be.I should get a result of 46 [months] instead I get 2,83.Now the question is how can I solve this?Thank you a lot in advance    Read More

​

By Shawn Hays

The National Institute of Standards and Technology (NIST) published the first version of its Cybersecurity Framework (CSF) in 2014. Ten years later NIST released the second iteration of CSF, entitled NIST CSF 2.0. Microsoft and its partners have supported organizations in implementing the original CSF guidance, going as far as building and enhancing an assessment in Microsoft Purview Compliance Manager since 2018. This blog and series will look to apply NIST CSF 2.0 to Microsoft 365 and discuss changes from the previous publication.

It is somewhat improper to look at any particular CSF Functions in a vacuum or singular vantage point. NIST CSWP 29 (the primary document) illustrates and describes CSF Functions as “a wheel because all of the Functions relate to one another. For example, an organization will categorize assets under IDENTIFY and take steps to secure those assets under PROTECT. Investments in planning and testing in the GOVERN and IDENTIFY Functions will support timely detection of unexpected events in the DETECT Function, as well as enabling incident response and recovery actions for cybersecurity incidents in the RESPOND and RECOVER Functions. GOVERN is in the center of the wheel because it informs how an organization will implement the other five Functions.”

Read the full post here: NIST CSF 2.0 – Protect (PR) – Applications for Microsoft 365 (Part 1)

​By Shawn Hays
 

 
 
The National Institute of Standards and Technology (NIST) published the first version of its Cybersecurity Framework (CSF) in 2014. Ten years later NIST released the second iteration of CSF, entitled NIST CSF 2.0. Microsoft and its partners have supported organizations in implementing the original CSF guidance, going as far as building and enhancing an assessment in Microsoft Purview Compliance Manager since 2018. This blog and series will look to apply NIST CSF 2.0 to Microsoft 365 and discuss changes from the previous publication.
 
It is somewhat improper to look at any particular CSF Functions in a vacuum or singular vantage point. NIST CSWP 29 (the primary document) illustrates and describes CSF Functions as “a wheel because all of the Functions relate to one another. For example, an organization will categorize assets under IDENTIFY and take steps to secure those assets under PROTECT. Investments in planning and testing in the GOVERN and IDENTIFY Functions will support timely detection of unexpected events in the DETECT Function, as well as enabling incident response and recovery actions for cybersecurity incidents in the RESPOND and RECOVER Functions. GOVERN is in the center of the wheel because it informs how an organization will implement the other five Functions.”
 
Read the full post here: NIST CSF 2.0 – Protect (PR) – Applications for Microsoft 365 (Part 1)  Read More

​

The rapid growth of SaaS apps makes it challenging to gain visibility across the apps used in an organization’s environment. SaaS apps are often used without the awareness of IT departments, a phenomenon known as Shadow IT. Moreover, the swift adoption of generative AI apps introduces an additional layer of security complexity and risks. Organizations need effective app security solutions more than ever to ensure that employees only access approved and safe apps.

A key aspect of combating Shadow IT is ensuring comprehensive visibility into all the apps used in an organization’s environment, which includes the identification of all SaaS apps used by employees, regardless of the operating system. This broad scope of app discovery is essential in providing a holistic view of your SaaS landscape to help security teams discover and control Shadow IT effectively.

Gain full visibility into your SaaS apps with Microsoft Defender for Cloud Apps
Microsoft Defender for Cloud Apps offers powerful discovery capabilities to help you gain visibility into all SaaS apps, including AI apps, in your organization, assess their risk, and leverage sophisticated analytics to enhance the overall SaaS security. Previously, cloud discovery discovered SaaS apps in Windows devices through the network logs via defender for endpoint integration. To continuously improve our product experience and deliver a broader scope of app discovery, today we are thrilled to announce the public preview of our new comprehensive discovery capabilities within Microsoft Defender for Cloud Apps to help customers:

Discover Shadow IT SaaS apps on macOS devices in addition to windows
Detect and remediate risky/unusual behaviors on SaaS apps on macOS devices

This discovery capabilities on macOS work seamlessly with the native Microsoft Defender for Endpoint. This enhancement not only bolsters the discovery process but also improves the overall discovery comprehensiveness for security administrators, facilitating effective management of SaaS security risks.

For customers who do not use the native Microsoft Defender for Endpoint integration with Defender for Cloud Apps, we have ensured comprehensive discovery coverage by introducing the highly requested enhancements to our log collector. Specifically, we have added two new capabilities to the Defender for Cloud Apps log collector, enabling it to operate on popular container runtimes such as Podman and AKS. With these capabilities, customers can leverage the log collector to thoroughly discover shadow IT apps on other popular distros like RedHat and AKS.

Discovery on macOS via Microsoft Defender for Endpoint

Customers need to enable the network protection component in Defender for Endpoint in order to discover shadow IT on macOS. Here is a quick guide on how to enable network protection.

Once you have the network protection enabled, to access the discovery of shadow IT SaaS apps, navigate to the Cloud discovery tab in the Microsoft Defender portal. Once there, you’ll find Defender – managed endpoints listed under the stream as shown in Figure 1. Selecting this stream presents you with a comprehensive view of discovered applications, resources, IP addresses, users, and devices information for both Windows and macOS devices within your tenant. This unified view facilitates a more complete understanding of Shadow IT across different operating systems.

Figure 1: The navigation path in the Microsoft Defender portal, under the Defender – managed endpoints stream.

Podman support for the Defender for Cloud Apps log collector

The log collector, a container image that traditionally operates on Docker runtime, now extends its support to Podman, the container runtime and orchestrator endorsed by Red Hat Enterprise Linux (RHEL) 8 and subsequent versions. This expansion ensures that the Defender for Cloud Apps log collector is compatible with Podman’s runtime, accommodating customers who use RHEL versions 8 or higher.

The Defender for Cloud Apps log collector is extremely useful for organizations not using the integration with Defender for Endpoint, or a direct integration with a built-in network device isn’t available. It helps to integrate logs from various network appliances, such as firewalls, to generate a discovery report that is pivotal for administrators to pinpoint Shadow IT apps and gather usage telemetry.

Log collector in Azure Kubernetes (AKS)

The log collector feature in AKS enhances Defender for Cloud Apps by enabling the collection of logs from network devices that lack built-in integration. This feature is particularly beneficial for customers who prioritize disaster recovery and resilience for their services hosted on AKS.

With the introduction of log collector support for AKS, Defender for Cloud Apps has expanded its capabilities to include support for Syslog-tls receiver types. Once the log collector deploys log collector on AKS, Customers can configure log sources on AKS and get insights into Shadow IT on AKS workloads.

Our latest enhancements to the shadow IT discovery capabilities in Defender for Cloud Apps empower security administrators with comprehensive app discovery across both Windows and macOS and offer better compatibility and flexibility in deployment. Furthermore, Defender for Cloud Apps is a core component of Microsoft Defender XDR, and these new advancements contribute to delivering a more streamlined and easier to deployable XDR platform that enables security teams to more efficiently protect organizations from today’s advanced cyberthreats.

Getting started

Enable discovery on MacOS by integrating with Microsoft Defender for Endpoint
Configure automatic log upload using Podman
Deploy Log Collector on AKS

​Microsoft Tech Community – Latest Blogs –Read More 

Native logical replication is a built-in feature which was introduced in PostgreSQL 10. It is aimed to replicate data between tables in two separate Postgres databases using a replication identity, usually a primary key. It follows a publisher and subscriber model where there can be one or more publishers or subscribers. The publisher and subscriber databases can be on the same Azure Database for Postgres – flexible server or on two different servers.  

The typical use-cases for logical replication are: 

Sending incremental changes in a single database or a subset of a database to subscribers as they occur. 
Sending the change to subscribers could also help in event driven architectures where the subscriber can process and publish the change to, for example, message brokers like Kafka etc. 
Consolidating multiple databases into one data source (for analytical purposes). 
Replicating data between different major versions of PostgreSQL. 
Replicating between PostgreSQL instances on different platforms (for example Linux to Windows) 
Replicating data between servers hosted in different clouds or between on-prem and cloud. 

A typical flow involves copying the initial data snapshot from the publisher database to subscriber database and, once the data snapshot copy is completed, any incremental changes on the publisher database are sent to the subscriber database in near real-time. 

Using a use-case scenario this blog will share tuning techniques that can be used to improve the performance of the initial data snapshot copy from the publisher to the subscriber database. 

The blog is further divided into the following sections: 

Server parameters
Use-case scenario
Best practices

Server parameters

The parameters that are used to configure and tune logical replication are as follows: 

wal_level: Sets the level of information written to the Write Ahead Log (WAL) to logical.

 max_worker_processes: Sets the maximum number of background processes that the system can support. The default value is 8. You can set this value to a number which is greater than the number of subscribers and reserve some for-table data synchronization and parallel apply worker.

max_logical_replication_workers: Specifies maximum number of workers to replicate table data between publisher and subscriber. It includes leader apply workers, parallel apply workers, and table synchronization workers. The default value is 4. Logical replication workers are taken from the pool defined by max_worker_processes. 

max_replication_slots: The default is 10. You can set this parameter depending on the number of publishers and subscribers. It can be influenced by the value set to parameter max_sync_workers_per_subscription.

max_sync_workers_per_subscription: Controls the amount of parallelism of the initial data copy during the subscription initialization or when new tables are added. Currently, there can be only one synchronization worker per table. The workers are dependent on max_logical_replication_workers. The default value is 2. 

The recommended value is generally about 50% – 75% of the number of vCores on the subscriber server. For example, if the subscriber server is an instance of Azure Database for PostgreSQL Flexible Server running on a Memory Optimized tier, and with an SKU of 16 vCores, then the value of the parameter can be set to 8. This indicates 8 tables can be copied parallelly during initial data copy. To make this change you need to set max_replication_slots, max_worker_processes and max_logical_replication_workers parameters also. A good value for max_worker_processes could be 16 (one worker per one core of CPU), max_replication_slots could be 16 (8 for table data synchronization + additional slots for other purposes) and max_logical_replication_workers could be 12 (for table data synchronization + parallel apply worker workers for in-flight transactions). 

Note we recommend 50% -75% of total vCores because, if the number of workers doing initial copy is equal to number of vCores, then you could see 100% CPU utilization on the subscriber server. 50%-75% is just a general recommendation, you can set a higher value but making it too aggressive might result in 100% CPU utilization during initial data copy process on subscriber servers.

max_wal_senders: The maximum number of simultaneously running WAL sender processes. Default is 10. It is recommended to set a value equal to or greater than the number of logical replication slots. 

Use-case scenario

In the use case scenario, the publisher database has 6+ tables on Postgres version 16, and the goal is to complete the initial snapshot as fast as possible. 

The publisher server is 16 vCore General Purpose Azure Database for Postgres – flexible server with 1 TB storage, and the subscriber server is also a 16 vCore General Purpose Azure Database for Postgres – flexible server with 1 TB storage.

Setup On publisher server:

Server parameters: 

wal_level: Logical

max_replication_slots: 12

Execute the following statements on the publisher database: 

Grant required permissionsALTER ROLE <role name> WITH REPLICATION;

Note: Please make sure the replication user has read permissions on tables you are trying to replicate. 

Create a publicationCREATE PUBLICATION publisher1 FOR TABLE member_claims, member_claims_May,member_claims_April,member_claims_March,member_claims_Feb,member_claims_January;

Setup on subscriber server:

Server parameters: 

wal_level: Logical
max_logical_replication_workers: 12
max_worker_processes: 16
max_sync_workers_per_subscription: 6
Autovacuum: OFF
max_wal_size: 64 (GB)
checkpoint_timeout: 1(Hour)

Execute the following statements on the subscriber database: 

 Create a subscriptionCREATE SUBSCRIPTION <subscription name> CONNECTION’host=xx dbname=xx user=xx password=xx’ PUBLICATION <publisher name>;

Monitoring

Publisher database 

Execute the following statement SELECT application_name, wait_event_type, wait_event, query, backend_type FROM pg_stat_activity WHERE state=’active’;

The output should look like this where 6 tables are being copied parallelly

Subscriber database 

Execute the following statement to monitor the progress of the initial data copy. SELECT * FROM pg_stat_progress_copy;

         For more details on pg_stat_progress_copy you can follow the document COPY-PROGRESS-REPORTING.

Best Practices

In general, it is recommended to have a publisher database and subscriber database on two different servers. 
Follow the server parameter guidelines described above. 
During the initial data copy phase, you can turn OFF autovacuum, increase max_wal_size to 64 GB if you have a storage size of 1 TB or greater than that [Generally recommended for large storage sizes] and have checkpoint_timeout parameter set to 1 Hour.  [We would like the checkpoint to happen less frequently as checkpoint process consumes additional server resources for that purpose 1 hour is recommended. This is again a general recommendation value can be less than 1 hour, it depends on data volume being moved and you can also take into consideration time to recover in case there is a crash during initial data copy phase before coming up with a value] 

 Note: Once the initial copy is completed it is recommended to vacuum analyze the tables and then turn back autovacuum to ON. The checkpoint_timeout and max_wal_size parameters should also be changed back to previous values.

 vacuumdb utility can be used to manually vacuum analyze the tables. For more information follow the document – vacuumdb.

Conclusion

Initial data copy after logical replication is set up between publisher and subscriber databases can be time consuming, particularly when the database size is large with hundreds of tables or database size is large with few large tables. With proper tuning of server parameters mentioned in the blog and having the publisher and subscriber databases on separate servers, you can help reduce the time required significantly.

If you have questions, please contact us at ASK Azure DB for PostgreSQL.

​Microsoft Tech Community – Latest Blogs –Read More 

How to check token consumption?
Response got from MS copilot support team: CoPilot has certain limits in place to manage the usage of LLM tokens. These limits help prevent overloading the system and ensure fair resource allocation. Specifically, users are allowed a certain number of LLM tokens per hour. If this limit is reached, they’ll need to wait before continuing their interaction with CoPilot.

It’s important to note that the calculation of LLM tokens isn’t always straightforward—it doesn’t directly correlate with the user’s input prompt. CoPilot employs complex logic to determine token usage. If you encounter any throttling, consider optimizing your prompt and reducing overall token consumption”

​How to check token consumption?Response got from MS copilot support team: CoPilot has certain limits in place to manage the usage of LLM tokens. These limits help prevent overloading the system and ensure fair resource allocation. Specifically, users are allowed a certain number of LLM tokens per hour. If this limit is reached, they’ll need to wait before continuing their interaction with CoPilot.It’s important to note that the calculation of LLM tokens isn’t always straightforward—it doesn’t directly correlate with the user’s input prompt. CoPilot employs complex logic to determine token usage. If you encounter any throttling, consider optimizing your prompt and reducing overall token consumption”  Read More

​