The Copilot for Security team is continuously enhancing threat intelligence (TI) capabilities in Copilot for Security to provide a more comprehensive and integrated TI experience for customers. We’re excited to share that the Copilot for Security threat Intelligence plugin has broadened beyond just MDTI to now encapsulate data from other TI sources, including Microsoft Threat Analytics (TA) and SONAR, with even more sources becoming available soon. 

To reflect this evolution of the plugin, customers may notice a change in its name from “Microsoft Defender Threat Intelligence (MDTI) to “Microsoft Threat Intelligence,” reflecting its broader scope and enhanced capabilities. 

Since launch in April, Copilot for Security customers have been able to access, operate on, and integrate the raw and finished threat intelligence from MDTI developed from trillions of daily security signals and the expertise of over 10 thousand multidisciplinary analysts through simple natural language prompts. Now, with the ability for Copilot for Security’s powerful generative AI to reason over more threat intelligence, customers have a more holistic, contextualized view of the threat landscape and its impact on their organization. 

New plugin name in Copilot for Security reflects broader range of capabilities

This broader range of information, delivered instantly and in-context, adds to the ability to enable different security personas to defend at machine speed and scale. For example, a customer may ask “Tell me more about the Threat actor Silk Typhoon” for the latest threat intelligence information from MDTI, including IoCs, data from mass collection and analysis, intelligence articles, Intel Profiles (vulnerabilities, threat actors, threat tooling], and guidance. Copilot for Security now also shows customers the impact of threat to their organization and which assets may be vulnerable though threat analytics and reputation information from SONAR for indicators associated with incidents and other threat activity. 

In this example, impacted asset data from Threat Analytics is available alongside MDTI intelligence for complete context about a threat and its impact on the organization.

It’s important to note that customers will only see threat intelligence associated with the products they are provisioned for. For example, a Copilot for Security customer that isn’t provisioned for Defender XDR will not see any threat intelligence from Threat Analytics. 

Conclusion  

Microsoft delivers leading threat intelligence built on visibility across the global threat landscape made possible protecting Azure and other large cloud environments, managing billions of endpoints and emails, and maintaining a continuously updated graph of the internet. By processing an astonishing 78 trillion security signals daily, Microsoft can deliver threat intelligence in Copilot for Security providing an all-encompassing view of attack vectors across various platforms, ensuring customers have comprehensive threat detection and remediation.  

If you are interested in learning more about MDTI and how it can help you unmask and neutralize modern adversaries and cyberthreats such as ransomware, and to explore the features and benefits of MDTI please visit the MDTI product web page. To learn more about Copilot for Security, visit the Tech Community page here.  

Also, be sure to contact our sales team to request a demo or a quote. Learn how you can begin using MDTI with the purchase of just one Copilot for Security SCU here. 

​Microsoft Tech Community – Latest Blogs –Read More 

My formulas are suddenly not working anymore. I copy and paste a formula and it shows the formula is there, however the value amount is wrong. It has copied the value amount from the previous cell. I can click on the formula and delete one number and then put back that same number and now it changes it to the correct value amount. Also, I created a formula for the sum of a few cells and the formula is correct. However the amount is not correct. I can highlight those cells and see the total at the bottom and it is not the total in the formula cell.

​My formulas are suddenly not working anymore. I copy and paste a formula and it shows the formula is there, however the value amount is wrong. It has copied the value amount from the previous cell. I can click on the formula and delete one number and then put back that same number and now it changes it to the correct value amount. Also, I created a formula for the sum of a few cells and the formula is correct. However the amount is not correct. I can highlight those cells and see the total at the bottom and it is not the total in the formula cell.  Read More

​

This Blog was Co-Authored by Shuangjia Lu,Research Intern at Microsoft Research & Ph.D. student at Yale University, Department of Genetics

Abstract:

In this blog, we show how we added genomics domain knowledge to large language models (LLMs), such as GPT-4 and GPT-4o through retrieval augmented generation (RAG) and fine-tuning using Azure Open AI and Azure AI Search platforms. The specific genomics knowledge we added is genetics annotation data, crucial for genetic report interpretation and disease diagnosis. Users can now query specific variants to gpt-4o and receive accurate variant annotations and interpretations supported by advanced reasoning and summarizing capabilities of LLMs.

Introduction:

What is RAG:

LLM knowledge might not be sufficient in domains you queried, or you may want your question answered using your private data. This is where RAG could help. RAG is an approach that improves accuracy and relevance in model answer generation by searching in your own data and retrieving relevant information1.

What is fine-tuning:

We refer to supervised fine-tuning (SFT). SFT is a continuous training process from a pre-trained base model, adapting the model to specific domains, tones or formats. By fine-tuning the model on a new dataset, the model weights are adjusted to generate desired responses2.

Genetic variant annotation data:

Variant annotation datasets provide detailed information about genetic variants and their biological and clinical implications. These datasets are crucial for interpreting and prioritizing potential genetic variants for diseases. Variant annotation contains several aspects including variant description and identifications, gene affected, protein affected, molecular consequences, population frequencies, clinical conditions, etc.

Methods:

We incorporated 189 million variant annotations from several datasets to gpt-4o and gpt-4 models through either RAG or fine-tuning. To assess the model’s performance, we evaluated its output accuracy.

Build RAG for GPT-4o using Azure AI Search and Azure OpenAI:

R1) Downloading data and converting to csv format: We downloaded variant annotation data (in vcf3 format) from databases. To ensure compatibility with the next step search index creation process, we converted vcf files to csv format. This conversion was achieved using bcftools4 and awk commands. Here is an example illustrating the process, including three variants in vcf format, the command used for format conversion, and the resulting output csv file.

R2) Splitting csv files into smaller files to accommodate the limitation of the Azure AI Search S1 price tier, each containing less than 4 million characters.

R3) Transferring the data to Azure Blob Storage to ensure accessibility for the Azure AI Search Service.

R4) Creating an Azure AI Search service and defining the data source as Azure Blob Storage at Azure AI Search Azure blob indexer – Azure AI Search | Microsoft Learn.

R5) Defining search index schema based on our data structure: Create an index – Azure AI Search. Creating 5 searchable and retrievable fields: “chr:pos, rsid, gene, condition, and content”.

 R6) Loading data from data resource into the search index through blob indexer Load an index – Azure AI Search.

 R7) Adding search index to GPT-4o at Azure OpenAI through ‘add your own data’   feature: Using your data with Azure OpenAI Service – Azure OpenAI | Microsoft .

Fine-tune GPT-4 at Azure OpenAI

F1) Converting annotation vcf file to jsonl prompt for GPT-4 requirements. Here’s an example of three input variants, along with the command to convert the file format and the resulting output in jsonl.

F2) Uploading 3,000 training variants and 1,000 validation variants to Azure OpenAI.

F3) Fine-tuning GPT-4 model at Azure OpenAI.

Evaluate model performance

We assessed the performance of our models using output accuracy. For fine-tuning models, to measure the model’s ability to memorize the trained information and accurately reproduce the desired output, we randomly selected 100 variants from the training sets and counted the exact matches between outputs and the true sets. For RAG models, we also randomly selected 100 variants from input datasets and counted exact matches.

Results:

Our initial testing of the base GPT-4o model revealed limited knowledge in genomics, as it returned 0 out of 100 correct gene information when queried about variants.

We made significant improvement by leveraging the power of RAG. Through RAG, we successfully incorporated 189 million variant annotations into the GPT-4o model, and we achieved 100% accuracy in all annotation fields of our test set of 100 variants. Now, when users query about variants, they not only receive accurate annotation information but also benefit from the interpretations supported by GPT-4o capabilities. Here is an example that demonstrates the improvement in model performance for the same user query when external data is incorporated through RAG. Before using RAG, the model often provided general or incorrect information. After implementing RAG, the model now delivers accurate information and can offer extra interpretations.

                                          Before RAG                                                                                                After RAG

Fine-tuning GPT-4 on variant annotation data also improved performance in some annotation fields, although the accuracy across more fields remains suboptimal. Initially, we fine-tuned GPT-4 to predict 13 annotation fields (such as ID, gene, disease name, etc.) using chromosome position information (e.g. chr16:14555693) provided by users. Despite testing multiple input formats and repeat strategies, the average output accuracy for each field was still around 0.2. To better understand how to improve fine-tuned model performance, we adjusted our approach and fine-tune for a single field. Specifically, we fine-tuned the model only on the gene field, resulting in 95% accuracy. However, when we expanded to predict multiple fields simultaneously, the accuracy significantly decreased. This led us to conclude that the more information we added and the less frequency of information occurrence, the more challenging for model to learn through fine-tuning.

Discussion:

In our exploration, we have found that RAG outperforms supervised fine-tuning in adding factual information to LLMs in terms of data amount, accuracy and cost-effectiveness. Moving forward, to expand the application scenarios and incorporate more genomics and clinical data that have complex relationships, we aim to explore embedding strategy and GraphRAG. To further improve the model’s performance and usability in genomics, we are considering enhancing basic genomics knowledge of the LLM through unsupervised learning or pretraining. Additionally, we will explore the use of small language models, such as phi-3, to ensure the secure utilization of private data the model.

As the first model for adding genomics information to LLMs, our model paves the way for developing a more comprehensive and helpful genomics AI to support clinical diagnoses and research projects, and it demonstrates the potential for LLM in specialized domains.  

Disclaimer:

While we strive to ensure the accuracy and reliability of the information generated by this large language model, we cannot guarantee its completeness or correctness. Users should independently verify any information and consult with experts in the respective field to obtain accurate and up-to-date advice. We used fully public datasets from Microsoft Genomics Data Lake [5] and did not used any personal identification data.

Resources:

Learning more about RAG: Retrieval augmented generation in Azure AI Studio – Azure AI Studio | Microsoft Learn
Fine-tuning at Azure OpenAI: Customize a model with Azure OpenAI Service – Azure OpenAI | Microsoft Learn
Introduction of vcf file format: VCF – Variant Call Format – GATK (broadinstitute.org)
Using bcftools to extract vcf information: bcftools(1) (samtools.github.io)
Microsoft Genomics Data Lake: Genomics Data Lake – Azure Open Datasets | Microsoft Learn

​Microsoft Tech Community – Latest Blogs –Read More 

I work in supermarket retail and have to report on scheduling metrics. I show on a separate chart how many weeks in a row a store missed scheduling goals by coloring the store a certain color that represents the number of weeks. Right now I color each store cell manually.

I’m looking for a formula to color in the store certain colors based on how many weeks in a row they missed the metric goal. Example, store 473 missed meeting 65% 4 weeks in a row out of 6 weeks. I want store 473 to be automatically colored orange instead of manually coloring in orange. Store 70 missed meeting 65% 2 weeks in a row out of 6 weeks, I want to color that store gray.

Any help will be appreciated!

​I work in supermarket retail and have to report on scheduling metrics. I show on a separate chart how many weeks in a row a store missed scheduling goals by coloring the store a certain color that represents the number of weeks. Right now I color each store cell manually.I’m looking for a formula to color in the store certain colors based on how many weeks in a row they missed the metric goal. Example, store 473 missed meeting 65% 4 weeks in a row out of 6 weeks. I want store 473 to be automatically colored orange instead of manually coloring in orange. Store 70 missed meeting 65% 2 weeks in a row out of 6 weeks, I want to color that store gray. Any help will be appreciated!  Read More

​

Hi 
I dont know if this is been addressed in the past or if someone could help me.

I am trying to sum or count the times one item shows in a list but has values between one amount and the other. For example, there are three columns: one has phone numbers, the other shows the area they belong to, and the third shows how many times that number has been used. I need to calculate, for example, how many calls are between 0 and 10 times for one area, one for calls between 11 and 30, etc. I appreciate the help; below is an example of the table.

Toll-Free NumberMarket AreaCompleted Calls  Total between 0 and 10total between 11 and 30Phone number 1Market Area 11 Market Area 1????????Phone number 2Market Area 11    Phone number 3Market Area 12    Phone number 4Market Area 13    Phone number 5Market Area 13    Phone number 6Market Area 14    Phone number 7Market Area 15    Phone number 8Market Area 16    Phone number 9Market Area 17    Phone number 10Market Area 110    Phone number 11Market Area 111    Phone number 12Market Area 115    Phone number 13Market Area 123    Phone number 14Market Area 128    Phone number 15Market Area 129    

​Hi I dont know if this is been addressed in the past or if someone could help me.I am trying to sum or count the times one item shows in a list but has values between one amount and the other. For example, there are three columns: one has phone numbers, the other shows the area they belong to, and the third shows how many times that number has been used. I need to calculate, for example, how many calls are between 0 and 10 times for one area, one for calls between 11 and 30, etc. I appreciate the help; below is an example of the table. Toll-Free NumberMarket AreaCompleted Calls  Total between 0 and 10total between 11 and 30Phone number 1Market Area 11 Market Area 1????????Phone number 2Market Area 11    Phone number 3Market Area 12    Phone number 4Market Area 13    Phone number 5Market Area 13    Phone number 6Market Area 14    Phone number 7Market Area 15    Phone number 8Market Area 16    Phone number 9Market Area 17    Phone number 10Market Area 110    Phone number 11Market Area 111    Phone number 12Market Area 115    Phone number 13Market Area 123    Phone number 14Market Area 128    Phone number 15Market Area 129      Read More

​

Nonprofit Leader sitting at desk handling daily operations

Are you ready to embark on the journey of digital transformation? Looking for innovative tools and solutions created for nonprofits in mind? Microsoft’s nonprofit program empowers organizations towards leveraging powerful tools and resources that can amplify your impact. This guide will meticulously guide you step by step through enrolling your organization into the program. You will need to navigate to Microsoft’s Nonprofit Hub to sign up. The Nonprofit Hub allows you to check your eligibility, see the current nonprofit offers, and convert your Microsoft Business Licenses when signing up. If you need any assistance with getting signed up, please contact Nonprofit Support | Contact Us. 

Sign Up to Microsoft Nonprofit Program

Image of Microsoft Nonprofit Portal Hub website

Check Your Eligibility

Navigate to Microsoft’s Nonprofit Hub.
Check your eligibility under “Eligibility pre-check,” then select “Start.“
Then select the “Organization location” and “Primary activity.”
Select Yes or No if your organization is recognized by your regional government’s relevant authorities.
Then press the “Continue” button.
You will see if your eligible to apply for the program.

Creating Your Microsoft Nonprofit Account

To sign up you will need to create your free Microsoft Nonprofit Account by clicking on the “Apply” button at the bottom-left corner of the screen.
In Step 1 – Tell us about yourself, fill out your “Organization Point of Contact”
If you already have a Microsoft 365 Business license, click on the “Convert” button, then sign in with your Microsoft 365 Business Account.
Agree to Microsoft & TechSoup privacy, terms, and conditions.
Select “Continue” to proceed.
Verify your email by submitting the code sent to your email.
Fill out your nonprofit organization’s contact information, address, and nonprofit identification number (EIN for US nonprofits or International Governmental Identifier).
Lastly, create your organization’s Microsoft Nonprofit Account by providing your organization’s information and creating your nonprofit account login. Please keep this information for your records.
Verify your phone number and finish creating your account.
Congratulations! You’ve finished applying. You will receive approval notification within 3-7 business days. 

Additional Resources

Microsoft Nonprofit Offers Support | Contact Us:
Microsoft Nonprofit Offers Eligibility:
Microsoft Nonprofit Offers | Frequently Asked Questions:

​Microsoft Tech Community – Latest Blogs –Read More