By Julie Sanford, Vice President, Partner GTM, Programs & Experiences

As malicious actors continue to intensify their use of AI, security professionals must also incorporate AI into their solutions to counter the increasing threat. To support our partners and customers in securing their businesses, we are excited to announce the general availability of Microsoft Copilot for Security in all commerce channels, including CSP, on April 1, 2024. This new Copilot provides Microsoft partners with a powerful resource to safeguard their organizations, while improving the security services and solutions they offer. 

Copilot for Security is the first generative AI security product designed to defend organizations at the speed and scale of AI. This announcement continues our AI momentum following the recent general availability of Copilot for Microsoft 365, Copilot for Finance, Copilot for Sales, Copilot for Service, and Copilot for the education market. 

We have also added new resources for Copilot for Microsoft 365 ready to help you deliver more value, adoption, and seat growth with customers. Read our blog for these updates. 

Copilot for Security is designed to complement, rather than replace human skills. Our partners bring their experience, skills, and established methods for dealing with vulnerabilities. This new tool enables them to apply their expertise to services and offerings that AI solutions without human insights cannot match.  

Continue reading here

​Microsoft Tech Community – Latest Blogs –Read More 

What’s new in Defender: How Copilot for Security can transform your SOC

Today at Secure, we announced that Microsoft Copilot for Security will be generally available on April 1.  Copilot equips security teams with purpose-built capabilities at every stage of the security lifecycle, embedded right into the unified security operations platform in the Defender portal. Early users of Copilot for Security have already seen significant measurable results when integrated in their SOC, transforming their operations and boosting their defense and posture against both ongoing and emerging threats. Read on to learn about the capabilities to GA on 4/1 embedded in the Defender portal for Defender XDR and Microsoft Sentinel data and how early access customers are already enjoying its value.

Prevent breaches with dynamic threat insight

Copilot for Security leverages the rich portfolio of Microsoft Security products to produce enriched insights for security analysts in the context of their workflow. At GA, you will be able to use Copilot for Security with Microsoft Defender Threat Intelligence and Threat Analytics in the Defender portal to tap into high-fidelity threat intelligence on threat actors, tooling and infrastructure and easily discover and summarize recommendations specific to your environment’s risk profile, all using natural language. These insights can help security teams improve their security posture by prioritizing threats and managing exposures proactively against adversaries, keeping their organizations protected from potential breaches.

Identify and prioritize with built-in context

“Copilot for Security is allowing us to re-envision security operations. It will be critical in helping us close the talent gap.” Greg Petersen Sr. Director – Security Technology & Operations, Avanade 

Automation of common manual tasks with Copilot frees up analyst time and allows them to focus on more complex and urgent demands. For example, analysts need to understand the attack story and impact to determine next steps, and this often requires time and effort to collect and understand all of the relevant details. To make this task faster and easier, Copilot’s incident summaries, with AI-powered data processing and contextualization, provides this content readily available, saving significant triage time. Complimenting Microsoft Defender XDR’s unique ability to correlate incidents from a variety of workloads, Copilot’s incident summary provides the attack story and potential impact directly in the incident page. At GA, asset summaries become available for use in investigation. The first of these is a device summary, where Copilot provides highlights about the device based on all cross-workload information available in Defender XDR, as well as other device data integrated in from Intune. This further improves efficiency during triage and enables analysts to more quickly assess and prioritize incidents, leading to faster response.

As part of incident investigation and response, analysts often reach out to employees to get more information about unusual activity on their devices or to communicate about an incident or a limitation in access. New at GA, Copilot now makes this faster by generating tailored messages with all the details an employee would need and enabling analysts to send those messages through Microsoft Teams or Outlook – directly from the portal. Copilot links directly to many tasks that would normally require going to another view or product – another example of added efficiency for security teams.

During Early access, 97%* of security professionals reported they would make consistent use of Copilot capabilities in their day-to-day workflows.

Accelerate full resolution for every incident

“Copilot for Security can democratize security to the end user. It is no longer just with the subject matter expert. The average analyst training time used to be a couple of months, and that can reduce drastically if you’re using Copilot.”  Chandan Pani, Chief Information Security Officer, LTIMindtree 

During an incident, every second counts. With additional Copilot capabilities, like guided response and automated incident reports, analysts of all levels can move an average of 22% faster* and accelerate time to resolution.

Guided response, provided by Copilot during incident investigation and report in the Defender portal, helps analysts determine what to do next, based on the specific incident at hand.

Example recommendations include:

Triaging an incident with a recommended classification and threat category
Steps to take to contain an incident, such as suspending a compromised account
Investigation actions, such as finding all emails that were part of a phishing campaign
How to remediate an incident, such as resetting a user’s password

Action recommendations are provided with links to the next steps, which can be taken directly in the Copilot window, reducing time spent switching views.

After successfully closing out an incident, analysts often spend time drafting reports for peers and leadership to provide a summary of the attack and remediation steps taken. Using Copilot, an incident report is easily generated with the click of a button, instantly delivering a high-quality summary ready to share or save for documentation. For GA, exporting the report to a detailed formatted PDF is now available, making for a great executive-shareable report.

Elevate analysts with intelligent assistance

“Copilot for Security allows us to quickly analyze Python and PowerShell scripts. This means that staff with less experience can quickly analyze scripts, saving valuable time in the cybersecurity area where time is so important.”  Mark Marshall, Assistant Chief Information Officer , Peel District School Board 

Security teams are made up of individuals with a variety of different skillsets and levels of experience, and as demands and requirements change, up-leveling becomes critical. It can take time and expertise to learn how to effectively manage hunting jobs or analyze malicious scripts, which many organizations simply don’t have. Copilot makes expert tasks significantly simpler, reducing the time spent onboarding new recruits and training analysts while driving faster results.

For example, Copilot assists less experienced analysts with hunting during an investigation in the Defender Portal. An analyst can now create KQL queries simply using natural language – for example just asking for “all devices that logged on in the last hour”.  The user can then choose to run the generated query or have Copilot execute them automatically. Copilot can also recommend the best filters to apply after results are surfaced or suggest common next steps. Security teams see significant benefits with this as more senior analysts are now able to delegate threat hunting projects to newer or less experienced employees.

Another task commonly reserved to more experienced analysts is reverse engineering PowerShell, Python or other scripts, often used in HumOR and other attacks, and not every team even has this  expertise. Copilot’s script analysis feature gives security teams the ability to examine these scripts easily, without needing any prior knowledge of how to do so. This feature is also into the investigation process with a button prompting a user to “analyze with Copilot” anytime an alert contains a script. The resulting analysis is a line-by-line explanation of what the script is trying to do, with excerpts from the script for each explained section. Wit this, an analyst can quickly tell if a script is potentially harmful or not. New at GA, these capabilities extend to suspicious file analysis as well (executable or other), delivering details about the file’s internal characteristics and behavior and an easy way to assess maliciousness.

Interested in getting started with Copilot for Security?

The pace of innovation in AI is moving at lightning speed and we expect many more security teams to see significant benefits of the technology with the general availability of Copilot for Security. To learn more about Microsoft Copilot for Security, click here  or contact your Microsoft sales representative.

Learn more about Copilot skills for Defender XDR announced at early access : Operationalizing Microsoft Security Copilot to Reinvent SOC Productivity

*Microsoft Copilot for Security randomized controlled trial (RCT) with experienced security analysts conducted by Microsoft Office of the Chief Economist, January 2024.

​Microsoft Tech Community – Latest Blogs –Read More 

General Availability (GA): Azure Container Apps Managed Certificates!

Managed Certificates on Azure Container Apps will allow you to create certificates free of charge for custom domains added to your container app. The service will also manage the life cycle of these certificates and auto-renew them when they’re close to expiring.

To learn more, see Azure Container Apps managed certificate documentation.

​Microsoft Tech Community – Latest Blogs –Read More 

Sync Up Episode 9 is now available on all your favorite podcast apps! This month, Arvind Mishra and I are talking with Liz Scoble and Libby McCormick about the power of Create.Microsoft.com and how we’re bringing that power into the OneDrive experience! Along the way, we learn a little more about ourselves, about TPS reports, and much more!

Show: https://aka.ms/SyncUp | Apple Podcasts: https://aka.ms/SyncUp/Apple | Spotify: https://aka.ms/SyncUp/Spotify | RSS: https://aka.ms/SyncUp/RSS

As always, we hope you enjoyed this episode! Let us know what you think in the comments below!

​Microsoft Tech Community – Latest Blogs –Read More 

​AOL, Yahoo, and Verizon (using the same email server) are now requiring that SPF, DKIM, and DMARC be enabled for emails sent to those addresses. I saw a report that Google will soon require this too. My problem is that to enable DKIM, I have to create 2 CNAME records. I have created these records on 4 accounts on 4 different providers. I have another account that actually works, but DKIM was enabled even without the CNAMES. I also passed the DMARC check on that account, so I know that I have managed to put in the correct entries in the CNAME records. But, I still get the error message about needing the CNAME records before enabling DKIM. Another curiosity is that on none of my accounts can I detect CNAME records, even CNAMEs that were already there? What do I have to do to enable DKIM on these accounts (a total of 6 accounts on 4 different hosts)?  Read More

​

Hello, 

We have been trying for many months now to publish our PowerApps canvas apps as a SaaS offer, thereby allowing us to have transactable, licensed customers. 

Through a lot of effort, we managed to get some support where we were told quite vaguely how to achieve this, but without any meaningful guidance we are struggling as we are a small ISV with expertise in our actual business, and Power Platform. We are not, however, experts in SQL (that’s being generous) or in creating APIs which seem to be a requirement to keep track of user licenses and feeding the data to our canvas app. 

If anyone here has any info on this, or how they might have achieved it themselves it would be massively appreciated, as we can’t seem to get there by ourselves and MS support is very difficult to find. 

It does feel as though the app development part is so much simpler than getting it onto the marketplace, which is surely the wrong way round. 

Thank you,

Craig 

​Hello, We have been trying for many months now to publish our PowerApps canvas apps as a SaaS offer, thereby allowing us to have transactable, licensed customers.  Through a lot of effort, we managed to get some support where we were told quite vaguely how to achieve this, but without any meaningful guidance we are struggling as we are a small ISV with expertise in our actual business, and Power Platform. We are not, however, experts in SQL (that’s being generous) or in creating APIs which seem to be a requirement to keep track of user licenses and feeding the data to our canvas app.  If anyone here has any info on this, or how they might have achieved it themselves it would be massively appreciated, as we can’t seem to get there by ourselves and MS support is very difficult to find.  It does feel as though the app development part is so much simpler than getting it onto the marketplace, which is surely the wrong way round.  Thank you,Craig   Read More

​

Hello everyone!

I just won a PRO ticket for the Paris RAISE Summit 2024.
The ticket price on the official website is €799, and I want to sell it for half price.
Can you please help me with where I can sell it, or maybe someone from here wants to buy it?

​Hello everyone!I just won a PRO ticket for the Paris RAISE Summit 2024.The ticket price on the official website is €799, and I want to sell it for half price.Can you please help me with where I can sell it, or maybe someone from here wants to buy it?  Read More

​

Welcome to the third installment of this series simplifying azure Kubernetes service authentication. Part two is here Part 2  .In this third part we’ll continue from where we left off and set up cert manager, create a CA issuer, upgrade our ingress routes, register our app, and create secrets and a cookie for authentication. You can also refer to the official documentation here for some of the steps TLS with an ingress controller.

Install cert-manager Let’s Encrypt

In the previous post we uploaded cert manager images to our ACR. Now lets install the cert manager images by running the following:

# Set variable for ACR location to use for pulling images
$AcrUrl = (Get-AzContainerRegistry -ResourceGroupName $ResourceGroup -Name $RegistryName).LoginServer

# Label the ingress-basic namespace to disable resource validation
kubectl label namespace ingress-basic cert-manager.io/disable-validation=true

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install cert-manager jetstack/cert-manager –namespace ingress-basic –version $CertManagerTag –set installCRDs=true –set nodeSelector.”kubernetes.io/os”=linux –set image.repository=”${AcrUrl}/${CertManagerImageController}” –set image.tag=$CertManagerTag –set webhook.image.repository=”${AcrUrl}/${CertManagerImageWebhook}” –set webhook.image.tag=$CertManagerTag –set cainjector.image.repository=”${AcrUrl}/${CertManagerImageCaInjector}” –set cainjector.image.tag=$CertManagerTag

You should get some output and make sure the READY column is set to True.

Create a CA Issuer

A certificate authority (CA) validates the identities of entities (such as websites, email addresses, companies, or individual persons) and binds them to cryptographic keys through the issuance of digital certificates. We are using the letsencrypt CA. We can create a CA by applying a ClusterIssuer to our ingress-basic namespace. Create the following cluster-issuer.yaml file:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: MY_EMAIL_ADDRESS
privateKeySecretRef:
name: letsencrypt
solvers:
– http01:
ingress:
class: nginx
podTemplate:
spec:
nodeSelector:
“kubernetes.io/os”: linux

Now apply this yaml file by running the following kubectl command:

kubectl apply -f cluster-issuer.yaml –namespace ingress-basic

Update your ingress route

In the previous part of this series we created a FQDN which enabled us to route to our apps in the web browser via a URL. We need to update our ingress routes to handle this change. Update the hello-world-ingress.yaml as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: “true”
cert-manager.io/cluster-issuer: letsencrypt
spec:
ingressClassName: nginx
tls:
– hosts:
– hello-world-ingress.MY_CUSTOM_DOMAIN
secretName: tls-secret
rules:
– host: hello-world-ingress.MY_CUSTOM_DOMAIN
http:
paths:
– path: /hello-world-one(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80
– path: /hello-world-two(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-two
port:
number: 80
– path: /(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80
—
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress-static
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: “false”
nginx.ingress.kubernetes.io/rewrite-target: /static/$2
spec:
ingressClassName: nginx
tls:
– hosts:
– hello-world-ingress.MY_CUSTOM_DOMAIN
secretName: tls-secret
rules:
– host: hello-world-ingress.MY_CUSTOM_DOMAIN
http:
paths:
– path: /static(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80

Then apply the update:

kubectl apply -f hello-world-ingress.yaml –namespace ingress-basic

You should get some output and make sure the READY column is set to True.

Register your app in Entra ID and create a client secret

An Azure Active Directory (AAD) App referred to as Entra ID now, is an application registered in Entra ID, which allows it to interact with Azure services and authenticate users. We can then use the Entra ID App to obtain a client secret for authentication purposes. Perform the following actions to register an app and create a client secret.

In the Azure portal search for Microsoft Entra ID
Click App registrations in the left side navigation
Click new registration button
Add a name and enter your redirect URL (Web) https://FQDN/oauth2/callback
Register and take note of your Application (client) ID
Click Certificates and Secrets and click New client secret and take note of the Secret Value

Create a cookie secret and set Kubernetes secrets

Now register the following client-id, client-secret, and cookie secret. Remember this series is for educational purposes and thus may not meet all security requirements. If you need to store your secrets in a more secure location you can also refer to how to use Key Vault to do so here Key Vault. Run the following commands in PowerShell:

$cookie_secret=“$(openssl rand -hex 16)”

# or with python
python -c ‘import os,base64; print(base64.urlsafe_b64encode(os.urandom(32)).decode())’

kubectl create secret generic client-id –from-literal=oauth2_proxy_client_id=<APPID> -n ingress-basic
kubectl create secret generic client-secret –from-literal=oauth2_proxy_client_secret=<SECRETVALUE> -n ingress-basic
kubectl create secret generic cookie-secret –from-literal=oauth2_proxy_cookie_secret=<COOKIESECRET> -n ingress-basic

Create a Redis Password

 Azure uses large cookies when authenticating over Oauth2, thus it is recommended to setup Redis to handle these large cookies. For now we will create a Redis password and set the Kubernetes secret. In the next post we will install and setup Redis. Run the following command in PowerShell:

$REDIS_PASSWORD=“<YOUR_PASSWORD>”
kubectl create secret generic redis-password –from-literal=redis-password=$REDIS_PASSWORD -n ingress-basic

This ends the third post in our series. Look out for the fourth and final post.

​Microsoft Tech Community – Latest Blogs –Read More 

As we mark International Women’s Day, we celebrate the work of our partners to empower women around the world with skills for the AI economy. We know that when women have access to education, digital skills, and opportunities, they can build a better future for themselves, their families, and their communities.

Women in Digital Business

With the rise of the digital economy, women have new opportunities to start and grow a business, but still face many challenges. In low-income countries especially, female small business owners lack the competencies to develop a digital transformation strategy and implement it. In this context, equipping women entrepreneurs with digital skills is essential to the growth of their business. To tackle this challenge, Microsoft has partnered with the International Training Center of the International Labor Organization (ITCILO) to offer the Women in Digital Business (WIDB) program.

WIDB offers training programs in digital skills to women entrepreneurs who are looking to digitalize their business. By training partners all over the world in using the ILO’s platform and methodologies, the program will enable over 30,000 women-led micro and small businesses in 10+ countries to gain role-based skills, employability skills, and digital skills through online and residential training centers.  

In Colombia, our partner ImpactHub is integrating this training to support the personal and professional growth of female entrepreneurs who have been affected by armed conflict in their communities. Through a blend of business training, leadership skills development, and access to economic opportunities, ImpactHub’s implementation of this program is nurturing the potential of these women to persevere through political crisis, thereby strengthening the country’s business and social fabric.

Learn more about Women in Digital Business and how you can become a master trainer for the program.

Cybersecurity skilling

Cybersecurity roles are high-wage, high-growth jobs across every industry. Yet globally only 1 in 4cybersecurity roles are filled by women. Microsoft is proud to partner with women-focused organizations to help change that. Some of our resources, partnerships, and opportunities to support women in cyber include:

In honor of Women’s History Month, we just launched the Microsoft Cybersecurity Certification Scholarship, awarded by Women in Cloud. This scholarship equips women in the U.S. with access to industry recognized certifications, mentorship networks, and monthly job preparedness sessions. Learn more about how to apply for this scholarship at aka.ms/WiC.  
As part of the expansion of our cybersecurity skills initiative, we are partnering with Women in CyberSecurity (WiCyS) to bring their student chapters to a global audience. WiCyS student chapters receive funding, access to resources and conferences as well as networking opportunities for both students and faculty advisors. Learn more about how to create a student chapter on the WiCyS website. 
Our partner LATAM Women in Cybersecurity (WOMCY) has a mission to minimize the knowledge gap and increase the talent pool in cybersecurity across Latin America. Through multiple grants from Microsoft, WOMCY has provided 5,200 women with coursework and vouchers to complete a SC-900 certification in Cybersecurity. Find out more about WOMCY.
The International Telecommunications Union (ITU), a UN agency, recently finished the third cycle of their Women in Cyber Mentorship Program. With support from Microsoft, this program provided over 300 women mentors and mentees in the field of cybersecurity with courses, live trainings, and multiple forms of mentorship activities to foster continued growth in their roles. 

See all the resources Microsoft offers for Cybersecurity skilling at aka.ms/Cybersecurity_Skills.  

International Women’s Day is an opportunity to reflect on our progress and recognize the impact of our partners and programs around the world. But there is more work to do. Together, we can ensure women everywhere have access to the skills and opportunities they need to thrive in a rapidly changing economy.

Access digital skills resources to help empower women in your community at: aka.ms/MicrosoftDigitalSkillsHub

​Microsoft Tech Community – Latest Blogs –Read More 

Retrieval Augmented Generation (RAG) is a popular technique to get LLMs to provide answers that are grounded in a data source. When we use RAG, we use the user’s question to search a knowledge base (like Azure AI Search), then pass along both the question and the relevant content to the LLM (gpt-3.5-turbo or gpt-4), with a directive to answer only according to the sources. In psuedo-code:

user_query = “what’s in the Northwind Plus plan?”
user_query_vector = create_embedding(user_query, “ada-002”)
results = search(user_query, user_query_vector)
response = create_chat_completion(system_prompt, user_query, results)

If the search function can find the right results in the index (assuming the answer is somewhere in the index), then the LLM can typically do a pretty good job of synthesizing the answer from the sources.

Unstructured queries

This simple RAG approach works best for “unstructured queries”, like:

What’s in the Northwind Plus plan?
What are the expectations of a product manager?
What benefits are provided by the company?

When using Azure AI Search as the knowledge base, the search call will perform both a vector and keyword search, finding all the relevant document chunks that match the keywords and concepts in the query.

Structured queries

But you may find that users are instead asking more “structured” queries, like:

Summarize the document called “perksplus.pdf”
What are the topics in documents by Pamela Fox?
Key points in most recent uploaded documents

We can think of them as structured queries, because they’re trying to filter on specific metadata about a document. You could imagine a world where you used a syntax to specify that metadata filtering, like:

Summarize the document title:perksplus.pdf
Topics in documents author:PamelaFox
Key points time:2weeks

We don’t want to actually introduce a query syntax to a a RAG chat application if we don’t need to, since only power users tend to use specialized query syntax, and we’d ideally have our RAG just do the right thing in that situation.

Using function calling in RAG

Fortunately, we can use the OpenAI function-calling feature to recognize that a user’s query would benefit from a more structured search, and perform that search instead.

If you’ve never used function calling before, it’s an alternative way of asking an OpenAI GPT model to respond to a chat completion request. In addition to sending our usual system prompt, chat history, and user message, we also send along a list of possible functions that could be called to answer the question. We can define those in JSON or as a Pydantic model dumped to JSON. Then, when the response comes back from the model, we can see what function it decided to call, and with what parameters. At that point, we can actually call that function, if it exists, or just use that information in our code in some other way.

To use function calling in RAG, we first need to introduce an LLM pre-processing step to handle user queries, as I described in my previous blog post. That will give us an opportunity to intercept the query before we even perform the search step of RAG.

For that pre-processing step, we can start off with a function to handle the general case of unstructured queries:

tools: List[ChatCompletionToolParam] = [
{
“type”: “function”,
“function”: {
“name”: “search_sources”,
“description”: “Retrieve sources from the Azure AI Search index”,
“parameters”: {
“type”: “object”,
“properties”: {
“search_query”: {
“type”: “string”,
“description”: “Query string to retrieve documents from azure search eg: ‘Health care plan'”,
}
},
“required”: [“search_query”],
},
},
}
]

Then we send off a request to the chat completion API, letting it know it can use that function.

chat_completion: ChatCompletion = self.openai_client.chat.completions.create(
messages=messages,
model=model,
temperature=0.0,
max_tokens=100,
n=1,
tools=tools,
tool_choice=”auto”,
)

When the response comes back, we process it to see if the model decided to call the function, and extract the search_query parameter if so.

response_message = chat_completion.choices[0].message

if response_message.tool_calls:
for tool in response_message.tool_calls:
if tool.type != “function”:
continue
function = tool.function
if function.name == “search_sources”:
arg = json.loads(function.arguments)
search_query = arg.get(“search_query”, self.NO_RESPONSE)

If the model didn’t include the function call in its response, that’s not a big deal as we just fall back to using the user’s original query as the search query. We proceed with the rest of the RAG flow as usual, sending the original question with whatever results came back in our final LLM call.

Adding more functions for structured queries

Now that we’ve introduced one function into the RAG flow, we can more easily add additional functions to recognize structured queries. For example, this function recognizes when a user wants to search by a particular filename:

{
“type”: “function”,
“function”: {
“name”: “search_by_filename”,
“description”: “Retrieve a specific filename from the Azure AI Search index”,
“parameters”: {
“type”: “object”,
“properties”: {
“filename”: {
“type”: “string”,
“description”: “The filename, like ‘PerksPlus.pdf'”,
}
},
“required”: [“filename”],
},
},
},

We need to extend the function parsing code to extract the filename argument:

if function.name == “search_by_filename”:
arg = json.loads(function.arguments)
filename = arg.get(“filename”, “”)
filename_filter = filename

Then we can decide how to use that filename filter. In the case of Azure AI search, I build a filter that checks that a particular index field matches the filename argument, and pass that to my search call. If using a relational database, it’d become an additional WHERE clause.

Simply by adding that function, I was able to get much better answers to questions in my RAG app like ‘Summarize the document called “perksplus.pdf”‘, since my search results were truly limited to chunks from that file. You can see my full code changes to add this function to our RAG starter app repo in this PR.

Considerations

This can be a very powerful technique, but as with all things LLM, there are gotchas:

Function definitions add to your prompt token count, increasing cost.
There may be times where the LLM doesn’t decide to return the function call, even when you thought it should have.
The more functions you add, the more likely the LLM will get confused about which one to pick, especially if functions are similar to each other. You can try to make it more clear to the LLM by prompt engineering the function name and description, or even providing few shots.

Here are additional approaches you can try:

Content expansion: Store metadata inside the indexed field and compute the embedding based on both the metadata and content. For example, the content field could have “filename:perksplus.pdf text:The perks are…”.
Add metadata as separate fields in the search index, and append those to the content sent to the LLM. For example, you could put “Last modified: 2 weeks ago” in each chunk sent to the LLM, if you were trying to help it’s ability to answer questions about recency. This is similar to the content expansion approach, but the metadata isn’t included when calculating the embedding. You could also compute embeddings separately for each metadata field, and do a multi-vector search.
Add filters to the UI of your RAG chat application, as part of the chat box or a sidebar of settings.
Use fine-tuning on a model to help it realize when it should call particular functions or respond a certain way. You could even teach it to use a structured query syntax, and remove the functions entirely from your call. This is a last resort, however, since fine-tuning is costly and time-consuming.

​Microsoft Tech Community – Latest Blogs –Read More