Tag Archives: microsoft
SharePoint Notifications for Terminated Employee
Hello,
A former member of staff set up many different SharePoint notifications before they left the company. Once they left their MS Office license was removed and their email was suspended. SharePoint continues to send out notifications to this non-working email address, which is creating cases in our CRM.
How can I, as the System Admin, turn off these notifications?
Thank you
Hello, A former member of staff set up many different SharePoint notifications before they left the company. Once they left their MS Office license was removed and their email was suspended. SharePoint continues to send out notifications to this non-working email address, which is creating cases in our CRM. How can I, as the System Admin, turn off these notifications? Thank you Read More
Can’t edit signature on outlook 365 mail
Hi,
All of a sudden, I can’t edit my signature after having done so in the same place for a very long time. The option just disappeared under “compose and reply” section.
I did see this new message the top, but clicking on that “here” link doesn’t provide access.
Thanks for anyone’s help!
Hi,All of a sudden, I can’t edit my signature after having done so in the same place for a very long time. The option just disappeared under “compose and reply” section. I did see this new message the top, but clicking on that “here” link doesn’t provide access. Thanks for anyone’s help! Read More
Formatting Multiline Text fields in SPO Library
I have a multiline text column (In a library, not a list) that contains text that has characters such as , { and } that I would like to remove. each, I want to be replaced with a line break and remove characters such as {} or replace with a space. My code seems to only work on the first instance of these characters. I would love to get idea or suggestions as I know there are limitations around Multiline Text fields.
My Code:
{
“$schema”: “https://developer.microsoft.com/json-schemas/sp/v2/column-formatting.schema.json“,
“elmType”: “div”,
“style”: {
“white-space”: “pre-line”
},
“txtContent”: “=if(@currentField, replace(replace(replace(@currentField, ‘,’, ‘n’), ‘{‘, ”), ‘}’, ”), ”)”
}
I have a multiline text column (In a library, not a list) that contains text that has characters such as , { and } that I would like to remove. each, I want to be replaced with a line break and remove characters such as {} or replace with a space. My code seems to only work on the first instance of these characters. I would love to get idea or suggestions as I know there are limitations around Multiline Text fields.My Code:{ “$schema”: “https://developer.microsoft.com/json-schemas/sp/v2/column-formatting.schema.json”, “elmType”: “div”, “style”: { “white-space”: “pre-line” }, “txtContent”: “=if(@currentField, replace(replace(replace(@currentField, ‘,’, ‘n’), ‘{‘, ”), ‘}’, ”), ”)”} Read More
Formula Excel Help
I need help figuring out a formula for the below (see image). I am trying to determine if every employee has a rating for each of their goals. For example, I need a formula that would look down column A and determine that Employee A who has 3 goals (Column B) has a rating for each.
I need help figuring out a formula for the below (see image). I am trying to determine if every employee has a rating for each of their goals. For example, I need a formula that would look down column A and determine that Employee A who has 3 goals (Column B) has a rating for each. Read More
Will Microsoft’s Fundraising Template App Be Supported Long-Term?
Since Microsoft retired the Fundraising and Engagement app, they’ve introduced template apps for partners and customers to build solutions. Over the past year, we did lot of customizations on top of the original Fundraising and Engagement app, and we’re now considering moving these customizations and needed functionality to the ‘Fundraising’ template app. However, we’re concerned about long-term stability. Can Microsoft confirm if there’s any risk of these templates being retired or delisted? This would help us and others plan with greater confidence.
Since Microsoft retired the Fundraising and Engagement app, they’ve introduced template apps for partners and customers to build solutions. Over the past year, we did lot of customizations on top of the original Fundraising and Engagement app, and we’re now considering moving these customizations and needed functionality to the ‘Fundraising’ template app. However, we’re concerned about long-term stability. Can Microsoft confirm if there’s any risk of these templates being retired or delisted? This would help us and others plan with greater confidence. Read More
My laptop has been blocked by BitLocker.
However, there is no BitLocker recovery keys on my Microsoft account. I have tried to call Microsoft support, but I only get bot messages that take me to sites that asks me to go and check my Microsoft account.
Is there any way I can chat with a human that can actually help me how to get around this BitLocker?
thanks
However, there is no BitLocker recovery keys on my Microsoft account. I have tried to call Microsoft support, but I only get bot messages that take me to sites that asks me to go and check my Microsoft account. Is there any way I can chat with a human that can actually help me how to get around this BitLocker? thanks Read More
.NET Web API CancellationToken don’t work with Angular .unsuscribe()?
In Angular, I make API calls by subscribing to observables. When a user cancels a request, the application unsubscribes to the subscription which cancels the request (I can see this in the network tab of the Chrome browser). However, it does not trigger the endpoint and so the cancellation token param in our endpoint is of no use. Is there a way around this? I need the cancellation token to cancel all database request.
In Angular, I make API calls by subscribing to observables. When a user cancels a request, the application unsubscribes to the subscription which cancels the request (I can see this in the network tab of the Chrome browser). However, it does not trigger the endpoint and so the cancellation token param in our endpoint is of no use. Is there a way around this? I need the cancellation token to cancel all database request. Read More
Seeking Insights on Security Considerations for Premium APK Files (Answer)
A User Put a Question and i try my best to answer his question as:
Understanding the Risks
Premium APK files, especially those distributed outside official app stores, are inherently more vulnerable to security threats. This heightened risk stems from various factors:
Lack of Verification: Unlike official app stores, third-party sources often lack rigorous security checks.Tampering: Malicious actors can modify APKs to introduce malware, steal sensitive data, or compromise user devices.Reverse Engineering: Attackers can disassemble APKs to understand their inner workings, potentially identifying vulnerabilities.
Key Security Considerations
Source Verification:
Trusted Sources: Download APKs only from reputable sources to minimize the risk of malicious modifications.Digital Signatures: Verify the digital signature of the APK to ensure its authenticity and integrity.
Malware Scanning:
Antivirus Software: Use robust antivirus software on your device to scan downloaded APKs for malware.Online Scanners: Utilize online malware scanning tools to analyze APKs before installation.
Permission Review:
Essential Permissions: Grant only the necessary permissions to the app to avoid unnecessary risks.Suspicious Permissions: Be cautious about apps requesting excessive or unusual permissions.
Secure Installation:
Official Channels: Whenever possible, install apps from official app stores to benefit from their security measures.Side loading Precautions: If side loading is necessary, exercise extreme caution and follow best practices.
Regular Updates:
Patching Vulnerabilities: Keep your device and apps updated with the latest security patches to address known vulnerabilities.
User Awareness:
Phishing Attacks: Be wary of phishing attempts that may trick you into downloading malicious APKs.Suspicious Links and Emails: Avoid clicking on suspicious links or downloading attachments from unknown sources.
Additional Tips:
Research the Developer: If possible, research the developer’s reputation and track record.Read User Reviews: Pay attention to user reviews for insights into the app’s security and performance.Use Security-Focused Devices: Consider using devices with advanced security features like bio-metric authentication and secure boot.
By following these guidelines, you can significantly reduce the risks associated with Spotify Mod APK files and protect your device and personal information.
A User Put a Question and i try my best to answer his question as: Understanding the RisksPremium APK files, especially those distributed outside official app stores, are inherently more vulnerable to security threats. This heightened risk stems from various factors:Lack of Verification: Unlike official app stores, third-party sources often lack rigorous security checks.Tampering: Malicious actors can modify APKs to introduce malware, steal sensitive data, or compromise user devices.Reverse Engineering: Attackers can disassemble APKs to understand their inner workings, potentially identifying vulnerabilities.Key Security ConsiderationsSource Verification:Trusted Sources: Download APKs only from reputable sources to minimize the risk of malicious modifications.Digital Signatures: Verify the digital signature of the APK to ensure its authenticity and integrity.Malware Scanning:Antivirus Software: Use robust antivirus software on your device to scan downloaded APKs for malware.Online Scanners: Utilize online malware scanning tools to analyze APKs before installation.Permission Review:Essential Permissions: Grant only the necessary permissions to the app to avoid unnecessary risks.Suspicious Permissions: Be cautious about apps requesting excessive or unusual permissions.Secure Installation:Official Channels: Whenever possible, install apps from official app stores to benefit from their security measures.Side loading Precautions: If side loading is necessary, exercise extreme caution and follow best practices.Regular Updates:Patching Vulnerabilities: Keep your device and apps updated with the latest security patches to address known vulnerabilities.User Awareness:Phishing Attacks: Be wary of phishing attempts that may trick you into downloading malicious APKs.Suspicious Links and Emails: Avoid clicking on suspicious links or downloading attachments from unknown sources.Additional Tips:Research the Developer: If possible, research the developer’s reputation and track record.Read User Reviews: Pay attention to user reviews for insights into the app’s security and performance.Use Security-Focused Devices: Consider using devices with advanced security features like bio-metric authentication and secure boot.By following these guidelines, you can significantly reduce the risks associated with Spotify Mod APK files and protect your device and personal information. Read More
App attach now integrates with partner solutions
For many IT admins, application delivery management is one of their most complex tasks. However, with the recent general availability of App attach in Azure Virtual Desktop, this process has become significantly easier. App attach simplifies application delivery administration by enabling dynamic application attachment from an application package to a user session in Azure Virtual Desktop. This solution has been gaining momentum, with numerous organizations embracing it to lower the burden on admins, while enabling granular, secured application assignment to employees.
As organizations continue their digital transformation, many admins find themselves managing on-premises applications for their existing virtual desktop infrastructure (VDI) environments even as they are moving their organizations to the cloud. As a result, admins must often use multiple tools for application delivery, creating a new layer of unwelcome complexity.
We are excited to announce that Liquidware, Numecent, and Omnissa are launch partners that have integrated their application delivery solutions with App attach in Azure Virtual Desktop. Integrating the solution with three of the top application delivery providers will offer substantial benefits to admins in organizations of all sizes. Additionally, the App attach in Azure Virtual Desktop API is publicly available, allowing providers to take advantage of this integration for enhanced application delivery.
The benefits of solution integration
Solution integration offers two significant benefits: simplified application delivery management and increased application mobility.
Simplified application delivery management
Admins can more easily run and deliver apps on Azure Virtual Desktop in the Microsoft Azure cloud thanks to a streamlined tracking and delivery system. This system offers a unified management console that provides a centralized point of control for all applications, simplifying the administrative workload. With this “single pane of glass” approach, administrators can efficiently track and manage apps from one place, no matter where those apps are deployed.
Increased application mobility
Migrating on-premises apps to Azure can be a daunting task. However, by leveraging the power of the integration of App attach in Azure Virtual Desktop with the app delivery solutions provided by leading partners, the process for admins to transfer their on-premises applications to Azure is streamlined. This helps address such common challenges as data transfer, synchronization, and maintaining security and compliance standards. The integration with Azure Virtual Desktop ensures that app delivery is consistent and efficient across all environments.
Application delivery partner benefits
Liquidware FlexApp containerizes applications into self-sufficient executables, enabling rapid deployment through various methods, including popular cloud file-sharing services. It is designed for use with any Windows desktop including cloud-hosted virtualization.
“FlexApp’s seamless integration with App attach in Azure Virtual Desktop empowers organizations to manage applications without adding complexity for end users. FlexApp helps customers achieve increased compatibility for attached apps. Our technology delivers apps that feel native while reducing IT overhead, making it a game changer for IT teams and end users alike.” – Jason E. Smith, Vice President, Alliances and Product Marketing, Liquidware
Numecent Cloudpager makes it easy to create, automate, and manage desktop application configurations from the cloud. Delivered as a fully managed software-as-a-service (SaaS) solution, users can leverage the latest Windows technologies to enhance their virtual desktop environments.
“Cloudpager enables seamless provisioning of complex legacy, custom, and resource-intensive apps to any Windows desktop, including non-persistent Azure Virtual Desktop environments, securely and compliantly. App attach in Azure Virtual Desktop integration provides customers with even more flexibility around deploying applications to Microsoft Azure.” – Arthur Hitomi, Co-founder and Chief Executive Officer, Numecent
Omnissa (formerly the end-user computing division of VMware) App Volumes is a real-time delivery and lifecycle management solution that provides faster app delivery and simplified and unified management of apps and users. Enterprises can use App Volumes to centrally manage applications that are deployed to desktops via virtual disks.
“Expanding our partnership with Microsoft to integrate App Volumes with App attach in Azure Virtual Desktop adds even more value for customers, streamlining app management and cutting costs. Our teams have worked together to deliver this integration, allowing organizations using Azure Virtual Desktop to optimize their environments through real-time app delivery using App Volumes with Apps on Demand.” – Bharath Rangarajan, Chief Product Officer, Omnissa
Unlock new possibilities
App attach in Azure Virtual Desktop simplifies on-premises administration and application migration to the cloud by integrating seamlessly with third-party solutions, enhancing the overall application delivery experience. This integration provides increased security, scalability, and flexibility, ensuring a unified and efficient application delivery process across all environments. Microsoft is actively collaborating with partners to add more integrations, further improving the versatility and effectiveness of application delivery solutions.
To learn more about the third-party integration for app attach, visit the App attach documentation.
Stay up to date! Bookmark the Azure Virtual Desktop Tech Community.
Microsoft Tech Community – Latest Blogs –Read More
New data sync solution is now supported in forms created from SharePoint, OneDrive and Teams
We have been gradually updating the Forms data sync to Excel to a new solution since August, and we’re excited to announce that it’s now fully available in group forms, no matter created through Forms, SharePoint or Teams, as well as for personal forms created through Forms and OneDrive. Let’s take a look at how it works!
Create Forms survey from SharePoint and OneDrive
You can now create a form directly from SharePoint or OneDrive by adding a Forms survey, and it will automatically use the new data sync solution. For personal forms, you can start from OneDrive. For collaborative forms involving your team, start from SharePoint to keep everyone up-to-date.
Create Forms survey from Teams channels
You can also create a form from a Teams channel. There are two ways to do it: you can either go to the ‘Files’ tab in the tab bar to add a Forms survey or click the ‘+’ icon to add a Forms tab.
Redirect to Forms to create the form
No matter where you start creating the form – whether in SharePoint, OneDrive, or Teams – you’ll be redirected to Forms to create the survey and send it out.
Excel file generated accordingly
When getting enough responses, you can go to the result page to view the data in Excel. If you created the form in OneDrive, the Excel file will be generated there. For group forms created from a Teams channel or SharePoint, the Excel file will be generated in the corresponding SharePoint file library.
Sync the edited response
The new data sync solution also supports syncing edited responses. When respondents modify their answers, the changes will be updated in the Excel file.
The new data sync solution is starting to roll out in Oct and will gradually be available to all customers in the upcoming month.
Microsoft Tech Community – Latest Blogs –Read More
Update on classic storage account retirement and upcoming changes for classic storage customers
We previously announced that support would end for retired Azure classic storage accounts on 31 August 2024. Now that we are past the retirement date and Azure classic storage accounts are not supported, we have some updates to share on our plans and the considerations for customers who have not migrated their remaining classic storage accounts to Azure Resource Manager.
On or after 1 November 2024:
Your ability to perform write operations using the classic service model APIs, including PUT and PATCH, will be limited. You will only be able to perform read and list operations using the classic service model APIs.
Your remaining classic storage accounts will be migrated to Azure Resource Manager on your behalf on a rolling schedule. Your data will continue to be stored, but any applications that use the classic service model APIs to perform management plane operations will experience disruptions if you’re actively using any write operations. Write operations will only be available through the Azure Resource Manager APIs after your account(s) have been migrated.
Note: There are no impacts on the availability of the data plane APIs before, during, or after the migration of classic storage accounts.
Azure storage accounts under Azure Resource Manager provides the same capabilities as well as new features, including:
A management layer that simplifies deployment by enabling you to create, update, and delete resources.
Resource grouping, which allows you to deploy, monitor, manage, and apply access control policies to resources as a group.
To avoid service disruptions, you’ll need to migrate your classic storage accounts to Azure Resource Manager as soon as possible. Our recommendation is that customers self-service their migrations using the existing migration capabilities instead of waiting for Azure Storage to migrate any remaining classic storage accounts on your behalf. This ensures that you can migrate on your schedule and minimize impacts. We cannot guarantee that you will not experience service interruptions if you further delay your migration.
Required action
To avoid service disruptions, migrate your classic storage accounts as soon as possible. Additionally, update any management operations in code or applications that target the classic deployment model. Read our FAQ and migration guidance for more information.
Help and support
If you have questions, get answers from community experts in Microsoft Q&A. If you have a support plan and you need technical help, open the Azure portal and select the question mark icon at the top of the page.
FAQ
Q: I want to delete my account, but the operation is blocked. How do I delete my account if write operations are blocked on the classic control plane?
A: You can delete your account using the Azure Resource Manager APIs for Azure Storage after your account has been migrated from the classic service model.
Q: Will I be able to use the legacy PowerShell cmdlets to manage my classic storage accounts?
A: No. After your accounts have been migrated, they can be managed with the modern PowerShell cmdlets for Azure Storage and Azure Resource Manager.
Q: What resource group will my migrated resources appear in?
A: Any classic storage accounts that are migrated on your behalf will be placed in a new resource group. The name of the new resource group follows the pattern <classic-account-name>-Migrated. For more information on the migration process, visit Understand storage account migration from the classic deployment model to Azure Resource Manager.
Microsoft Tech Community – Latest Blogs –Read More
Project Online – 4000 characters limit for Multiple Lines of text for a Custom Field
I am wondering if there is a way to increase the limit of 4000 characters that there is for an Enterprise Custom Field that is defined as Multiple Lines of Text.
The error messages says: “When using rich text formatting, this limit depends on the extent of the formatting and may be less than 4000 characters.”
Therefore, it looks like changing the field formatting to plain instead to rich would remove this limit, but I do not know how to do it.
Thanks in advance for any help on this.
I am wondering if there is a way to increase the limit of 4000 characters that there is for an Enterprise Custom Field that is defined as Multiple Lines of Text.The error messages says: “When using rich text formatting, this limit depends on the extent of the formatting and may be less than 4000 characters.”Therefore, it looks like changing the field formatting to plain instead to rich would remove this limit, but I do not know how to do it.Thanks in advance for any help on this. Read More
Automatic appointment scheduling from Outlook
The following happened to a user. On Friday, October 18th, at around 9 p.m., he automatically sent out an older recurring appointment in Outlook on behalf of another person. What could be the cause and how can I identify this problem?
The following happened to a user. On Friday, October 18th, at around 9 p.m., he automatically sent out an older recurring appointment in Outlook on behalf of another person. What could be the cause and how can I identify this problem? Read More
How to highlight a row if different combinations of check boxes are true or false
Hi All,
I’m working in an excel sheet and I’m looking to set it up so that it highlights a row in a table depending on which check boxes are true or false. I was thinking I could use a function for the formatting but I’m not finding anything that I really understand how to implement.
This is the sheet I’m working from, and I’m struggling with how to setup the conditional formatting.
the goal is to get excel to automate like this:
I’m sure the formula is simple but I can’t find any examples that will highlight the whole row, the examples I find are only highlighting a single cell. Any guidance would be appreciated and helpful!
Hi All, I’m working in an excel sheet and I’m looking to set it up so that it highlights a row in a table depending on which check boxes are true or false. I was thinking I could use a function for the formatting but I’m not finding anything that I really understand how to implement. This is the sheet I’m working from, and I’m struggling with how to setup the conditional formatting. the goal is to get excel to automate like this: I’m sure the formula is simple but I can’t find any examples that will highlight the whole row, the examples I find are only highlighting a single cell. Any guidance would be appreciated and helpful! Read More
Tidak bisa memulai modul pembuatan keterampilan kustom untuk azure ai search
kenapa saya tidak bisa memulai modul tersebut, tidak ada tombol biru dengan bacaan mulai di dalam modul tersebut, padahal saya sudah menyelesaikan modul sebelumnya sebagai prasyarat untuk memulai modul tersebut. beriku ini modul yang mengalami masalah
https://learn.microsoft.com/id-id/training/modules/create-azure-ai-custom-skill/
kenapa saya tidak bisa memulai modul tersebut, tidak ada tombol biru dengan bacaan mulai di dalam modul tersebut, padahal saya sudah menyelesaikan modul sebelumnya sebagai prasyarat untuk memulai modul tersebut. beriku ini modul yang mengalami masalahhttps://learn.microsoft.com/id-id/training/modules/create-azure-ai-custom-skill/ Read More
DBCC CLONEDATABASE isn’t a clone in 2022
We use DBCC CLONEDATABASE currently in SQL Server 2019 and have done in previous versions too. Having recently begun a project to migrate some servers up to SQL 2022 I’ve found that the use of CLONEDATABASE is no longer giving a “clone” and instead is making some assumptions about my data and implementing (badly) the LEDGER functionality.
Example:
Say my database has 4 tables, (Table_A, Table_B, Table_C and Table_D) all have the attributes USER_TABLE, NON_TEMPORAL_TABLE and NON_LEDGER_TABLE in sys.tables.
Running DBCC CLONEDATABASE creates two of these tables as ‘HISTORY_TABLE’ types (instead of NON_LEDGER_TABLE), with one being paired to another one of the tables. You can see this relationship as the object_id of say Table_B is present in the ‘history_table_id’ column of Table_A when looking at sys.tables. The same is true of Table_C and Table_D in my scenario.
It is also apparent that on some of the columns in Table_A and Table_C (the ones still created as NON_LEDGER_TABLE types) that the clone has chosen to add ‘GENERATED ALWAYS AS ROW END’ to some of the fields.
I can’t run SYSTEM_VERSIONING = OFF as I’m presented with the error that “SYSTEM_VERSIONING is not turned ON” for those tables. I can drop and manually re-create the Table_A & Table_C which removes the ‘history_table_id’ values, but I still can’t drop Table_B and Table_D as they are still seen as a ledger history table.
I have no idea why SQL 2022 has made these choices on my behalf and I can’t see to get around them, so for now at least I am having to script out the schema of my database and create it that way which leads to its own challenges and isn’t as easy as simply specifying ‘DBCC CLONEDATABASE’ in code.
We use DBCC CLONEDATABASE currently in SQL Server 2019 and have done in previous versions too. Having recently begun a project to migrate some servers up to SQL 2022 I’ve found that the use of CLONEDATABASE is no longer giving a “clone” and instead is making some assumptions about my data and implementing (badly) the LEDGER functionality. Example:Say my database has 4 tables, (Table_A, Table_B, Table_C and Table_D) all have the attributes USER_TABLE, NON_TEMPORAL_TABLE and NON_LEDGER_TABLE in sys.tables. Running DBCC CLONEDATABASE creates two of these tables as ‘HISTORY_TABLE’ types (instead of NON_LEDGER_TABLE), with one being paired to another one of the tables. You can see this relationship as the object_id of say Table_B is present in the ‘history_table_id’ column of Table_A when looking at sys.tables. The same is true of Table_C and Table_D in my scenario. It is also apparent that on some of the columns in Table_A and Table_C (the ones still created as NON_LEDGER_TABLE types) that the clone has chosen to add ‘GENERATED ALWAYS AS ROW END’ to some of the fields. I can’t run SYSTEM_VERSIONING = OFF as I’m presented with the error that “SYSTEM_VERSIONING is not turned ON” for those tables. I can drop and manually re-create the Table_A & Table_C which removes the ‘history_table_id’ values, but I still can’t drop Table_B and Table_D as they are still seen as a ledger history table. I have no idea why SQL 2022 has made these choices on my behalf and I can’t see to get around them, so for now at least I am having to script out the schema of my database and create it that way which leads to its own challenges and isn’t as easy as simply specifying ‘DBCC CLONEDATABASE’ in code. Read More
The Future of AI: Maximize your fine-tuned model performance with the new Azure AI Evaluation SDK
The Future of AI: LLM Distillation just got easier
Part 4 – Maximize your fine-tuned model performance with the new Azure AI Evaluation SDK
By Cedric Vidal, Principal AI Advocate, Microsoft
Part of the Future of AI 🚀 series initiated by Marco Casalaina with his Exploring Multi-Agent AI Systems blog post.
Generated using Azure OpenAI DALL-E 3
In earlier posts of this distillation series, we detailed the process of distilling a Llama 3.1 405B model into a more compact Llama 3.1 8B model. This journey included generating a synthetic dataset using RAFT, as well as fine-tuning and deploying our student model on Azure AI Serverless.
But how can we confirm that our distilled model performs optimally? The crucial final step is evaluating the model.
Effective model evaluation is key to ensuring that our AI systems function as expected and meet the desired standards. With the introduction of the Azure AI Evaluation Python SDK, we now have a powerful toolkit for assessing AI models through advanced metrics. In this blog post, we’ll look at evaluating a distilled student model, which was trained with data generated by RAFT, and compare it against a baseline model.
In our setup, Llama 3.1 405B functions as the teacher, Llama 3.1 8B serves as the student model and GPT-4 serves as the judge.
Why evaluate?
Evaluating distilled student models is crucial because it allows us to assess how effectively knowledge has been transferred from the teacher model to the student model. Distillation aims to compress a larger, more complex model into a smaller, more efficient one without significantly sacrificing performance. By thoroughly evaluating the distilled models, we ensure they not only mimic the teacher model’s outputs but also maintain high levels of accuracy, coherence, and relevance. This evaluation process helps identify areas where the student model may need further fine-tuning and ensures that the distilled models are ready for deployment in resource-constrained environments where computational efficiency is paramount.
Process Overview
Evaluating the performance of our models involves several key steps, which can be broadly categorized under Testing and Scoring.
Testing
Run the Baseline Model on the Evaluation Split: Our first step is to run the teacher model (Llama 3.1 405B) on the evaluation split to generate its predictions.
Run the Student Model on the Evaluation Split: Next, we run the student model on the same evaluation dataset to generate its predictions.
Scoring
Calculate Metrics for the Baseline Model: Using the predictions from the baseline model, we calculate various performance metrics.
Calculate Metrics for the Student Model: Similarly, we calculate the performance metrics for the student model’s predictions.
Compare Metrics: Finally, we compare the performance of both models, highlighting the results through visuals and diagrams.
Testing the baseline and student models
Installing the SDK
First, you need to install the Azure AI Evaluation SDK:
pip install openai azure-ai-evaluation azure-identity promptflow-azure
Note on SDK Availability: It’s important to highlight that the Azure AI Evaluation SDK is currently in beta. This means that while the SDK offers a comprehensive suite of tools and features for evaluating AI models, it may still undergo changes and improvements. Users should stay updated with any modifications or enhancements introduced by Azure, and consider providing feedback to help refine and optimize the SDK for wider use in its official release.
Baseline Model Testing
This will generate answers to the questions in the evaluation dataset using the baseline model:
env $(cat .env .env.state) python .gorilla/raft/eval.py
–question-file $dataset_path_hf_eval
–answer-file $dataset_path_hf_eval_answer_baseline
–model $BASELINE_OPENAI_DEPLOYMENT
–env-prefix BASELINE
–mode $BASELINE_MODEL_API
Note: JSONL file format needs to be further converted to a format suitable for testing, see eval notebook for details.
Student Model Testing
This will generate answers to the questions in the evaluation dataset using the student model:
env $(cat .env .env.state) python .gorilla/raft/eval.py
–question-file $dataset_path_hf_eval
–answer-file $dataset_path_hf_eval_answer
–model $STUDENT_DEPLOYMENT_NAME
–env-prefix STUDENT
–mode $STUDENT_MODEL_API
Note: JSONL file format needs to be further converted to a format suitable for testing, see eval notebook for details.
Let’s look at a sample
This sample is extracted from the evaluation split and shows the baseline and student answers:
question: What types of waves do strong direct offshore winds create?
gold_final_answer: plunging or large barrel waves
context <DOCUMENT>Lefts, Rights, and A-frames could be directed from this pump design providing forWave intensity Artificial reefs Artificial wavesSurfing a stationary, artificialwave in Southern California A surfer going for the tube Catching waves at a surfing conteston the North Shore of Oahu, Hawaiirippable surf and barrel rides. The Ocean Dome cost about $2 billion tobuild and was expensive to maintain.[31] The Ocean Dome was closed in2007.</DOCUMENT> <DOCUMENT>However, thewaves that are produced by reef breaks are some of the best in the world. Famous reef breaks arepresent in Padang Padang (Indonesia), Pipeline (Hawaii), Uluwatu (Bali), and Teahupo’o(Tahiti).[49][52]A ledge break is formed by steep rocks ledges that make intense waves because the waves travelthrough deeper water then abruptly reach shallower water at the ledge. Shark Island, Australia is alocation with a ledge break.</DOCUMENT>
baseline_answer: Strong direct offshore winds create plunging or large barrel waves. These waves are characterized by their increased height and intensity due to the shallow water depth when they break.
student_answer: plunging or large barrel waves
This sample was chosen randomly and in this case, the student model answer is identical to the gold answer. This is not always the case.
Evaluating the baseline and student model responses
Built-in Evaluators
The Azure AI Evaluation SDK offers an extensive suite of built-in metrics, designed to facilitate comprehensive evaluation of AI models. In the following sections, we’ll highlight selected evaluators and provide detailed examples of their application, showcasing how they can enhance your model assessments.
They are categorized into two main groups: (1) metrics that leverage GPT models for scoring, providing advanced qualitative assessments, and (2) metrics that utilize straightforward mathematical calculations for evaluation.
GPT based metrics
Category
Evaluator Class
Notes
Quality
GroundednessEvaluator
Groundedness measures the extent to which the generated content is based on factual correctness and aligns with the provided data or context.
RelevanceEvaluator
Relevance assesses how pertinent the generated text is to the given input or prompt. Higher relevance scores indicate that the generated responses are more appropriate and closely aligned with the query or topic.
CoherenceEvaluator
Coherence measures how logically consistent and semantically meaningful the generated text is. Higher coherence indicates better understanding and logical consistency.
FluencyEvaluator
Fluency evaluates how naturally the generated text reads. Fluent text should be grammatically correct and smooth in its flow.
SimilarityEvaluator
Measures the similarity between the predicted answer and the correct answer
Content Safety
ViolenceEvaluator
SexualEvaluator
SelfHarmEvaluator
HateUnfairnessEvaluator
Composite
QAEvaluator
Built on top of individual quality evaluators.
ChatEvaluator
Similar to QAEvaluator but designed for evaluating chat messages.
ContentSafetyEvaluator
Built on top of individual content safety evaluators.
Math based metrics
Evaluator Class
Notes
BleuScoreEvaluator
BLEU (Bilingual Evaluation Understudy) is a widely-used metric for evaluating the quality of text generated by an AI by comparing it to one or more reference texts. It particularly looks at the precision of n-grams in the generated text.
RougeScoreEvaluator
ROUGE (Recall-Oriented Understudy for Gisting Evaluation) primarily measures recall, comparing n-grams between the generated text and reference texts. It is commonly used for evaluation in summarization tasks.
F1ScoreEvaluator
A balance between precision and recall, the F1 score provides a single metric that combines both, offering a more comprehensive view of performance in classification problems.
Running metrics individually
The Azure AI Evaluation SDK enables the utilization of individual metrics. This feature is particularly useful for experimentation, gaining deeper insights, and incorporating metrics into bespoke evaluation workflows.
Tech Tip: This blog post is crafted using the Quarto writing system, a versatile tool for publishing with code. The Azure AI Evaluation metrics are seamlessly executed and displayed inline within this post.
Let’s look first at the F1 Score math metric
For a response that is accurate but includes additional information not found in the ground truth:
from azure.ai.evaluation import F1ScoreEvaluator
f1_score_evaluator = F1ScoreEvaluator()
f1_score = f1_score_evaluator(
ground_truth=”The capital of Japan is Tokyo.”,
response=”Tokyo is Japan’s capital, known for its blend of traditional culture”
)
print(f”The F1 Score is {round(f1_score[‘f1_score’], 2)}”)
The F1 Score is 0.5
For a response that is accurate but uses the same words turned differently:
from azure.ai.evaluation import F1ScoreEvaluator
f1_score_evaluator = F1ScoreEvaluator()
f1_score = f1_score_evaluator(
ground_truth=”The capital of Japan is Tokyo.”,
response=”Tokyo is Japan’s capital”
)
print(f”The F1 Score is {round(f1_score[‘f1_score’], 2)}”)
The F1 Score is 0.67
Let’s look first at the Similarity GPT metric
We first need to instantiate the Judge model client:
from os import getenv
from azure.ai.evaluation import AzureOpenAIModelConfiguration
model_config = AzureOpenAIModelConfiguration(
azure_endpoint = getenv(“JUDGE_AZURE_OPENAI_ENDPOINT”),
azure_deployment = getenv(“JUDGE_AZURE_OPENAI_DEPLOYMENT”),
api_version = getenv(“JUDGE_OPENAI_API_VERSION”),
)
Let’s now instantiate the Similarity score metric:
from azure.ai.evaluation import SimilarityEvaluator
similarity_evaluator = SimilarityEvaluator(model_config)
For a response that is accurate but includes additional information not found in the ground truth:
similarity = similarity_evaluator(
query=”What’s the capital of Japan?”,
ground_truth=”The capital of Japan is Tokyo.”,
response=”Tokyo is Japan’s capital, known for its blend of traditional culture”
)
print(f”The Similarity is {similarity[‘gpt_similarity’]}”)
The Similarity is 4.0
For a response that is accurate but uses the same words turned differently:
similarity = similarity_evaluator(
query=”What’s the capital of Japan?”,
ground_truth=”The capital of Japan is Tokyo.”,
response=”Tokyo is Japan’s capital”
)
print(f”The Similarity is {similarity[‘gpt_similarity’]}”)
The Similarity is 5.0
GPT-based similarity metrics demonstrate greater robustness in evaluating correct responses that are phrased differently compared to traditional F1 Scores.
Running metrics in bulk
While evaluating metrics individually helps in understanding their functionality, acquiring statistically significant results necessitates running them on a larger scale across an evaluation dataset.
The Azure AI Evaluation SDK provides a convenient bulk evaluation capability via the evaluate function.
To begin, we need to initialize the evaluators that will be used to assess the student and baseline models:
from azure.ai.evaluation import CoherenceEvaluator, F1ScoreEvaluator, FluencyEvaluator, GroundednessEvaluator, RelevanceEvaluator, SimilarityEvaluator, BleuScoreEvaluator, RougeScoreEvaluator, RougeType
# Initializing evaluators
evaluators = {
# GPT based metrics
“coherence” : CoherenceEvaluator(model_config),
“f1_score” : F1ScoreEvaluator(),
“fluency” : FluencyEvaluator(model_config),
“groundedness” : GroundednessEvaluator(model_config),
“relevance” : RelevanceEvaluator(model_config),
“similarity” : SimilarityEvaluator(model_config),
# Math metrics
“bleu” : BleuScoreEvaluator(),
“rouge_1” : RougeScoreEvaluator(RougeType.ROUGE_1),
“rouge_2” : RougeScoreEvaluator(RougeType.ROUGE_2),
}
Note that we have previously executed the baseline and student models on the evaluation dataset, which means the JSONL file provided to the evaluate function already includes their responses. Consequently, further model invocations are unnecessary at this stage.
Recommendation: It’s often beneficial to run the baseline and student models once initially. By doing so, you can execute the evaluate function multiple times with various metrics configurations without re-incurring the inference time and costs associated with model executions. Note that while this avoids repeated inference expenses, using GPT-based metrics will still incur costs and time for each evaluate execution, as the Judge model is utilized.
from azure.ai.evaluation import evaluate
result = evaluate(
data=”test-results-[baseline|student].jsonl”,
evaluators=evaluators,
evaluator_config={
“default”: {
“column_mapping”: {
“query”: “${data.question}”,
“response”: “${data.final_answer}”,
“ground_truth”: “${data.gold_final_answer}”,
“context”: “${data.context}”,
}
}
},
)
This command initiates a background process that hosts a user interface locally. Here is an example of its appearance:
The interface updates in real-time to display the progress of the scoring process on the evaluation dataset.
Additionally, you can click on each completed line to view the detailed trace of the calls. This feature is particularly useful for GPT-based metrics, as it reveals the system prompt used and provides insights into the underlying logic that contributed to the final score:
Comparing Metrics and Visualizing Results
Note: You can find the implementation details for generating the comparison figures of baseline and student metrics in the repository notebook. This resource provides comprehensive insights into how the metric comparisons were conducted, along with the code necessary to reproduce these visualizations.
Going further with continuous model evaluation and GenAIOps
This marks the beginning of a continuous improvement journey. It’s quite common to find that the student model’s initial performance does not meet expectations. Through our evaluation, we may uncover areas needing adjustment—whether it’s refining the synthetically generated dataset, optimizing fine-tuning parameters, or other elements. This initiates a cycle of iterative improvement and reassessment before the model is ready for deployment in production.
To effectively help you navigate this process, we came up with the GenAIOps Maturity Model, which serves as a comprehensive guide for evaluating your progress and maturity in operationalizing AI models.
Conclusion
By leveraging the Azure AI Evaluation Python SDK, we gain a detailed understanding of how our distilled student model compares to the baseline model across a spectrum of performance indicators. This structured evaluation framework not only helps in refining our models but also ensures that we are continuously improving and delivering robust AI solutions.
Explore, fork and clone the comprehensive 🚀🔥 GitHub Recipe Repository for complete code coverage on executing the full distillation process, including in-depth evaluations as detailed in this blog post. Discover step-by-step notebooks and resources to master the entire pipeline efficiently.
Stay tuned for more insights and tutorials on advanced AI topics and the latest tools available in the Azure ecosystem!
Microsoft Tech Community – Latest Blogs –Read More
Announcing the general availability of sidecar extensibility in Azure App Service
The cloud has changed the business landscape—continuous innovation is no longer optional, and enterprise apps must modernize or perish. Generative AI is being adopted quickly not only to build net-new apps, but also to transform existing apps to deliver higher value AI-assisted outcomes. App modernization means updating and improving apps so they’re able to work well with evolving software and cloud. There are several ways to achieve this, whether by a simple ‘lift and shift’ of applications to the cloud or rewriting them as cloud-based services that work well with other modern solutions.
Today, we are excited to announce general availability of Sidecars in Azure App Service, a versatile pathway for organizations to modernize existing apps and add various kinds of powerful new capabilities without having to significantly rewrite the main application code.
What are sidecars, and where could they fit into your modernization story?
What is sidecar pattern? Think of it like a motorcycle with an attached sidecar. You pair your primary enterprise app (the motorcycle) with a modern, supplementary set of tasks placed in their own container (the sidecar). These tasks enable you to add new capabilities by adding SDKs—like AI, logging, monitoring, and security features—to your primary application without the need to significantly modify and redeploy the app.
The sidecar pattern enables you to bring modern capabilities such as AI to your legacy apps much faster without rebuilding the primary app from the ground up. There are lifecycle and integration advantages as well: the primary app and sidecar capabilities can be treated and managed as a single unit. They’re easier to maintain since they remain distinct apps, and you can modify components of the sidecar independently without changing the entire app (and vice versa).
Using sidecars to add modern features to your existing apps
Let’s walk through a few app modernization scenarios where the sidecar approach may help.
Infuse generative AI into existing apps. Microsoft offers multiple AI models to help you get started in your AI journey, whether you choose a large language model (LLM) such as Azure OpenAI, or a small language model (SLM) like Phi-3. SLMs are lightweight, efficient, and accessible, making them ideal for limited computational resources and real-time inference. They can be fine-tuned for specific domains, enhancing performance and accuracy. SLMs are more secure, environmentally friendly, and suitable for edge computing due to their smaller codebases, lower energy use, and faster inference times. In this example, you can integrate Phi-3 ONNX using a sidecar pattern to deploy and run intelligent apps on Azure App Service without utilizing GPUs. This approach can offer a faster way to get started with AI with relatively lower cost. It also serves as an onramp to scale up to more generalized and powerful large language models (LLMs) such as deploying OpenAI-powered apps on Azure App Service in the future.
Gain modern observability, reporting, and analytics. Observability has become crucial for modern AI-infused applications. A sidecar gives you the ability to integrate functionality for metrics, logging, and tracing without having to completely rebuild the primary app. This includes first party solutions such as Azure Monitor Application Insights, a feature of Azure Monitor that excels in application performance monitoring (APM) for live web applications, as well as Azure Native ISV service partners such as Dynatrace, Datadog, and others. Learn more.
Improve web app performance during high traffic. Redis Cache is an in-memory data store, which means it can retrieve data much faster than traditional databases. This can lead to application performance improvements, especially for read-heavy workloads. Azure Cache for Redis integrates seamlessly with other Azure services, such as Azure App Service, Azure Kubernetes Service (AKS), and Azure Functions. By deploying Redis cache as a sidecar, you can more easily unlock the benefits of caching and enable improved performance for your web applications.
These are just a few examples of how you can use sidecar to modernize your applications. The possibilities are only limited by your imagination!
Get started today with Azure App Service
Visit the product documentation to learn more and get started. Watch the learn live session on-demand for a demo of the new features from our experts. Join us at Microsoft Ignite 2024 and discover solutions that help modernize and manage intelligent apps, including demos of sidecar pattern and an opportunity to connect with the engineers building this technology in the Expert meet-up areas.
If you’re new to Azure, take advantage of our free credit program by signing up for a free account at Azure App Service.
Here are some other helpful resources to learn more:
Catch A Glimpse into the Future: The Sidecar Pattern on Linux App Service
Visit Microsoft Learn to watch a tutorial on configuring a sidecar container
Read about building smarter apps by integrating Phi-3 SLM with Linux App Service
Learn how to implement local RAG using Phi-3 ONNX Runtime and Sidecar Pattern on Linux App Service
Revisit many of the links featured in this blog:
Optimizing SLM with ONNX Runtime: Phi-3 on CPU with Sidecars for App Service – Azure App Service
A Step-by-Step Guide to Datadog Integration with Linux App Service via Sidecars – Azure App Service
Powering Observability: Dynatrace Integration with Linux App Service via Sidecars – Azure App Service
Leveraging Redis as a Sidecar for Linux App Service – Azure App Service
Deploying OpenAI-powered apps on Azure App Service
Microsoft Tech Community – Latest Blogs –Read More
Issues with populated by Power Automate Date and Time column in excel
Good day.
I created a Power Automate flow that takes a date and time and populates them into an Excel spreadsheet. However, once imported, the date and time are recognized by Excel as text, not as date and time values. This persists until I select the cell, place my cursor in the formula bar, and press Enter. Only then are they recognized as date and time. I am using formatDateTime in Power Automate to export the date and time.
Any help would be appreciated.
Good day.I created a Power Automate flow that takes a date and time and populates them into an Excel spreadsheet. However, once imported, the date and time are recognized by Excel as text, not as date and time values. This persists until I select the cell, place my cursor in the formula bar, and press Enter. Only then are they recognized as date and time. I am using formatDateTime in Power Automate to export the date and time.Any help would be appreciated. Read More
Inbound Sensitive Information
Hello All,
We currently have some DLP policies to restrict Financial Data, HIPPA, and PII data from leaving our org.
However, is there a way to restrict this type of sensitive data from being sent into the org? For example, an external address sends some sensitive data to a specific mailbox. Can a DLP policy be created to block that data from reaching a specific mailbox and reply back the email was blocked due to the content?
Thanks for any info!
Hello All,We currently have some DLP policies to restrict Financial Data, HIPPA, and PII data from leaving our org. However, is there a way to restrict this type of sensitive data from being sent into the org? For example, an external address sends some sensitive data to a specific mailbox. Can a DLP policy be created to block that data from reaching a specific mailbox and reply back the email was blocked due to the content? Thanks for any info! Read More