Tag Archives: microsoft
Still possible to remove password from PDF even forgot the password?
I have a password protected PDF file and can’t open it now because I forgot the password. This PDF contains important information that I need to access urgently. I’ve tried several passwords that I thought might work, but no luck so far.
I am wondering if it is possible to remove password from PDF file? If so, could you please share the methods or tools you used? I’m looking for a reliable and safe way to recover the password without compromising the contents of the file.
Regards,
Andrew
I have a password protected PDF file and can’t open it now because I forgot the password. This PDF contains important information that I need to access urgently. I’ve tried several passwords that I thought might work, but no luck so far. I am wondering if it is possible to remove password from PDF file? If so, could you please share the methods or tools you used? I’m looking for a reliable and safe way to recover the password without compromising the contents of the file. Regards,Andrew Read More
Renew App registration secret with Powershell
Hello everyone,
is it possible to create a new secret for an existing app registration via powershell?
Thanks for tips and suggestions in advance!
Hello everyone, is it possible to create a new secret for an existing app registration via powershell? Thanks for tips and suggestions in advance! Read More
Elevating BFF Apps with .NET Aspire
Back in April I wrote a post on how to implement the Backend for Frontend (BFF) pattern with Azure Container Apps. At the time I said I would avoid bringing in .NET Aspire as it was in preview and earlier explorations pointed to it being somewhat buggy in those preliminary stages.
About a month ago at Build .NET Aspire went GA so that raises the question anew – should I use Aspire? Let’s revisit our sample app and take a closer look.
Revisit that post to read about the BFF concept and implementation details: https://techcommunity.microsoft.com/t5/microsoft-developer-community/implementing-a-blazor-based-bff-in-azure-container-apps/ba-p/4111208
All the code can be found here:
https://github.com/ahelland/Container-Apps-BFF/tree/main/src/BFF_Aspire
Adding Aspire to an existing project is quite easy – right-click and choose to add “.NET Aspire Orchestrator Support”.
Hitting F5 brings up the app and dashboard. But wait, the WeatherAPI doesn’t work.
We originally made the choice to have the API as a separate solution and that boundary still applies. Aspire isn’t able to step out of the boundary of the web app solution by itself. You can sort of wire in other csproj files manually, but I guess it’s a design decision if you want to do this. Personally I feel this can lead to scenarios where a solution isn’t as independent and complete as it should be. Do note that in this specific case there’s an additional minor snag because Aspire currently does not support APIs/apps where you checked “Place solution and project in the same directory” during scaffolding. In this sample it is not critical to maintain the API totally separate from the rest of the solution so I went with the easy fix to bring it all into one solution.
But what if you want/need these two to be separate entities? Aspire can bring in containers instead of the csproj files so if that is a logical boundary you can work with I would say that is a feasible path.
Ok, we’re still able to run our app – that’s nice and dandy.
Any other value adds? Well, yes actually there is.
Service Discovery
In the first iteration of our code we used a somewhat janky approach switching between the local API address of the weather API and the one running in the cluster:
var apiBaseAddress = “http://localhost:5041”;
//For ACA
if (Environment.GetEnvironmentVariable(“ASPNETCORE_ENVIRONMENT”) == “Production”)
{
apiBaseAddress = “http://weatherapi”;
}
builder.Services.AddHttpClient<IWeatherForecaster, ServerWeatherForecaster>(httpClient =>
{
httpClient.BaseAddress = new(apiBaseAddress);
});
On a larger scale It’s not practical to do if checks like this, and of course if the port number changes for some reason it will break. It works when running in Container Apps because a service discovery mechanism is running in the cluster. Our previous approach was basically to not use service discovery in debug mode and now .NET Aspire is able to fill this role for us.
How about us changing the code to:
builder.Services.AddHttpClient<IWeatherForecaster, ServerWeatherForecaster>(httpClient =>
{
httpClient.BaseAddress = new(“https://weatherapi”);
});
There’s a few extra lines to make this work – in the Program.cs for the BFF_Web_App.AppHost project you have these two lines:
builder.AddProject<Projects.WeatherAPI>(“weatherapi”);
builder.AddProject<Projects.BFF_Web_App>(“bff-web-app”);
Tweak this to the following:
var weatherapi = builder.AddProject<Projects.WeatherAPI>(“weatherapi”);
builder.AddProject<Projects.BFF_Web_App>(“bff-web-app”)
.WithReference(weatherapi);
Adding references means Aspire takes care of injecting the correct name resolution for us. Note that we only added a reference to the Web App project as the API project does not rely on making outbound calls. For a larger project you shold figure out which project needs what based on mapping out the communication paths.
Application settings
Another “trick” we pulled that isn’t really recommended is how we handled appsettings – basically hardcoding inline:
builder.Services.AddAuthentication()
.AddJwtBearer(“Bearer”, jwtOptions =>
{
// The API does not require an app registration of its own, but it does require a registration for the calling app.
// These attributes can be found in the Entra ID portal when registering the client.
jwtOptions.Authority = “https://sts.windows.net/{TENANT ID}/”;
jwtOptions.Audience = “api://{CLIENT ID}”;
});
We could have put this into the appsettings.json file instead of course, but laziness 🙂 Thing is, the Aspire AppHost project brings its own appsettings. So we can do the following (in the AppHost’s appsettings.json):
{
“Logging”: {
“LogLevel”: {
“Default”: “Information”,
“Microsoft.AspNetCore”: “Warning”,
“Aspire.Hosting.Dcp”: “Warning”
}
},
“Parameters”: {
“TenantId”: “guid”,
“ClientId”: “guid”,
“ClientSecret”: “secret”
}
}
In the Program.cs for the AppHost you need to bring them in as variables:
var tenantId = builder.AddParameter(“TenantId”);
var clientId = builder.AddParameter(“ClientId”);
var clientSecret = builder.AddParameter(“ClientSecret”, secret:true);
And afterwards we can invoke them like this (exemplified by the Weather API’s Program.cs, but same goes in the Web App):
var tenantId = builder.Configuration.GetValue<string>(“TenantId”);
var clientId = builder.Configuration.GetValue<string>(“ClientId”);
builder.Services.AddAuthentication()
.AddJwtBearer(“Bearer”, jwtOptions =>
{
jwtOptions.Authority = $”https://sts.windows.net/{tenantId}/”;
jwtOptions.Audience = $”api://{clientId}”;
});
Is there a point to doing this instead of the classic appsettings per csproj? In this solution we use the same clientId in two projects – clearly it’s easier maintaining this in one place. For settings that only apply to one project it might not make as much sense apart from the value of centralizing settings. While I didn’t do it here you can also wire up the AppHost to look up secrets in a Key Vault solving another dev challenge.
There’s only so much you can do in a small sample like this, but Aspire allows you to bring up containers, databases and more using the same pattern. (Meaning you can easily bring in frontends written in other languages for instance.) Visit the (updated) eShop sample for more on that.
https://github.com/dotnet/eShop
Should you run out and implement this immediately? Well, if you have a working setup you feel happy with this isn’t going to magically improve anything, but if you are in the early phases of setting up a new project I would certainly attempt to bring in Aspire if manual wiring is the alternative. Otherwise plan to add in a maintenance sprint or bring in it when upgrading to .NET 9 in a few months or something.
What about deployments to the cloud? I kinda saw that one coming 🙂
Aspire plus Azure equals ?
In early demos we saw how Aspire-based solutions were being deployed to Azure at the click of a few buttons. This lead many to think that it’s this great developer focused deployment vehicle. Actually…no.
Aspire in itself is focused on enhancing the developer experience locally on your developer computer. You don’t have to deploy to Azure, on-prem or in any way alter your current CI/CD processes. You don’t deploy Aspire itself either. It does however enable other tools like the Azure Developer CLI to process the Aspire manifest and turn it into Bicep code for creating Azure resources.
There’s also a community effort called Aspir8 that takes care of generating Kubernetes deployment packaging. It is apparently in the process of being brought into the MS-hosted Aspire repo so we’ll see how that integrates, but still think of Aspire as something that runs on your machine only.
OK – so technically Aspire isn’t responsible for pushing my code to Azure, but I can still have developers releasing their code this way? Well…
If you have the Azure Developer CLI (azd) generate the IaC for you the code should still be deployed to Azure DevOps or GitHub and have pipelines/actions run there as a general practice. (Azure sandbox subscriptions are an exception, but I’m not diving into those concepts for now.) Azd will even generate these for you.
The Azure Developer CLI is able to understand things like you needing a container environment, container registry, etc. But for obvious reasons it is not able to infer that you want to force all outbound traffic through an Azure Firewall and stand up an Azure Application Gateway to handle the inbound traffic. And even with the knowledge it takes quite a bit more Bicep code to stand up all of that in addition to the added cost.
Azure Developer CLI
Creating the necessary bits for playing with code isn’t complicated.
(Installing azd: https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/install-azd?tabs=winget-windows%2Cbrew-mac%2Cscript-linux&pivots=os-windows)
Run azd init:
Run azd infra synth:
Pausing for a moment here. This is the folder and file structure from my previous blog post:
This is what azd generated:
Which sort of leads to the question of whether this is me over-engineering things or the azd team under-engineering? As so often the answer is possible a place in-between. If what you want to do is to run a quick test and debug session to verify your code works what azd generates is perfectly fine. If what you want is a solution that is more production-like I believe my Bicep is better suited. (I have simplified my implementation to be suited for a blog post so I must stress that it is not production grade on my part either, but it has a structure you can work from.)
Currently you are driven towards deploying the services as container apps. As I also based myself on that it’s comparable in this context, but maybe you want something else. The Kubernetes tooling I mentioned can bring the containers to a different runtime environment, but if what you are looking for is non-containerized Azure Functions, Azure App Services, or something else you’re not really supported by azd for now. To be fair when Project Tye was previewed a few years ago it was about solving the pain of microservices development, not monoliths. (Project Tye being an experiment that was discontinued, but is/was the inspiration for .NET Aspire.)
And that’s sort of the general state of azd at the moment – dev friendly and quick to get going. Thumbs up. Governance, scalable structure, etc. Not so much.
A quick azd up will deploy and confirm that it works. Both the web app and the api are deployed with internal ingress only though, so you will need to flip to external ingress on the web app to be able to browse to it:
There is one thing I’d like to point out with the azd approach that I like. In my approach with levels the container apps are called “Level-4” and there’s a copy & paste job adding to the list for each app in your solution. This can be done as a templated loop in a CI/CD pipeline, but it’s still a slight disconnect between the lower level infra and the developer stuff. With the azd & Aspire combo you will notice that in addition to the root infra folder there is also an infra folder underneath the Aspire AppHost project:
This contains yaml files that are specific to the individual apps and are easy to tweak. For instance the ingress setting can easily be set to external here instead of using the Azure Portal. You can even reference Bicep files in the Aspire orchestration code:
https://learn.microsoft.com/en-us/dotnet/aspire/deployment/azure/custom-bicep-templates?tabs=dotnet-cli
I think that might be a topic to dive deeper into at a later point, but as you can see there are new options for integrating app code and infra code tighter.
Look at the rest of the code here:
https://github.com/ahelland/Container-Apps-BFF/tree/main/src/BFF_Aspire
Aspire Dashboard
I stated a few paragraphs ago that Aspire itself only pertains to your local environment and is not deployed to Azure even if it plays nice with azd. That wasn’t a lie, but while the orchestration only applies locally there is actually a component from the observability puzzle you can deploy. The Aspire Dashboard:
Azure Monitor and Application Insights have their use and this doesn’t remove the need for those tools, but I think this is a nice addition for giving the rather direct access to traces and console logs. (Console logs are available directly in the Azure Portal for Container Environments as well, but it’s slightly more clunky.) Sure, there are other tools in the observability stack, but this one didn’t cost me any additional calories to get going with and it brings parity between the things on my local computer and what I can do after deployment.
All in all I believe .NET Aspire is in a much better state than at the start of the year and you should do a proof of concept or two to see if it fits into your developer workflow.
Microsoft Tech Community – Latest Blogs –Read More
Fine-Tune and Integrate Custom Phi-3 Models with Prompt Flow: Step-by-Step Guide
Fine-Tune and Integrate Custom Phi-3 Models with Prompt Flow
Introduction
Phi-3 is a family of small language models (SLMs) developed by Microsoft that delivers exceptional performance and cost-effectiveness. In this tutorial, you will learn how to fine-tune the Phi-3 model and integrate it with Prompt Flow. By leveraging Azure Machine Learning, and Prompt flow you will establish a workflow for deploying and utilizing custom AI models. This tutorial is divided into three series:
Series 1: Set up Azure resources and Prepare the Environment
In Series 1, You will learn how to configure the necessary Azure resources and set up your environment for model fine-tuning.
Series 2: Fine-Tune and Deploy the Phi-3 model
In Series 2, you will fine-tune the Phi-3 model. After fine-tuning, you will deploy the model, making it accessible for integration with Prompt Flow.
Series 3: Integrate the custom phi-3 model with Prompt Flow
In Series 3, You will Integrate the fine-tuned Phi-3 model with Prompt flow.
Here is an overview of this tutorial.
Note
For more detailed information and to explore additional resources about Phi-3, please visit the Phi-3CookBook.
Prerequisites
Python
Azure subscription
Visual Studio Code
Azure CLI
Table of Contents
Series 1: Set Up Azure resources and Prepare the environment
Request GPU quotas in Azure subscription
Create Azure Machine Learning workspace
Set up the project and install the libraries
Set up project files in Visual Studio Code
Prepare dataset for fine-tuning
Series 2: Fine-tune and Deploy the Phi-3 model
Fine-tune the Phi-3 Model
Deploy the fine-tuned Phi-3 Model
Series 3: Integrate the custom Phi-3 model with Prompt Flow
Integrate the custom Phi-3 model with Prompt Flow
Congratulation!
Request GPU Quotas in Azure Subscription
In this tutorial, you will learn how to fine-tune and deploy a Phi-3 model, using GPUs. For fine-tuning, you will use the Standard_NC6s_v3 GPU, which requires a quota request. For deployment, you will use the Standard_E4s_v3 CPU, which does not require a quota request.
Note
Only Pay-As-You-Go subscriptions (the standard subscription type) are eligible for GPU allocation; benefit subscriptions are not currently supported.
For those using benefit subscriptions (such as Visual Studio Enterprise Subscription) or those looking to quickly test the fine-tuning and deployment process, this tutorial also provides guidance for fine-tuning with a minimal dataset using a CPU. However, it is important to note that fine-tuning results are significantly better when using a GPU with larger datasets.
In this exercise, you will:
Request GPU Quotas in your Azure Subscription
Request GPU Quotas in Azure Subscription
Type subscriptions in the search bar at the top of the portal page and select Subscriptions from the options that appear.
Select the subscription you’d like to use.
Perform the following tasks:
Select Usage + quotas from the left side tab.
Select the Region you’d like to use.
Select the Quota name to use. For example, select Standard NCSv3 Family vCPUs, which includes the Standard_NC6s_v3 GPU.
Select the New Quota Request.
Select the Enter a new limit.
Inside the New Quota Request page, enter the New limit you’d like to use.
Inside the New Quota Request page, select Submit to request the GPU quota.
Create Azure Machine Learning Workspace
In this exercise, you will:
Create an Azure Machine Learning Workspace.
Create an Azure Machine Learning Workspace
Type azure machine learning in the search bar at the top of the portal page and select Azure Machine Learning from the options that appear.
Select + Create from the navigation menu.
Select New workspace from the navigation menu.
Perform the following tasks:
Select your Azure Subscription.
Select the Resource group to use (create a new one if needed).
Enter Workspace Name. It must be a unique value.
Select the Region you’d like to use.
Select the Storage account to use (create a new one if needed).
Select the Key vault to use (create a new one if needed).
Select the Application insights to use (create a new one if needed).
Select the Container registry to None.
Tip
When you create or use a Storage account in Azure Machine Learning, a container named “azureml” is automatically created within the Storage account. This container is used for storing model artifacts, training outputs, and other data generated during the machine learning process. In this tutorial, you will use the “azureml” container to manage and store all the necessary files and outputs related to our machine learning workflows.
Select Review + Create.
Select Create.
Add role assignment
To fine-tune and deploy your models, you must first ceate a User Assigned Managed Identity (UAI) and assign it the appropriate permissions. This UAI will be used for authentication during deployment, so it is critical to grant it access to the storage accounts, container registry, and resource group.
In this exercise, you will:
Create User Assigned Managed Identity(UAI).
Add Contributor role assignment to Managed Identity.
Add Storage Blob Data Reader role assignment to Managed Identity.
Add AcrPull role assignment to Managed Identity.
Create User Assigned Managed Identity(UAI)
Type managed identities in the search bar at the top of the portal page and select Managed Identities from the options that appear.
Select + Create.
Perform the following tasks to navigate to Add role assignment page:
Select your Azure Subscription.
Select the Resource group to use (create a new one if needed).
Select the Region you’d like to use.
Enter the Name. It must be a unique value.
Select Review + create.
Select + Create.
Add Contributor role assignment to Managed Identity
Navigate to the Managed Identity resource that you created.
Select Azure role assignments from the left side tab.
Select +Add role assignment from the navigation menu.
Inside Add role assignment page, Perform the following tasks:
Select the Scope to Resource group.
Select your Azure Subscription.
Select the Resource group to use.
Select the Role to Contributor.
Select Save.
Add Storage Blob Data Reader role assignment to Managed Identity
Type azure storage accounts in the search bar at the top of the portal page and select Storage accounts from the options that appear.
Select the storage account that associated with the Azure Machine Learning workspace. For example, finetunephistorage.
Perform the following tasks to navigate to Add role assignment page:
Navigate to the Azure Storage account that you created.
Select Access Control (IAM) from the left side tab.
Select + Add from the navigation menu.
Select Add role assignment from the navigation menu.
Inside Add role assignment page, Perform the following tasks:
Inside the Role page, type Storage Blob Data Reader in the search bar and select Storage Blob Data Reader from the options that appear.
Inside the Role page, select Next.
Inside the Members page, select Assign access to Managed identity.
Inside the Members page, select + Select members.
Inside Select managed identities page, select your Azure Subscription.
Inside Select managed identities page, select the Managed identity to Manage Identity.
Inside Select managed identities page, select the Manage Identity that you created. For example, finetunephi-managedidentity.
Inside Select managed identities page, select Select.
Select Review + assign.
Add AcrPull role assignment to Managed Identity
Type container registries in the search bar at the top of the portal page and select Container registries from the options that appear.
Select the container registry that associated with the Azure Machine Learning workspace. For example, finetunephicontainerregistries
Perform the following tasks to navigate to Add role assignment page:
Select Access Control (IAM) from the left side tab.
Select + Add from the navigation menu.
Select Add role assignment from the navigation menu.
Inside Add role assignment page, Perform the following tasks:
Inside the Role page, Type AcrPull in the search bar and select AcrPull from the options that appear.
Inside the Role page, select Next.
Inside the Members page, select Assign access to Managed identity.
Inside the Members page, select + Select members.
Inside Select managed identities page, select your Azure Subscription.
Inside Select managed identities page, select the Managed identity to Manage Identity.
Inside Select managed identities page, select the Manage Identity that you created. For example, finetunephi-managedidentity.
Inside Select managed identities page, select Select.
Select Review + assign.
Set up the project and install the libraries
Now, you will create a folder to work in and set up a virtual environment to develop a program that interacts with users and uses stored chat history from Azure Cosmos DB to inform its responses.
In this exercise, you will
Create a folder to work inside it.
Create a virtual environment.
Install the required packages.
Create a folder to work inside it
Open a terminal window and type the following command to create a folder named finetune-phi in the default path.
mkdir finetune-phi
Type the following command inside your terminal to navigate to the finetune-phi folder you created.
cd finetune-phi
Create a virtual environment
Type the following command inside your terminal to create a virtual environment named .venv.
python -m venv .venv
Type the following command inside your terminal to activate the virtual environment.
.venvScriptsactivate.bat
Note
If it worked, you should see (.venv) before the command prompt.
Install the required packages
Type the following commands inside your terminal to install the required packages.
pip install datasets==2.19.1
pip install transformers==4.41.1
pip install azure-ai-ml==1.16.0
pip install torch==2.3.1
pip install trl==0.9.4
pip install promptflow==1.12.0
Set up project files in Visual Studio Code
In this exercise, you will create the essential files for our project. These files include scripts for downloading the dataset, setting up the Azure Machine Learning environment, fine-tuning the Phi-3 model, and deploying the fine-tuned model. You will also create a conda.yml file to set up the fine-tuning environment.
In this exercise, you will:
Create a download_dataset.py file to download the dataset.
Create a setup_ml.py file to set up the Azure Machine Learning environment.
Create a fine_tune.py file in the finetuning_dir folder to fine-tune the Phi-3 model using the dataset.
Create a conda.yml file to setup fine-tuning environment.
Create a deploy_model.py file to deploy the fine-tuned model.
Create a integrate_with_promptflow.py file, to integrate the fine-tuned model and execute the model using Prompt Flow.
Create a flow.dag.yml file, to set up the workflow structure for Promptflow.
Create a config.py file to enter Azure information.
Note
Complete folder structure:
└── YourUserName
. └── finetune-phi
. ├── finetuning_dir
. │ └── fine_tune.py
. ├── conda.yml
. ├── config.py
. ├── deploy_model.py
. ├── download_dataset.py
. ├── flow.dag.yml
. ├── integrate_with_promptflow.py
. └── setup_ml.py
Create Project Files
Open Visual Studio Code.
Select File from the menu bar.
Select Open Folder.
Select the finetune-phi folder that you created, which is located at C:UsersyourUserNamefinetune-phi.
In the left pane of Visual Studio Code, right-click and select New File to create a new file named download_dataset.py.
In the left pane of Visual Studio Code, right-click and select New File to create a new file named setup_ml.py.
In the left pane of Visual Studio Code, right-click and select New File to create a new file named deploy_model.py.
In the left pane of Visual Studio Code, right-click and select New Folder to create a new forder named finetuning_dir.
In the finetuning_dir folder, create a new file named fine_tune.py.
Create and Configure conda.yml file
In the left pane of Visual Studio Code, right-click and select New File to create a new file named conda.yml.
Add the following code to the conda.yml file to set up the fine-tuning environment for the Phi-3 model.
name: phi-3-training-env
channels:
– defaults
– conda-forge
dependencies:
– python=3.10
– pip
– numpy<2.0
– pip:
– torch~=2.0
– torchvision~=0.18
– trl==0.8.6
– transformers~=4.41
– datasets~=2.19
– azureml-core~=1.30
– azure-storage-blob==12.19
– azure-ai-ml~=1.16
– azure-identity~=1.16
– accelerate~=0.30
– mlflow==2.14.1
– azureml-mlflow==1.56.0
Create and Configure config.py file
In the left pane of Visual Studio Code, right-click and select New File to create a new file named config.py.
Add the following code to the config.py file to include your Azure information.
# Azure settings
AZURE_SUBSCRIPTION_ID = “your_subscription_id”
AZURE_RESOURCE_GROUP_NAME = “your_resource_group_name” # “TestGroup”
# Azure Machine Learning settings
AZURE_ML_WORKSPACE_NAME = “your_workspace_name” # “finetunephi-workspace”
# Azure Managed Identity settings
AZURE_MANAGED_IDENTITY_CLIENT_ID = “your_azure_managed_identity_client_id”
AZURE_MANAGED_IDENTITY_NAME = “your_azure_managed_identity_name” # “finetunephi-mangedidentity”
AZURE_MANAGED_IDENTITY_RESOURCE_ID = f”/subscriptions/{AZURE_SUBSCRIPTION_ID}/resourceGroups/{AZURE_RESOURCE_GROUP_NAME}/providers/Microsoft.ManagedIdentity/userAssignedIdentities/{AZURE_MANAGED_IDENTITY_NAME}”
# Dataset file paths
TRAIN_DATA_PATH = “data/train_data.jsonl”
TEST_DATA_PATH = “data/test_data.jsonl”
# Fine-tuned model settings
AZURE_MODEL_NAME = “your_fine_tuned_model_name” # “finetune-phi-model”
AZURE_ENDPOINT_NAME = “your_fine_tuned_model_endpoint_name” # “finetune-phi-endpoint”
AZURE_DEPLOYMENT_NAME = “your_fine_tuned_model_deployment_name” # “finetune-phi-deployment”
AZURE_ML_API_KEY = “your_fine_tuned_model_api_key”
AZURE_ML_ENDPOINT = “your_fine_tuned_model_endpoint_uri” # “https://{your-endpoint-name}.{your-region}.inference.ml.azure.com/score”
Add Azure Environment Variables
Perform the following tasks to add the Azure Subscription ID:
Type subscriptions in the search bar at the top of the portal page and select Subscriptions from the options that appear.
Select the Azure Subscription you are currently using.
Copy and paste your Subscription ID into the config.py file.
Perform the following tasks to add the Azure Workspace Name:
Navigate to the Azure Machine Learning resource that you created.
Copy and paste your account name into the config.py file.
Perform the following tasks to add the Azure Resource Group Name:
Navigate to the Azure Machine Learning resource that you created.
Copy and paste your Azure Resource Group Name into the config.py file.
Perform the following tasks to add the Azure Managed Identity name
Navigate to the Managed Identities resource that you created.
Copy and paste your Azure Managed Identity name into the config.py file.
Prepare Dataset for Fine-Tuning
In this exercise, you will run the download_data.py file to download the wikitext datasets to your local environment. You will then use this datasets to fine-tune the Phi-3 model in Azure Machine Learning.
In this exercise, you will:
Add code to the download_dataset.py file to download the datasets.
Run the download_dataset.py file to download datasets to your local environment.
Download your dataset using download_data.py
Open the download_data.py file in Visual Studio Code.
Add the following code into download_data.py.
import json
import os
from datasets import load_dataset
from config import (
TRAIN_DATA_PATH,
TEST_DATA_PATH)
def load_and_split_dataset(dataset_name, config_name, split_ratio):
“””
Load and split a dataset.
“””
# Load the dataset with the specified name and configuration
dataset = load_dataset(dataset_name, config_name, split=split_ratio)
print(f”Original dataset size: {len(dataset)}”)
# Split the dataset into train and test sets (80% train, 20% test)
split_dataset = dataset.train_test_split(test_size=0.2)
print(f”Train dataset size: {len(split_dataset[‘train’])}”)
print(f”Test dataset size: {len(split_dataset[‘test’])}”)
return split_dataset
def save_dataset_to_jsonl(dataset, filepath):
“””
Save a dataset to a JSONL file.
“””
# Create the directory if it does not exist
os.makedirs(os.path.dirname(filepath), exist_ok=True)
# Open the file in write mode
with open(filepath, ‘w’, encoding=’utf-8′) as f:
# Iterate over each record in the dataset
for record in dataset:
# Dump the record as a JSON object and write it to the file
json.dump(record, f)
# Write a newline character to separate records
f.write(‘n’)
print(f”Dataset saved to {filepath}”)
def main():
“””
Main function to load, split, and save the dataset.
“””
# Load and split the dataset with a specific configuration and split ratio
dataset = load_and_split_dataset(“wikitext”, ‘wikitext-2-v1’, ‘train[:1%]’)
# Extract the train and test datasets from the split
train_dataset = dataset[‘train’]
test_dataset = dataset[‘test’]
# Save the train dataset to a JSONL file
save_dataset_to_jsonl(train_dataset, TRAIN_DATA_PATH)
# Save the test dataset to a separate JSONL file
save_dataset_to_jsonl(test_dataset, TEST_DATA_PATH)
if __name__ == “__main__”:
main()
Tip
Guidance for fine-tuning with a minimal dataset using a CPU
If you want to use a CPU for fine-tuning, this approach is ideal for those with benefit subscriptions (such as Visual Studio Enterprise Subscription) or to quickly test the fine-tuning and deployment process.
Replace dataset = load_and_split_dataset(“wikitext”, ‘wikitext-2-v1’, ‘train[:1%]’) with dataset = load_and_split_dataset(“wikitext”, ‘wikitext-2-v1’, ‘train[:10]’)
Type the following command inside your terminal to run the script and download the dataset to your local environment.
python download_data.py
Verify that the datasets were saved successfully to your local finetune-phi/data directory.
Note
Note on dataset size and fine-tuning time
In this tutorial, you use only 1% of the dataset (split=’train[:1%]’). This significantly reduces the amount of data, speeding up both the upload and fine-tuning processes. You can adjust the percentage to find the right balance between training time and model performance. Using a smaller subset of the dataset reduces the time required for fine-tuning, making the process more manageable for a tutorial.
Fine-Tune the Phi-3 Model
In this exercise, you will fine-tune the Phi-3 model using the provided dataset. First, you will define the fine-tuning process in the fine_tune.py file. Then, you will configure the Azure Machine Learning environment and initiate the fine-tuning process by running the setup_ml.py file. This script ensures that the fine-tuning occurs within the Azure Machine Learning environment.
By running setup_ml.py, you will run the fine-tuning process in the Azure Machine Learning environment.
In this exercise, you will:
Set up Azure CLI to authenticate environment
Add code to the fine_tune.py file to fine-tune the model.
Add code to and run the setup_ml.py file to initiate the fine-tuning process in Azure Machine Learning.
Run the setup_ml.py file to fine-tune the Phi-3 model using Azure Machine Learning.
Set up Azure CLI
You need to set up Azure CLI to authenticate your environment. Azure CLI allows you to manage Azure resources directly from the command line and provides the credentials necessary for Azure Machine Learning to access these resources. To get started install Azure CLI
Open a terminal window and type the following command to log in to your Azure account.
az login
Select your Azure account to use.
Select your Azure subscription to use.
Tip
Having trouble signing in to Azure? Try using a device code
Open a terminal window and type the following command to log in to your Azure account.
az login –use-device-code
Visit the website displayed in the terminal window and enter the provided code on that site.
Inside the website, select Next.
Inside the website, select the account to use in this tutorial
Inside the website, select continue to complete login.
After successful login, go back to your terminal and select your Azure subscription to use.
Add code to the fine_tune.py file
Navigate to the finetuning_dir folder and Open the fine_tune.py file in Visual Studio Code.
Add the following code into fine_tune.py.
import argparse
import sys
import logging
import os
from datasets import Dataset
import torch
import mlflow
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from trl import SFTTrainer
# To avoid the INVALID_PARAMETER_VALUE error in MLflow, disable MLflow integration
os.environ[“DISABLE_MLFLOW_INTEGRATION”] = “True”
logger = logging.getLogger(__name__)
# Logging setup
logging.basicConfig(
format=”%(asctime)s – %(levelname)s – %(name)s – %(message)s”,
datefmt=”%Y-%m-%d %H:%M:%S”,
handlers=[logging.StreamHandler(sys.stdout)],
level=logging.WARNING
)
# Model and tokenizer setup
pretrained_model_name = “microsoft/Phi-3-mini-4k-instruct”
model_kwargs = dict(
use_cache=False,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map=None,
attn_implementation=”eager”
)
def initialize_model_and_tokenizer(pretrained_model_name, model_kwargs):
“””
Initialize the model and tokenizer with the given pretrained model name and arguments.
“””
model = AutoModelForCausalLM.from_pretrained(pretrained_model_name, **model_kwargs)
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name)
tokenizer.model_max_length = 2048
tokenizer.pad_token = tokenizer.unk_token
tokenizer.pad_token_id = tokenizer.convert_tokens_to_ids(tokenizer.pad_token)
tokenizer.padding_side = ‘right’
return model, tokenizer
def preprocess_function(examples, tokenizer):
“””
Preprocess function for tokenizing the dataset.
“””
tokens = tokenizer(examples[‘text’], padding=”max_length”, truncation=True, max_length=512)
tokens[‘labels’] = tokens[‘input_ids’].copy()
return tokens
def load_and_preprocess_data(train_filepath, test_filepath, tokenizer):
“””
Load and preprocess the dataset.
“””
train_dataset = Dataset.from_json(train_filepath)
test_dataset = Dataset.from_json(test_filepath)
train_dataset = train_dataset.map(lambda examples: preprocess_function(examples, tokenizer), batched=True)
test_dataset = test_dataset.map(lambda examples: preprocess_function(examples, tokenizer), batched=True)
return train_dataset, test_dataset
def main(train_file, eval_file, model_output_dir):
“””
Main function to fine-tune the model.
“””
with mlflow.start_run():
model, tokenizer = initialize_model_and_tokenizer(pretrained_model_name, model_kwargs)
train_dataset, test_dataset = load_and_preprocess_data(train_file, eval_file, tokenizer)
# Fine-tuning settings
finetuning_settings = {
“bf16”: True,
“do_eval”: True,
“output_dir”: model_output_dir,
“eval_strategy”: “epoch”,
“learning_rate”: 1e-4,
“logging_steps”: 20,
“lr_scheduler_type”: “linear”,
“num_train_epochs”: 3,
“overwrite_output_dir”: True,
“per_device_eval_batch_size”: 4,
“per_device_train_batch_size”: 4,
“remove_unused_columns”: True,
“save_steps”: 500,
“seed”: 0,
“gradient_checkpointing”: True,
“gradient_accumulation_steps”: 1,
“warmup_ratio”: 0.2,
}
training_args = TrainingArguments(
**finetuning_settings
)
trainer = SFTTrainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
max_seq_length=2048,
dataset_text_field=”text”,
tokenizer=tokenizer,
packing=True
)
train_result = trainer.train()
metrics = train_result.metrics
trainer.log_metrics(“train”, metrics)
mlflow.transformers.log_model(
transformers_model={“model”: trainer.model, “tokenizer”: tokenizer},
artifact_path=model_output_dir, # This is a relative path to save model files within MLflow run
)
# Evaluation
tokenizer.padding_side = ‘left’
metrics = trainer.evaluate()
metrics[“eval_samples”] = len(test_dataset)
trainer.log_metrics(“eval”, metrics)
if __name__ == “__main__”:
parser = argparse.ArgumentParser()
parser.add_argument(“–train-file”, type=str, required=True, help=”Path to the training data”)
parser.add_argument(“–eval-file”, type=str, required=True, help=”Path to the evaluation data”)
parser.add_argument(“–model_output_dir”, type=str, required=True, help=”Directory to save the fine-tuned model”)
args = parser.parse_args()
main(args.train_file, args.eval_file, args.model_output_dir)
Save and close the fine_tune.py file.
Add code to the setup_ml.py file
Open the setup_ml.py file in Visual Studio Code.
Add the following code into setup_ml.py.
import logging
from azure.ai.ml import MLClient, command, Input
from azure.ai.ml.entities import Environment, AmlCompute
from azure.identity import AzureCliCredential
from config import (
AZURE_SUBSCRIPTION_ID,
AZURE_RESOURCE_GROUP_NAME,
AZURE_ML_WORKSPACE_NAME,
TRAIN_DATA_PATH,
TEST_DATA_PATH
)
# Constants
# Uncomment the following lines to use a CPU instance for training
# COMPUTE_INSTANCE_TYPE = “Standard_E16s_v3” # cpu
# COMPUTE_NAME = “cpu-e16s-v3”
# DOCKER_IMAGE_NAME = “mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest”
# CONDA_FILE = “conda_cpu.yml”
# Uncomment the following lines to use a GPU instance for training
COMPUTE_INSTANCE_TYPE = “Standard_NC6s_v3”
COMPUTE_NAME = “gpu-nc6s-v3”
DOCKER_IMAGE_NAME = “mcr.microsoft.com/azureml/curated/acft-hf-nlp-gpu:59”
CONDA_FILE = “conda.yml”
LOCATION = “eastus2” # Replace with the location of your compute cluster
FINETUNING_DIR = “./finetuning_dir” # Path to the fine-tuning script
TRAINING_ENV_NAME = “phi-3-training-environment” # Name of the training environment
MODEL_OUTPUT_DIR = “./model_output” # Path to the model output directory in azure ml
# Logging setup to track the process
logger = logging.getLogger(__name__)
logging.basicConfig(
format=”%(asctime)s – %(levelname)s – %(name)s – %(message)s”,
datefmt=”%Y-%m-%d %H:%M:%S”,
level=logging.WARNING
)
def get_ml_client():
“””
Initialize the ML Client using Azure CLI credentials.
“””
credential = AzureCliCredential()
return MLClient(credential, AZURE_SUBSCRIPTION_ID, AZURE_RESOURCE_GROUP_NAME, AZURE_ML_WORKSPACE_NAME)
def create_or_get_environment(ml_client):
“””
Create or update the training environment in Azure ML.
“””
env = Environment(
image=DOCKER_IMAGE_NAME, # Docker image for the environment
conda_file=CONDA_FILE, # Conda environment file
name=TRAINING_ENV_NAME, # Name of the environment
)
return ml_client.environments.create_or_update(env)
def create_or_get_compute_cluster(ml_client, compute_name, COMPUTE_INSTANCE_TYPE, location):
“””
Create or update the compute cluster in Azure ML.
“””
try:
compute_cluster = ml_client.compute.get(compute_name)
logger.info(f”Compute cluster ‘{compute_name}’ already exists. Reusing it for the current run.”)
except Exception:
logger.info(f”Compute cluster ‘{compute_name}’ does not exist. Creating a new one with size {COMPUTE_INSTANCE_TYPE}.”)
compute_cluster = AmlCompute(
name=compute_name,
size=COMPUTE_INSTANCE_TYPE,
location=location,
tier=”Dedicated”, # Tier of the compute cluster
min_instances=0, # Minimum number of instances
max_instances=1 # Maximum number of instances
)
ml_client.compute.begin_create_or_update(compute_cluster).wait() # Wait for the cluster to be created
return compute_cluster
def create_fine_tuning_job(env, compute_name):
“””
Set up the fine-tuning job in Azure ML.
“””
return command(
code=FINETUNING_DIR, # Path to fine_tune.py
command=(
“python fine_tune.py “
“–train-file ${{inputs.train_file}} “
“–eval-file ${{inputs.eval_file}} “
“–model_output_dir ${{inputs.model_output}}”
),
environment=env, # Training environment
compute=compute_name, # Compute cluster to use
inputs={
“train_file”: Input(type=”uri_file”, path=TRAIN_DATA_PATH), # Path to the training data file
“eval_file”: Input(type=”uri_file”, path=TEST_DATA_PATH), # Path to the evaluation data file
“model_output”: MODEL_OUTPUT_DIR
}
)
def main():
“””
Main function to set up and run the fine-tuning job in Azure ML.
“””
# Initialize ML Client
ml_client = get_ml_client()
# Create Environment
env = create_or_get_environment(ml_client)
# Create or get existing compute cluster
create_or_get_compute_cluster(ml_client, COMPUTE_NAME, COMPUTE_INSTANCE_TYPE, LOCATION)
# Create and Submit Fine-Tuning Job
job = create_fine_tuning_job(env, COMPUTE_NAME)
returned_job = ml_client.jobs.create_or_update(job) # Submit the job
ml_client.jobs.stream(returned_job.name) # Stream the job logs
# Capture the job name
job_name = returned_job.name
print(f”Job name: {job_name}”)
if __name__ == “__main__”:
main()
Replace COMPUTE_INSTANCE_TYPE, COMPUTE_NAME, and LOCATION with your specific details.
# Uncomment the following lines to use a GPU instance for training
COMPUTE_INSTANCE_TYPE = “Standard_NC6s_v3”
COMPUTE_NAME = “gpu-nc6s-v3”
…
LOCATION = “eastus2” # Replace with the location of your compute cluster
Tip
Guidance for fine-tuning with a minimal dataset using a CPU
If you want to use a CPU for fine-tuning, this approach is ideal for those with benefit subscriptions (such as Visual Studio Enterprise Subscription) or to quickly test the fine-tuning and deployment process.
Open the conda.yml file.
Delete torchvision~=0.18 as it is only compatible with GPU environments.
Open the setup_ml file.
Replace COMPUTE_INSTANCE_TYPE, COMPUTE_NAME, and DOCKER_IMAGE_NAME with the following. If you do not have access to Standard_E16s_v3, you can use an equivalent CPU instance or request a new quota.
Replace LOCATION with your specific details.
# Uncomment the following lines to use a CPU instance for training
COMPUTE_INSTANCE_TYPE = “Standard_E16s_v3” # cpu
COMPUTE_NAME = “cpu-e16s-v3”
DOCKER_IMAGE_NAME = “mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest”
CONDA_FILE = “conda.yml”
LOCATION = “eastus2” # Replace with the location of your compute cluster
Type the following command to run the setup_ml.py script and start the fine-tuning process in Azure Machine Learning.
python setup_ml.py
In this exercise, you successfully fine-tuned the Phi-3 model using Azure Machine Learning. By running the setup_ml.py script, you have set up the Azure Machine Learning environment and initiated the fine-tuning process defined in fine_tune.py file. Please note that the fine-tuning process can take a considerable amount of time. After running the python setup_ml.py command, you need to wait for the process to complete. You can monitor the status of the fine-tuning job by following the link provided in the terminal to the Azure Machine Learning portal. In the next series, you will deploy the fine-tuned model and integrate it with Prompt Flow.
Deploy the fine-tuned model
To integrate the fine-tuned Phi-3 model with Prompt Flow, you need to deploy the model to make it accessible for real-time inference. This process involves registering the model, creating an online endpoint, and deploying the model.
In this exercise, you will:
Set the model name, endpoint name, and deployment name for deployment.
Register the fine-tuned model in the Azure Machine Learning workspace.
Create an online endpoint.
Deploy the registered fine-tuned Phi-3 model.
Set the model name, endpoint name, and deployment name for deployment
Open config.py file.
Replace AZURE_MODEL_NAME = “your_fine_tuned_model_name” with the desired name for your model.
Replace AZURE_ENDPOINT_NAME = “your_fine_tuned_model_endpoint_name” with the desired name for your endpoint.
Replace AZURE_DEPLOYMENT_NAME = “your_fine_tuned_model_deployment_name” with the desired name for your deployment.
Deploy the fine-tuned model
Running the deploy_model.py file automates the entire deployment process. It registers the model, creates an endpoint, and executes the deployment based on the settings specified in the config.py file, which includes the model name, endpoint name, and deployment name.
Open the deploy_model.py file in Visual Studio Code.
Add the following code into deploy_model.py.
import logging
from azure.identity import AzureCliCredential
from azure.ai.ml import MLClient
from azure.ai.ml.entities import Model, ProbeSettings, ManagedOnlineEndpoint, ManagedOnlineDeployment, IdentityConfiguration, ManagedIdentityConfiguration, OnlineRequestSettings
from azure.ai.ml.constants import AssetTypes
# Configuration imports
from config import (
AZURE_SUBSCRIPTION_ID,
AZURE_RESOURCE_GROUP_NAME,
AZURE_WORKSPACE_NAME,
AZURE_MANAGED_IDENTITY_RESOURCE_ID,
AZURE_MANAGED_IDENTITY_CLIENT_ID,
AZURE_MODEL_NAME,
AZURE_ENDPOINT_NAME,
AZURE_DEPLOYMENT_NAME
)
# Constants
JOB_NAME = “your-job-name”
COMPUTE_INSTANCE_TYPE = “Standard_E4s_v3”
deployment_env_vars = {
“SUBSCRIPTION_ID”: AZURE_SUBSCRIPTION_ID,
“RESOURCE_GROUP_NAME”: AZURE_RESOURCE_GROUP_NAME,
“UAI_CLIENT_ID”: AZURE_MANAGED_IDENTITY_CLIENT_ID,
}
# Logging setup
logging.basicConfig(
format=”%(asctime)s – %(levelname)s – %(name)s – %(message)s”,
datefmt=”%Y-%m-%d %H:%M:%S”,
level=logging.DEBUG
)
logger = logging.getLogger(__name__)
def get_ml_client():
“””Initialize and return the ML Client.”””
credential = AzureCliCredential()
return MLClient(credential, AZURE_SUBSCRIPTION_ID, AZURE_RESOURCE_GROUP_NAME, AZURE_WORKSPACE_NAME)
def register_model(ml_client, model_name, job_name):
“””Register a new model.”””
model_path = f”azureml://jobs/{job_name}/outputs/artifacts/paths/model_output”
logger.info(f”Registering model {model_name} from job {job_name} at path {model_path}.”)
run_model = Model(
path=model_path,
name=model_name,
description=”Model created from run.”,
type=AssetTypes.MLFLOW_MODEL,
)
model = ml_client.models.create_or_update(run_model)
logger.info(f”Registered model ID: {model.id}”)
return model
def delete_existing_endpoint(ml_client, endpoint_name):
“””Delete existing endpoint if it exists.”””
try:
endpoint_result = ml_client.online_endpoints.get(name=endpoint_name)
logger.info(f”Deleting existing endpoint {endpoint_name}.”)
ml_client.online_endpoints.begin_delete(name=endpoint_name).result()
logger.info(f”Deleted existing endpoint {endpoint_name}.”)
except Exception as e:
logger.info(f”No existing endpoint {endpoint_name} found to delete: {e}”)
def create_or_update_endpoint(ml_client, endpoint_name, description=””):
“””Create or update an endpoint.”””
delete_existing_endpoint(ml_client, endpoint_name)
logger.info(f”Creating new endpoint {endpoint_name}.”)
endpoint = ManagedOnlineEndpoint(
name=endpoint_name,
description=description,
identity=IdentityConfiguration(
type=”user_assigned”,
user_assigned_identities=[ManagedIdentityConfiguration(resource_id=AZURE_MANAGED_IDENTITY_RESOURCE_ID)]
)
)
endpoint_result = ml_client.online_endpoints.begin_create_or_update(endpoint).result()
logger.info(f”Created new endpoint {endpoint_name}.”)
return endpoint_result
def create_or_update_deployment(ml_client, endpoint_name, deployment_name, model):
“””Create or update a deployment.”””
logger.info(f”Creating deployment {deployment_name} for endpoint {endpoint_name}.”)
deployment = ManagedOnlineDeployment(
name=deployment_name,
endpoint_name=endpoint_name,
model=model.id,
instance_type=COMPUTE_INSTANCE_TYPE,
instance_count=1,
environment_variables=deployment_env_vars,
request_settings=OnlineRequestSettings(
max_concurrent_requests_per_instance=3,
request_timeout_ms=180000,
max_queue_wait_ms=120000
),
liveness_probe=ProbeSettings(
failure_threshold=30,
success_threshold=1,
period=100,
initial_delay=500,
),
readiness_probe=ProbeSettings(
failure_threshold=30,
success_threshold=1,
period=100,
initial_delay=500,
),
)
deployment_result = ml_client.online_deployments.begin_create_or_update(deployment).result()
logger.info(f”Created deployment {deployment.name} for endpoint {endpoint_name}.”)
return deployment_result
def set_traffic_to_deployment(ml_client, endpoint_name, deployment_name):
“””Set traffic to the specified deployment.”””
try:
# Fetch the current endpoint details
endpoint = ml_client.online_endpoints.get(name=endpoint_name)
# Log the current traffic allocation for debugging
logger.info(f”Current traffic allocation: {endpoint.traffic}”)
# Set the traffic allocation for the deployment
endpoint.traffic = {deployment_name: 100}
# Update the endpoint with the new traffic allocation
endpoint_poller = ml_client.online_endpoints.begin_create_or_update(endpoint)
updated_endpoint = endpoint_poller.result()
# Log the updated traffic allocation for debugging
logger.info(f”Updated traffic allocation: {updated_endpoint.traffic}”)
logger.info(f”Set traffic to deployment {deployment_name} at endpoint {endpoint_name}.”)
return updated_endpoint
except Exception as e:
# Log any errors that occur during the process
logger.error(f”Failed to set traffic to deployment: {e}”)
raise
def main():
ml_client = get_ml_client()
registered_model = register_model(ml_client, AZURE_MODEL_NAME, JOB_NAME)
logger.info(f”Registered model ID: {registered_model.id}”)
endpoint = create_or_update_endpoint(ml_client, AZURE_ENDPOINT_NAME, “Endpoint for finetuned Phi-3 model”)
logger.info(f”Endpoint {AZURE_ENDPOINT_NAME} is ready.”)
try:
deployment = create_or_update_deployment(ml_client, AZURE_ENDPOINT_NAME, AZURE_DEPLOYMENT_NAME, registered_model)
logger.info(f”Deployment {AZURE_DEPLOYMENT_NAME} is created for endpoint {AZURE_ENDPOINT_NAME}.”)
set_traffic_to_deployment(ml_client, AZURE_ENDPOINT_NAME, AZURE_DEPLOYMENT_NAME)
logger.info(f”Traffic is set to deployment {AZURE_DEPLOYMENT_NAME} at endpoint {AZURE_ENDPOINT_NAME}.”)
except Exception as e:
logger.error(f”Failed to create or update deployment: {e}”)
if __name__ == “__main__”:
main()
Perform the following tasks to get the JOB_NAME:
Navigate to Azure Machine Learning resource that you created.
Select Studio web URL to open the Azure Machine Learning workspace.
Select Jobs from the left side tab.
Select the experiment for fine-tuning. For example, finetunephi.
Select the job that you created.
Copy and paste your job Name into the JOB_NAME = “your-job-name” in deploy_model.py file.
Replace COMPUTE_INSTANCE_TYPE with your specific details.
Type the following command to run the deploy_model.py script and start the deployment process in Azure Machine Learning.
python deploy_model.py
Warning
To avoid additional charges to your account, make sure to delete the created endpoint in the Azure Machine Learning workspace.
Check deployment status in Azure Machine Learning Workspace
Navigate to Azure Machine Learning resource that you created.
Select Studio web URL to open the Azure Machine Learning workspace.
Select Endpoints from the left side tab.
Select endpoint that you created.
On this page, you can manage the endpoints created during the deployment process.
Integrate the custom Phi-3 model with Prompt Flow
After successfully deploying your fine-tuned model, you can now integrate it with Prompt Flow to use your model in real-time applications, enabling a variety of interactive tasks with your custom Phi-3 model.
In this exercise, you will:
Set api key and endpoint uri of the fine-tuned Phi-3 model.
Add code to the flow.dag.yml file.
Add code to the integrate_with_promptflow.py file.
Test your custom Phi-3 model on Prompt flow.
Set api key and endpoint uri of the fine-tuned Phi-3 model
Navigate to the Azure Machine learning workspace that you created.
Select Endpoints from the left side tab.
Select endpoint that you created.
Select Consume from the navigation menu.
Copy and paste your REST endpoint into the config.py file, replacing AZURE_ML_ENDPOINT = “your_fine_tuned_model_endpoint_uri” with your REST endpoint.
Copy and paste your Primary key into the config.py file, replacing AZURE_ML_API_KEY = “your_fine_tuned_model_api_key” with your Primary key.
Add code to the flow.dag.yml file
Open the flow.dag.yml file in Visual Studio Code.
Add the following code into flow.dag.yml.
inputs:
input_data:
type: string
default: “Who founded Microsoft?”
outputs:
answer:
type: string
reference: ${integrate_with_promptflow.output}
nodes:
– name: integrate_with_promptflow
type: python
source:
type: code
path: integrate_with_promptflow.py
inputs:
input_data: ${inputs.input_data}
Add code to the integrate_with_promptflow.py file
Open the integrate_with_promptflow.py file in Visual Studio Code.
Add the following code into integrate_with_promptflow.py.
import logging
import requests
from promptflow.core import tool
import asyncio
import platform
from config import (
AZURE_ML_ENDPOINT,
AZURE_ML_API_KEY
)
# Logging setup
logging.basicConfig(
format=”%(asctime)s – %(levelname)s – %(name)s – %(message)s”,
datefmt=”%Y-%m-%d %H:%M:%S”,
level=logging.DEBUG
)
logger = logging.getLogger(__name__)
def query_azml_endpoint(input_data: list, endpoint_url: str, api_key: str) -> str:
“””
Send a request to the Azure ML endpoint with the given input data.
“””
headers = {
“Content-Type”: “application/json”,
“Authorization”: f”Bearer {api_key}”
}
data = {
“input_data”: [input_data],
“params”: {
“temperature”: 0.7,
“max_new_tokens”: 128,
“do_sample”: True,
“return_full_text”: True
}
}
try:
response = requests.post(endpoint_url, json=data, headers=headers)
response.raise_for_status()
result = response.json()[0]
logger.info(“Successfully received response from Azure ML Endpoint.”)
return result
except requests.exceptions.RequestException as e:
logger.error(f”Error querying Azure ML Endpoint: {e}”)
raise
def setup_asyncio_policy():
“””
Setup asyncio event loop policy for Windows.
“””
if platform.system() == ‘Windows’:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
logger.info(“Set Windows asyncio event loop policy.”)
@tool
def my_python_tool(input_data: str) -> str:
“””
Tool function to process input data and query the Azure ML endpoint.
“””
setup_asyncio_policy()
return query_azml_endpoint(input_data, AZURE_ML_ENDPOINT, AZURE_ML_API_KEY)
Type the following command to run the deploy_model.py script and start the deployment process in Azure Machine Learning.
pf flow serve –source ./ –port 8080 –host localhost
Here’s an example of the results: Now you can chat with your custom Phi-3 model. It is recommended to ask questions based on the data used for fine-tuning.
Congratulations!
You’ve completed this tutorial
Congratulations! You have successfully completed the tutorial on fine-tuning and integrating custom Phi-3 models with Prompt Flow. This tutorial introduced the simplest method of fine-tuning, avoiding additional techniques such as LoRA or QLoRA, and using MLflow to streamline the fine-tuning and deployment process. Advanced techniques and detailed explanations will be covered in the next series.
Clean Up Azure Resources
Cleanup your Azure resources to avoid additional charges to your account. Go to the Azure portal and delete the following resources:
The Azure Machine Learning resource.
The Azure Machine Learning model endpoint.
Source Code for the Tutorial
You can find the complete source code for this tutorial in the following repository:
skytin1004/Fine-Tune-and-Integrate-Custom-Phi-3-Models-with-Prompt-Flow
Reference
microsoft/Phi-3CookBook
Azure/azure-llm-fine-tuning
Microsoft Tech Community – Latest Blogs –Read More
Missing action item
In the last few Windows updates I’ve noticed that “Copilot in Windows (preview)” shows up at the rightmost edge of the taskbar. That’s fine, since I am experimenting with Copilot. However, before the last few updates, I was able to click on an icon (a vertical bar) to display the desktop, this item has gone missing.
Is there a way to restore this icon, or is it not available anymore??l
In the last few Windows updates I’ve noticed that “Copilot in Windows (preview)” shows up at the rightmost edge of the taskbar. That’s fine, since I am experimenting with Copilot. However, before the last few updates, I was able to click on an icon (a vertical bar) to display the desktop, this item has gone missing.Is there a way to restore this icon, or is it not available anymore??l Read More
Fingerprint not being accepted on UAC prompt
I have Windows 11 Pro 23H2 with all the latest updates. I use a Dell XPS laptop which has a fingerprint sensor. I am the only user of this laptop and I use my own user account which is a Microsoft account and I have set it as an admin account. The finger print sensor is working decently enough.
But the issue I am facing is, though it gives me an option to scan my fingerprint, after doing so it says ‘username or password is incorrect’. Everytime I have to enter the PIN. I wish I could share the screenshot, but it does not allow me to do that for the UAC prompt. On the UAC prompt, when I click on the fingerprint option, the ‘Yes/No’ buttons at the bottom change to only ‘No’.
Why is this happening? Please help me. Thanks.
I have Windows 11 Pro 23H2 with all the latest updates. I use a Dell XPS laptop which has a fingerprint sensor. I am the only user of this laptop and I use my own user account which is a Microsoft account and I have set it as an admin account. The finger print sensor is working decently enough. I set it to: ‘Prompt for credentials on the secure desktop’. So now, if I want to run any unknown exe file, or even make any changes to the Windows settings etc., I have to enter my PIN or Microsoft account password, or scan my fingerprint.But the issue I am facing is, though it gives me an option to scan my fingerprint, after doing so it says ‘username or password is incorrect’. Everytime I have to enter the PIN. I wish I could share the screenshot, but it does not allow me to do that for the UAC prompt. On the UAC prompt, when I click on the fingerprint option, the ‘Yes/No’ buttons at the bottom change to only ‘No’.Why is this happening? Please help me. Thanks. Read More
Missing timeline, people and goals view
Hi all,
I am missing some of the new views in planner, including timeline view, people view and goals view. I have the new teams, the new planner and are on a project subscription 3 so should have Planner Premium included. How do I get the views I am missing?
The views currently have are grid, board, charts and schedule.
Hi all, I am missing some of the new views in planner, including timeline view, people view and goals view. I have the new teams, the new planner and are on a project subscription 3 so should have Planner Premium included. How do I get the views I am missing? The views currently have are grid, board, charts and schedule. Read More
Restoring Open Tabs on File Explorer after Restart?
Hi, techcommunity.
Does anyone know of a way to restore/retain all the open tabs in file explorer after a restart? I poked around in preferences a bit but couldn’t find anything. I mean all the tabs in the same file explorer window.
Hi, techcommunity.Does anyone know of a way to restore/retain all the open tabs in file explorer after a restart? I poked around in preferences a bit but couldn’t find anything. I mean all the tabs in the same file explorer window. Read More
Help Needed: Install Windows 11 Without TPM on My PC
I am trying to upgrade my Windows 10 PC to Windows 11 as Microsoft asks me to do that from time to time. I downloaded the Windows 11 Installation Assistance but it tells me that this PC can’t run this version of Windows 11.
Later, I downloaded the PC Heather Check app and it says my PC is missing TPM. I’m really eager to upgrade, but I’m stuck at this roadblock and not sure how to proceed. Has anyone successfully installed Windows 11 without TPM?
If so, could you please share the steps or any workarounds you used? I would greatly appreciate any suggestion on how to bypass this requirement. I’m open to trying different methods or tools to make this upgrade possible.
I am trying to upgrade my Windows 10 PC to Windows 11 as Microsoft asks me to do that from time to time. I downloaded the Windows 11 Installation Assistance but it tells me that this PC can’t run this version of Windows 11. Later, I downloaded the PC Heather Check app and it says my PC is missing TPM. I’m really eager to upgrade, but I’m stuck at this roadblock and not sure how to proceed. Has anyone successfully installed Windows 11 without TPM? If so, could you please share the steps or any workarounds you used? I would greatly appreciate any suggestion on how to bypass this requirement. I’m open to trying different methods or tools to make this upgrade possible. Read More
System tray icons fuzzy/blurry/pixelated
Hello, been having an odd issue for the last few months that I am at a loss for how to fix, my system tray icons are getting all blurred and fuzzed for no apparent reason:
I have tried a few things, cleared the icon cache with winaerotweaker, reset all winaerotweaker settings, changed my DPI up and down, changed scaling settings in nvidia control panel, but nothing seems to work reliably. The best i got was most of the icons looking normal after some dpi changes, except for the safely remove hardware icon which refused to go back to normal with the others. However even managing this, a restart put them back to blurry mode… Does anyone have a clue whats going on? I’d prefer not to do a fresh windows install just to fix this.
Hello, been having an odd issue for the last few months that I am at a loss for how to fix, my system tray icons are getting all blurred and fuzzed for no apparent reason: I have tried a few things, cleared the icon cache with winaerotweaker, reset all winaerotweaker settings, changed my DPI up and down, changed scaling settings in nvidia control panel, but nothing seems to work reliably. The best i got was most of the icons looking normal after some dpi changes, except for the safely remove hardware icon which refused to go back to normal with the others. However even managing this, a restart put them back to blurry mode… Does anyone have a clue whats going on? I’d prefer not to do a fresh windows install just to fix this. Read More
How to stop Outlook from auto-sorting Scheduling emails into deleted?
Title says it all.
You can’t even set rules in outlook; it bypasses them. Emails get put into deleted automatically.
Title says it all. You can’t even set rules in outlook; it bypasses them. Emails get put into deleted automatically. Read More
Hello friends
I am from India I am happy :smiling_face_with_smiling_eyes:
I am from India I am happy :smiling_face_with_smiling_eyes: Read More
Scanning Report using MS Defender.
I am trying to create a report that will give me current AVscan status and timestamp, last time AVscan status with timestamp, ActionType and StartedBy.
I am trying to create a report that will give me current AVscan status and timestamp, last time AVscan status with timestamp, ActionType and StartedBy.DeviceEvents| where ActionType has_any (“AntivirusScanCancelled”, “AntivirusScanCompleted”)| extend AdditionalFields = todynamic(AdditionalFields)| extend ScanType = AdditionalFields.[“ScanTypeIndex”], StartedBy= AdditionalFields.[“User”]| extend ServerName = extract(“([^.]+)\.(.+)”, 1, DeviceName)| extend Domain = extract(“([^.]+)\.(.+)”, 2, DeviceName)| project Timestamp, ServerName, Domain, ActionType, ScanType, StartedByAbove query giving me a lot of devices where i am trying to get distinct values for the devices with last AV scan. Looking for better advise on that query please. Read More
Not able to add Basic Info to Project Details Page
I am creating a PDP and want to add the Basic Info web part to the page. When I am click on Add Web Part I am not seeing the Category for Project Web App under Categories.
Any ideas on why I am not able to find the Basic Info or any of the other Project Web App web parts?
I am creating a PDP and want to add the Basic Info web part to the page. When I am click on Add Web Part I am not seeing the Category for Project Web App under Categories. Any ideas on why I am not able to find the Basic Info or any of the other Project Web App web parts? Read More
Macro to move row to another sheet
Hello all,
I’m still new to VBA and practicing through trial and error. I’m currently working on an excel document that is an inventory spreadsheet. When we perform annual inventory checks I want to use a barcode scanner to quickly go through and check the inventory.
Is there a way to build a macro in VBA that will allow me to scan the barcodes and auto run it against a certain column and if the barcode scanned is present it will then pull that entire row from sheet 1 to and place it into sheet 2?
Thanks in advance for any and all your help!
Hello all, I’m still new to VBA and practicing through trial and error. I’m currently working on an excel document that is an inventory spreadsheet. When we perform annual inventory checks I want to use a barcode scanner to quickly go through and check the inventory. Is there a way to build a macro in VBA that will allow me to scan the barcodes and auto run it against a certain column and if the barcode scanned is present it will then pull that entire row from sheet 1 to and place it into sheet 2? Thanks in advance for any and all your help! Read More
Update your sign-in technology before September 16th, 2024 to maintain email access.
Hi, Received this email (subject line) and wondered how it effects a hotmail email address….?
Hi, Received this email (subject line) and wondered how it effects a hotmail email address….? Read More
How to get lane number of a USB 3.2 Gen 2×2(20Gb/s) device using Windows APIs
I can get USB 3.2 Gen 2(10Gb/s) information from DeviceIsOperatingAtSuperSpeedPlusOrHigher using IOCTL_USB_GET_NODE_CONNECTION_INFORMATION_EX_V2.
How can I get lane number of a USB 3.2 Gen 2×2 device using Windows APIs?
I can get USB 3.2 Gen 2(10Gb/s) information from DeviceIsOperatingAtSuperSpeedPlusOrHigher using IOCTL_USB_GET_NODE_CONNECTION_INFORMATION_EX_V2. How can I get lane number of a USB 3.2 Gen 2×2 device using Windows APIs? Read More
Reading PDFs or TXT files
I am new to CoPilot for MS 365. I thought if there was a document, whether Docx, pdf, txt, etc. that CoPilot could actually read its contents and include that information and knowledge in response to a prompt. Even though it says it can find that document in my One Drive folders, it says it cannot read them. I am pretty sure Open AI’s ChatGPT allows this so I don’t understand this.
Also, when I switch to the Web version, it appears there is no way to retain my prior Chat’s as there is with Open AI. Is that correct? If so that is not very good as I would like to be able to build on the prior chat’s and have Copilot remember our prior conversations.
I am new to CoPilot for MS 365. I thought if there was a document, whether Docx, pdf, txt, etc. that CoPilot could actually read its contents and include that information and knowledge in response to a prompt. Even though it says it can find that document in my One Drive folders, it says it cannot read them. I am pretty sure Open AI’s ChatGPT allows this so I don’t understand this. Also, when I switch to the Web version, it appears there is no way to retain my prior Chat’s as there is with Open AI. Is that correct? If so that is not very good as I would like to be able to build on the prior chat’s and have Copilot remember our prior conversations. Read More
A way to set maximum number of bookings per day
Hi, I currently have one service I offer in my Bookings but want to set it so that I only do a maximum of say 4 of that service in a day. On Gemini, Copilot, and another Microsoft forum post there’s instructions to “go to the “Booking page” tab in the Microsoft Bookings app, select the service you want to limit, and then click on the “Edit” button. From there, you can set the maximum number of bookings per day for that service.” or “Go to the Booking page tab in Bookings. Select the service you want to limit. Click “Edit” and set the “Maximum number of bookings per day.” but when I got to edit my service, I only see these options, nothing about a maximum number of bookings option:
Hi, I currently have one service I offer in my Bookings but want to set it so that I only do a maximum of say 4 of that service in a day. On Gemini, Copilot, and another Microsoft forum post there’s instructions to “go to the “Booking page” tab in the Microsoft Bookings app, select the service you want to limit, and then click on the “Edit” button. From there, you can set the maximum number of bookings per day for that service.” or “Go to the Booking page tab in Bookings. Select the service you want to limit. Click “Edit” and set the “Maximum number of bookings per day.” but when I got to edit my service, I only see these options, nothing about a maximum number of bookings option: Read More
What is the URL to open M365 Profile Card in modal?
Anyone know the URL to open the user’s M365 profile card?
Profile cards in Microsoft 365 – Microsoft Support
On a SharePoint list, when adding a people column, we could usually click on the name and it pops up the profile card. But since recent updates with SharePoint list form, this feature is no longer possible, at least in our tenant (GCC). Now, the profile card may or may not pop up when hovering over the name. I’m hoping there’s a way to add a hyperlink to open the same M365 profile card.
Use column formatting to customize SharePoint | Microsoft Learn
Anyone know the URL to open the user’s M365 profile card?Profile cards in Microsoft 365 – Microsoft SupportOn a SharePoint list, when adding a people column, we could usually click on the name and it pops up the profile card. But since recent updates with SharePoint list form, this feature is no longer possible, at least in our tenant (GCC). Now, the profile card may or may not pop up when hovering over the name. I’m hoping there’s a way to add a hyperlink to open the same M365 profile card.Use column formatting to customize SharePoint | Microsoft Learn Read More