Introduction

Microsoft Entra ID token-based auth mode for managed online endpoints in Azure Machine Learning is now generally available. This new auth mode makes identity and access management easier when using models hosted on Azure.

Plus, to deploy models securely and efficiently, Azure Machine Learning offers another great feature: managed online endpoints. In this blog, we’ll see how Microsoft Entra ID helps with endpoint authentication/authorization, and learn some important concepts about Azure Machine Learning managed online endpoints and how to use them with Microsoft Entra ID.

Managed Online Endpoints: A Turnkey Solution to simplify deployments

Managed online endpoints are designed to simplify machine learning model deployment. Here are some benefits:

Flexibility: Deploy models with no-code, low-code, or full code options
Scalability: Handle high request volumes and expand to hundreds of nodes
Cost Optimization: Autoscaling and cost monitoring for individual models
DevOps Efficiency: GitOps-friendly interfaces, local testing, and safe model rollouts
Security Compliance: Secure authentication, network isolation, and managed identity
Infrastructure Management : Reduce infrastructure complexity

Both Azure AI Studio and Azure Machine Learning use managed online endpoints to improve model deployment experiences.

Key concepts: Endpoints and deployments

Managed online endpoints decouple the concepts of endpoints and deployments.

Endpoint: A logical entity that represents a service or API where you can send requests for inferencing. It acts as the entry point for your machine learning models.
Deployment: A set of resources and logic required for hosting the actual model that performs inferencing. It includes the necessary infrastructure, such as CPU or GPU machines, the environment, and the inferencing logic to consume the models.

When you deploy a model, you create a deployment behind an endpoint. This deployment contains the model, any custom code, and other dependencies.

A key benefit of this separation is that it decouples the interface presented to clients (the endpoint) from the implementation details (the deployment). This separation allows you to manage deployments independently without affecting the overall endpoint. For example, you can use this to maintain multiple deployments for multiple model versions under a single endpoint, or to safely roll out new model versions without downtime.

Key concepts: Control plane operation vs data plane operation

Control plane operations

These operations manage and modify the online endpoints. They involve creating, reading, updating, and deleting online endpoints and deployments. They send requests to the Azure Machine Learning workspace.

To authenticate, you need a Microsoft Entra token. To authorize, you need Azure RBAC action for Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write, delete, and read. We’ll see what this means later.

Data plane operations

These operations do not change the online endpoints, but use data to work with them. For example, sending a scoring request to an online endpoint and getting a response. They send requests to the endpoint’s scoring URI.

To authenticate, you can use a key, Microsoft Entra token, or Azure Machine Learning token. You specify which mechanism to use by choosing `auth_mode` when creating a managed online endpoint. If you choose Microsoft Entra to authorize, you need Azure RBAC action for Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action.

What’s new about this? Data plane operations for online endpoint now supports Microsoft Entra token for full RBAC authorization.

Why Microsoft Entra ID is useful

Consume multiple endpoints using a single token with full RBAC support

Key auth endpoints have two keys for each endpoint, which may complicate integration with applications when you have many endpoints. Microsoft Entra ID lets you use one token for many endpoints, by giving the right role to a bigger scope you pick. For example, if the IT admin gives a group of Azure identities the right role for data plane actions over a resource group, those Azure identities can call all the endpoints in the resource group.

Streamline control plane and data plane operations

Expanding from the above scenario, if the group of Azure identities are assigned the proper role for both control plane and data plane operations over the scope of a subscription, the Azure identities will be able to not only create/update/delete/read the endpoints and deployments and but also invoke the endpoints in the subscription. You can adjust the RBAC actions and the scope as needed.

Seeing it in action

You don’t need tokens if you sign in and use CLI/SDK/UI. They manage them for you. But you can also choose to do it yourself, if you use REST API with endpoints. We use CLI to keep it simple in this blog. We assume you have set up:

Azure AI hub / project or Azure Machine Learning workspace
Development environment where you have set up CLI and ml extension, or a cloud-based setting that is already prepared for you, such as Compute Instance, Visual Studio Code for Web, or Azure Cloud Shell

1. Create the endpoint and the deployment

Suppose you want to deploy an LLM model for a chat completion scenario. You’ll want to create an endpoint and a deployment. Earlier, we saw that this is a control plane operation. First things first, you would sign into Azure.

az login

Once signed in, the authentication is done and the CLI session will run on your Azure identity. When you perform an operation, the CLI will interact with the backend services (via REST APIs) and backend services will check authorization if the operation is permitted for your Azure identity.

Note that the IT admin with the permission of `Microsoft.Authorization/roleAssignments/write` will be able to assign the proper role for an operation to your Azure identity, if it isn’t already assigned such a role.

Now suppose your definitions of the endpoint and deployment for your LLM model are defined as YAML files. Especially your endpoint definition YAML will look like:

$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
name: my-endp1
auth_mode: aad_token

You can run the following commands to create the endpoint and the deployment:

az ml online-endpoint create -n my-endp1 -f ./endpoint.yaml
az ml online-deployment create -n blue -e my-endp1 -f ./deployment.yaml

The above two CLI commands send the requests to the Azure AI Studio hub or Azure Machine Learning workspace, and if your Azure identity has the permission to create them (more specifically, if your Azure identity is assigned an Azure role with ` Microsoft.MachineLearningServices/workspaces/onlineEndpoints/write` action over the scope of the Azure AI hub or Azure Machine Learning workspace resource), the creation requests will be processed.

NOTE

If you’re using REST API to create endpoints and deployments, you’ll need to get the Microsoft Entra token from the resource endpoint https://management.azure.com and pass it in the header. More on this at Get the Microsoft Entra token for control plane operations.

2. Invoke the endpoint

Now it’s time to consume the model. As mentioned earlier, this is a data plane operation.

You can formulate a request file that has the expected payload that the model expects, for example, a JSON input including input text, chat history, and/or parameters such as max response, temperature etc. Then you can run the following command to invoke the endpoint:

az ml online-endpoint invoke -n my-endp1 -r ./request.json

The CLI command will send the request to the scoring URI of the endpoint, and if your Azure identity has the permission to invoke the endpoint (more specifically, if your Azure identity is assigned an Azure role with ` Microsoft.MachineLearningServices/workspaces/onlineEndpoints/score/action` action over the scope of the managed online endpoint resource), the invoke request will be processed.

NOTE

If you’re using REST API to invoke endpoints, you’ll need to get Microsoft Entra token from resource endpoint https://ml.azure.com and pass it in the header. More on this at Get the key or token for data plane operations.

Conclusion

Azure Machine Learning managed online endpoints, combined with Microsoft Entra ID, provide a seamless and secure way to deploy and consume your AI/ML models. By leveraging these features, you can focus on delivering value to your organization without worrying about infrastructure complexities.

Learn more

What are online endpoints? – Azure Machine Learning
Deploy machine learning models to online endpoints for inference – Azure Machine Learning | Microsoft Learn
Authentication for managed online endpoints – Azure Machine Learning | Microsoft Learn
Authenticate clients for online endpoints – Azure Machine Learning | Microsoft Learn

​Microsoft Tech Community – Latest Blogs –Read More 

2024年のB I T G E Tのベスト紹介コードは「qp29」です。このコードを使用すると、取引が 30% オフになります。さらに、プロモーション コード「qp29」を使用して B I T G E T に登録した新規ユーザーは、最大 5,005 USDT の独占報酬を確保できます。

B I T G E T 紹介コード qp29 の利点
B I T G E T 紹介コード qp29 は、魅力的な報酬を獲得しながら取引手数料を節約する優れた方法を提供します。このコードを入力すると、取引手数料が永続的に 30% 割引されます。さらに、個人紹介コードを友達と共有すると、取引手数料の 50% のボーナスを受け取ることができます。この機会を活用して、プラットフォームに新しいユーザーを呼び込みながら収益を増やしてください。

2024 年の最高の B I T G E T 紹介コード
2024年の推奨B I T G E T紹介コードはqp29です。このコードでサインアップすると、ボーナスとして最大 5005 USDT を獲得できます。このコードを友達と共有すると 50% のコミッションを獲得でき、最大 5005 USDT のサインアップ ボーナスを確保できます。これは、他のユーザーの参加を奨励しながら、追加のメリットで取引体験を向上させる優れた方法です。

B I T G E T紹介コードの使い方
B I T G E T 紹介コードは、プラットフォームにまだ登録していない新規ユーザー専用です。コードを使用するには、次の手順に従います。

B I T G E T Web サイトにアクセスし、「サインイン」をクリックします。
ユーザーの詳細を入力し、KYC および AML 手順を実行します。
紹介コードの入力を求められたら、「qp29」と入力します。
登録プロセスを完了し、必要な検証をすべて完了します。
すべての条件が満たされると、すぐにウェルカムボーナスを受け取ります。
B I T G E T 紹介コードを使用する理由?
永久割引: コード qp29 を使用すると、すべての取引手数料が自動的に 30% 割引されます。
寛大なウェルカムボーナス: 新規ユーザーは最大 5,005 USDT を受け取ることができます。
追加の収益: コードを共有すると、50% のコミッションを獲得できます。
この機会を利用して、現在の B I T G E T 紹介コード qp29 で特典を確保してください。最大 5,005 USDT を受け取り、取引手数料の永続的な割引の恩恵を受けます。

​2024年のB I T G E Tのベスト紹介コードは「qp29」です。このコードを使用すると、取引が 30% オフになります。さらに、プロモーション コード「qp29」を使用して B I T G E T に登録した新規ユーザーは、最大 5,005 USDT の独占報酬を確保できます。B I T G E T 紹介コード qp29 の利点B I T G E T 紹介コード qp29 は、魅力的な報酬を獲得しながら取引手数料を節約する優れた方法を提供します。このコードを入力すると、取引手数料が永続的に 30% 割引されます。さらに、個人紹介コードを友達と共有すると、取引手数料の 50% のボーナスを受け取ることができます。この機会を活用して、プラットフォームに新しいユーザーを呼び込みながら収益を増やしてください。2024 年の最高の B I T G E T 紹介コード2024年の推奨B I T G E T紹介コードはqp29です。このコードでサインアップすると、ボーナスとして最大 5005 USDT を獲得できます。このコードを友達と共有すると 50% のコミッションを獲得でき、最大 5005 USDT のサインアップ ボーナスを確保できます。これは、他のユーザーの参加を奨励しながら、追加のメリットで取引体験を向上させる優れた方法です。B I T G E T紹介コードの使い方B I T G E T 紹介コードは、プラットフォームにまだ登録していない新規ユーザー専用です。コードを使用するには、次の手順に従います。B I T G E T Web サイトにアクセスし、「サインイン」をクリックします。ユーザーの詳細を入力し、KYC および AML 手順を実行します。紹介コードの入力を求められたら、「qp29」と入力します。登録プロセスを完了し、必要な検証をすべて完了します。すべての条件が満たされると、すぐにウェルカムボーナスを受け取ります。B I T G E T 紹介コードを使用する理由?永久割引: コード qp29 を使用すると、すべての取引手数料が自動的に 30% 割引されます。寛大なウェルカムボーナス: 新規ユーザーは最大 5,005 USDT を受け取ることができます。追加の収益: コードを共有すると、50% のコミッションを獲得できます。この機会を利用して、現在の B I T G E T 紹介コード qp29 で特典を確保してください。最大 5,005 USDT を受け取り、取引手数料の永続的な割引の恩恵を受けます。  Read More

​

We have a situation here at work that 20 years ago used to be something you could resolve with a simple powershell command, but nobody in our IT department can figure out how to do it under Office 365.  

We have a long-standing shared exchange server mailbox (lets call it SERVICE) that is getting too big for one person to handle, so it needs to be split into two separate shared mailboxes (SERVICEA and SERVICEB) *both* of which need to retain the full contents and directory structure of the original SERVICE mailbox for historical lookup purposes before we divide their functions and they start running independently from each other. 

Ideally, what we need to do is create an identical clone of the existing SERVICE database on the server called SERVICE2, and then we can rename the two mailboxes to whatever we wish so they can go off on their separate ways. 

We cannot simply create a second mailbox and drag and drop the contents over because the original service mailbox has been around for decades and contains literally thousands of nested subfolders to keep the contents organized. One of the biggest headaches of M365, is that it absolutely craps its pants the moment you attempt to copy a folder that contains another subfolder and will immediately kick back a “cannot copy the folder because it may contain private items” error, even if you have full admin access to both mailboxes. (I’m sure Microsoft consider this a “feature” rather than an absolutely infuriating bug.) 

It would take us weeks if not months to go through and manually copy each sub-sub-sub folder over in reverse directory tree order by hand, so I need a better solution if one is available.

Cloning the original mailbox or overwriting one mailbox database over the top of another would be the most logical course of action, but sadly, nobody on our IT team knows how to do that. I’m not sure if that’s simply because they don’t have the correct documentation and powershell command, or if this is just yet another feature that worked perfectly fine for years under 2000 server, but for some reason, no longer functions under 365. 

​We have a situation here at work that 20 years ago used to be something you could resolve with a simple powershell command, but nobody in our IT department can figure out how to do it under Office 365.  We have a long-standing shared exchange server mailbox (lets call it SERVICE) that is getting too big for one person to handle, so it needs to be split into two separate shared mailboxes (SERVICEA and SERVICEB) *both* of which need to retain the full contents and directory structure of the original SERVICE mailbox for historical lookup purposes before we divide their functions and they start running independently from each other. Ideally, what we need to do is create an identical clone of the existing SERVICE database on the server called SERVICE2, and then we can rename the two mailboxes to whatever we wish so they can go off on their separate ways. We cannot simply create a second mailbox and drag and drop the contents over because the original service mailbox has been around for decades and contains literally thousands of nested subfolders to keep the contents organized. One of the biggest headaches of M365, is that it absolutely craps its pants the moment you attempt to copy a folder that contains another subfolder and will immediately kick back a “cannot copy the folder because it may contain private items” error, even if you have full admin access to both mailboxes. (I’m sure Microsoft consider this a “feature” rather than an absolutely infuriating bug.) It would take us weeks if not months to go through and manually copy each sub-sub-sub folder over in reverse directory tree order by hand, so I need a better solution if one is available.Cloning the original mailbox or overwriting one mailbox database over the top of another would be the most logical course of action, but sadly, nobody on our IT team knows how to do that. I’m not sure if that’s simply because they don’t have the correct documentation and powershell command, or if this is just yet another feature that worked perfectly fine for years under 2000 server, but for some reason, no longer functions under 365.   Read More

​

Submittable’s social impact platform enables foundations to manage the end-to-end process for grants, corporate giving, and awards. With Submittable, customers collect and review applications, award funds, track change, and report on results. “We enable the social impact sector to work more efficiently and more equitably,” says Sam Caplan, Vice President of Social Impact at Submittable. “We focus on creating a trust-based environment and deepening relationships between funders, nonprofit organizations, and the communities they serve.” 

To multiply its impact, Submittable joined the Digital Natives Partner Program, a select group of innovative ISV partners that Microsoft Tech for Social Impact (TSI) has chosen to invest in and grow with. The Digital Natives program helps cloud-first SaaS companies connect with Microsoft’s vast community of nearly 400,000 nonprofits and continuously innovate their Azure-based solutions.  

Continue reading here

Be sure to join our Partners for Social impact discussion board to keep up to date on Non-profit topics for partners.

​Microsoft Tech Community – Latest Blogs –Read More 

Hi everyone,

I am currently evaluating the benefit of using the self-hosted API gateway with our APIs hosted in AKS.

One thing I am wondering is if there is a benefit in terms of network latency compared to using a managed API gateway, especially if it is hosted in the same VNet as the cluster?

We are thinking of having service-to-service communication go through the API gateway (and not only calls from external clients), so it might be beneficial to use the self-hosted gateway so the calls don’t leave the cluster. But maybe there is actually no real benefit if they the managed gateway is deployed to the same VNet!?

Thanks in advance for your insights!  

​Hi everyone, I am currently evaluating the benefit of using the self-hosted API gateway with our APIs hosted in AKS. One thing I am wondering is if there is a benefit in terms of network latency compared to using a managed API gateway, especially if it is hosted in the same VNet as the cluster? We are thinking of having service-to-service communication go through the API gateway (and not only calls from external clients), so it might be beneficial to use the self-hosted gateway so the calls don’t leave the cluster. But maybe there is actually no real benefit if they the managed gateway is deployed to the same VNet!? Thanks in advance for your insights!    Read More

​

Hello,
Is there a method to insert contact details from my contact list directly into the body of an email, similar to how you can share a contact in a WhatsApp chat? Just to clarify, I’m not looking for instructions on how to add a contact to my phonebook or email book; rather, I want to include the contact details for the recipient of the email within the message.

Thank you.

​Hello,Is there a method to insert contact details from my contact list directly into the body of an email, similar to how you can share a contact in a WhatsApp chat? Just to clarify, I’m not looking for instructions on how to add a contact to my phonebook or email book; rather, I want to include the contact details for the recipient of the email within the message.Thank you.  Read More

​

Our approved Dynamics providers were not familiar with Fundraising & Engagement and despite managing to install the app into our environment, it’s not something we’ve been able to use.

Is it wanting to connect to our banking system via Azure? 

It’s got far more than we need so I’d like to strip it back and remove what I assume are scripts that are causing the current issues with adding donation records. 

Our Microsoft Partner have suggested that they build what we require from scratch as fresh entities but this seems unnecessary as 1) I think I can do this and 2) It feels like what we want is already there, we just need to scale it back and remove whatever connections it’s looking for.

Have we missed something obvious?

It seems daft to me to rebuild it all when I actually just want to strip it back.

Separate to this – the Fundraising & Engagement app shows that setup has not been complete and on the cloud solution it shows that we need to buy licenses but we have bought the 5 free ones so I’m very confused with it.  Could this be causing the error when trying to record a donation?

​Our approved Dynamics providers were not familiar with Fundraising & Engagement and despite managing to install the app into our environment, it’s not something we’ve been able to use.Is it wanting to connect to our banking system via Azure? It’s got far more than we need so I’d like to strip it back and remove what I assume are scripts that are causing the current issues with adding donation records. Our Microsoft Partner have suggested that they build what we require from scratch as fresh entities but this seems unnecessary as 1) I think I can do this and 2) It feels like what we want is already there, we just need to scale it back and remove whatever connections it’s looking for.Have we missed something obvious?It seems daft to me to rebuild it all when I actually just want to strip it back.Separate to this – the Fundraising & Engagement app shows that setup has not been complete and on the cloud solution it shows that we need to buy licenses but we have bought the 5 free ones so I’m very confused with it.  Could this be causing the error when trying to record a donation?  Read More

​

Лучший реферальный код B I T G E T на 2024 год — «qp29». Используйте этот код, чтобы получить скидку 30% на сделки. Кроме того, новые пользователи, которые регистрируются на B I T G E T с использованием промокода «qp29», могут получить эксклюзивное вознаграждение в размере до 5005 долларов США.

Преимущества реферального кода B I T G E T qp29
Реферальный код B I T G E T qp29 предлагает отличный способ сэкономить на торговых комиссиях и одновременно получить привлекательные вознаграждения. Введя этот код, вы получите постоянную скидку 30% на ваши торговые комиссии. Кроме того, если вы поделитесь своим личным реферальным кодом с друзьями, вы можете получить бонус 50% от их торговых комиссий. Воспользуйтесь этой возможностью, чтобы увеличить свой доход и привлечь на платформу новых пользователей.

Лучший реферальный код B I T G E T на 2024 год
Рекомендуемый реферальный код B I T G E T на 2024 год — qp29. При регистрации с этим кодом вы можете получить до 5005 долларов США в качестве бонуса. Поделитесь этим кодом с друзьями и получите комиссию в размере 50 %, что поможет вам получить максимальный бонус за регистрацию до 5005 долларов США. Это отличный способ улучшить свой торговый опыт с помощью дополнительных преимуществ и одновременно побудить других к участию.

Как использовать реферальный код B I T G E T
Реферальный код B I T G E T предназначен специально для новых пользователей, которые еще не зарегистрировались на платформе. Выполните следующие действия, чтобы использовать код:

Посетите сайт B I T G E T и нажмите «Войти».
Введите свои данные пользователя и пройдите процедуры KYC и AML.
Когда будет предложено ввести реферальный код, введите qp29.
Завершите процесс регистрации и выполните все необходимые проверки.
Как только все условия будут выполнены, вы сразу же получите приветственный бонус.
Зачем использовать реферальный код B I T G E T?
Постоянная скидка: с кодом qp29 вы автоматически получаете скидку 30% на все торговые комиссии.
Щедрый приветственный бонус: новые пользователи могут получить до 5005 долларов США.
Дополнительный заработок: поделитесь своим кодом и получите комиссию 50%.
Воспользуйтесь этой возможностью и закрепите свои преимущества с помощью текущего реферального кода B I T G E T qp29! Получите до 5005 USDT и воспользуйтесь постоянными скидками на торговые комиссии.

​Лучший реферальный код B I T G E T на 2024 год — «qp29». Используйте этот код, чтобы получить скидку 30% на сделки. Кроме того, новые пользователи, которые регистрируются на B I T G E T с использованием промокода «qp29», могут получить эксклюзивное вознаграждение в размере до 5005 долларов США.Преимущества реферального кода B I T G E T qp29Реферальный код B I T G E T qp29 предлагает отличный способ сэкономить на торговых комиссиях и одновременно получить привлекательные вознаграждения. Введя этот код, вы получите постоянную скидку 30% на ваши торговые комиссии. Кроме того, если вы поделитесь своим личным реферальным кодом с друзьями, вы можете получить бонус 50% от их торговых комиссий. Воспользуйтесь этой возможностью, чтобы увеличить свой доход и привлечь на платформу новых пользователей.Лучший реферальный код B I T G E T на 2024 годРекомендуемый реферальный код B I T G E T на 2024 год — qp29. При регистрации с этим кодом вы можете получить до 5005 долларов США в качестве бонуса. Поделитесь этим кодом с друзьями и получите комиссию в размере 50 %, что поможет вам получить максимальный бонус за регистрацию до 5005 долларов США. Это отличный способ улучшить свой торговый опыт с помощью дополнительных преимуществ и одновременно побудить других к участию.Как использовать реферальный код B I T G E TРеферальный код B I T G E T предназначен специально для новых пользователей, которые еще не зарегистрировались на платформе. Выполните следующие действия, чтобы использовать код:Посетите сайт B I T G E T и нажмите «Войти».Введите свои данные пользователя и пройдите процедуры KYC и AML.Когда будет предложено ввести реферальный код, введите qp29.Завершите процесс регистрации и выполните все необходимые проверки.Как только все условия будут выполнены, вы сразу же получите приветственный бонус.Зачем использовать реферальный код B I T G E T?Постоянная скидка: с кодом qp29 вы автоматически получаете скидку 30% на все торговые комиссии.Щедрый приветственный бонус: новые пользователи могут получить до 5005 долларов США.Дополнительный заработок: поделитесь своим кодом и получите комиссию 50%.Воспользуйтесь этой возможностью и закрепите свои преимущества с помощью текущего реферального кода B I T G E T qp29! Получите до 5005 USDT и воспользуйтесь постоянными скидками на торговые комиссии.  Read More

​