DemoType is a ZoomIt feature that allows you to synthesize keystrokes from a script. Queue code blocks or Copilot prompts and send them to target windows during a live demo. Additionally, ZoomIt counteracts editor specific auto formatting, allowing a script to be interchangeable between target windows. Watch a video overview here.

Standard mode

Default behavior immediately begins injecting keystrokes to the target window upon pressing the DemoType hotkey (e.g. Ctrl + 7). No user input is required. ZoomIt will simply run to the end of the current text segment and exit, returning control to the user.

User-driven mode

You can select the option to drive input with typing. Toggle this behavior in the ZoomIt options dialog. One user key press will trigger one output character in 1:1 injection ratio.

You can adjust the injection ratio between 1:1, 1:2, and 1:3 with the speed slider in the ZoomIt options dialog. Upon reaching the end of the current text segment in user-driven mode, DemoType will continuie blocking keyboard input until you press space bar.

Input Script

Your script can be sourced from a file or from the clipboard. To use the clipboard, you must put the control keyword [start] at the beginning of your selection. This deliberate safety prefix is meant to stop you from unintentionally presenting sensitive data in the clipboard.

To use a file, select it from the ZoomIt options dialog. If you were previously sourcing input from the clipboard and would like to switch to file, set the clipboard to some text that doesn’t include the [start] prefix, or clear the clipboard via Windows Settings > System > Clipboard > Clear clipboard data.

The [end] control keyword is used to split your script into text segments. It is important to note that DemoType will look to the left and right of an [end] and absorb a single newline from each side if present. This allows you to format your script and pad an [end] with newlines that won’t render.

Cancelling DemoType

To cancel an active session, press escape. DemoType will also quit if focus is changed to a different window. Terminating a DemoType session mid text segment will hop to the next text segment. To hop back to the previous text segment, enter the DemoType hotkey with the Shift key in the opposite mode (e.g. Ctrl + Shift + 7).

Control Keywords

Use the following keywords throughout your script to control behavior.

[start] is a safety prefix only used when tagging clipboard data as a viable script
[end] is a delimiter to segment your script into snippets
[enter], [up], [down], [left], [right] synthesizes keystrokes
[pause:n] synthesizes a pause of n seconds
[paste] with a closing [/paste] allows you to inject a chunk of text via the clipboard

​Microsoft Tech Community – Latest Blogs –Read More 

Ever wanted to learn more about what goes into making OneDrive? Then this is the podcast for you! Join Stephen Rice and Arvind Mishra as they talk with CVP for OneDrive Engineering, Steven Bailey and engineering Manager John Selbie! This month, we’re talking about how OneDrive became the product that it is today, how engineering itself has evolved at Microsoft, and how the OneDrive engineering team strives to deliver a product that surpasses your expectations!

​Microsoft Tech Community – Latest Blogs –Read More 

The journey toward IPv6 only networking is challenging, and today there are several different approaches to the transition from IPv4 to IPv6 with multiple dual-stack or tunneling stages along the way. As we prioritize future Windows work, we would like to know more about what customers like you are using to support your own IPv6 deployments. We have published the survey below to ask you a few questions that will contribute to that exercise.

Link to the survey

The survey is fairly short and anonymous (though we left a field for sharing your contact information if you would be ok with direct follow up). Thank you in advance for your responses; your experiences will help us focus on what you find most valuable in our future work.

​Microsoft Tech Community – Latest Blogs –Read More 

As the influence of GenAI-based applications continues to expand, the critical need to enhance their performance becomes ever more apparent. In the realm of production applications, responses are expected within a range of milliseconds to seconds. The integration of Large Language Models (LLMs) has the potential to extend response times of such applications to few more seconds. This blog intricately explores diverse strategies aimed at optimizing response times in applications that harness Large Language Models on the Azure platform. In broad context, the subsequent methodologies can be employed to optimize the responsiveness of Generative Artificial Intelligence (GenAI) applications:

Response Optimization of LLM models
Designing an Efficient Workflow orchestration
Improving Latency in Ancillary AI Services

Response Optimization of LLM models

The inherent complexity and size of Language Model (LLM) architectures contribute substantially to the latency observed in any application upon their integration. Therefore, prioritizing the optimization of LLM responsiveness becomes imperative. Let’s now explore various strategies aimed at enhancing the responsiveness of LLM applications, placing particular emphasis on the optimization of the Large Language Model itself.

Key factors influencing the latency of LLMs are

Prompt Size and Output token count

A token is a unit of text that the model processes. It can be as short as one character or as long as one word, depending on the model’s architecture. For example, in the sentence “ChatGPT is amazing,” there are five tokens: [“Chat”, “G”, “PT”, ” is”, ” amazing”]. Each word or sub-word is considered a token, and the model analyses and generates text based on these units. A helpful rule of thumb is that one token corresponds to ~4 characters of text for common English text. This translates to ¾ of a word (so 100 tokens ~= 75 words).

A deployment of the GPT-3.5-turbo instance on Azure comes with a rate limit of around 120,000 tokens per minute, equivalent to approximately 2,000 tokens per second ( Details of TPM limits of each Azure OpenAI model are given here . It is evident that the quantity of output tokens has a direct impact on the response of Large Language Models (LLMs), consequently influencing the application’s responsiveness. To Optimize application response times, it is recommended to minimize the number of output tokens generated. Set an appropriate value for the max_tokens parameter to limit the response length. This can help in controlling the length of the generated output.

The latency of LLMs is influenced not only by the output tokens but also by the input prompts. Input prompts can be categorized into two main types:

Instructions, which serve as guidelines for LLMs to follow, and
Information, providing a summary or context for the grounded data to be processed by LLMs.

While instructions are typically of standard lengths and crucial for prompt construction, but the inclusion of multiple tasks may lead to varied instructions, and ultimately increasing the overall prompt size. It is advisable to limit prompts to a maximum of one or two tasks to manage prompt size effectively. Additionally, the information or content can be condensed or summarized to optimize the overall prompt length.

Model size

The size of the LLMs is typically measured in terms of its parameters. A simple neural network with just one hidden layer has a parameter for each connection between nodes (neurons) across layers and for each node’s bias. The more layers and nodes a model have, the more parameters it will contain. A larger parameter size usually translates into a more complex model that can capture intricate patterns in the data.

Applications frequently utilize Large Language Models (LLMs) for various tasks such as classification, keyword extraction, reasoning, and summarization. It is crucial to choose the appropriate model for the specific task at hand. Smaller models like Davinci are well-suited for tasks like classification or key value extraction, offering enhanced accuracy and speed compared to larger models. On the other hand, large models are more suitable for complex use cases like summarization, reasoning and chat conversations. Selecting the right model tailored to the task optimizes both efficiency and performance.

Leverage Azure-hosted LLM Models

Azure AI Studio provides customers with cutting-edge language models like OpenAI’s GPT-4, GPT-3, Codex, DALL-E, and Whisper models, and other open-source models all backed by the security, scalability, and enterprise assurances of Microsoft Azure. The OpenAI models are co-developed by Azure OpenAI and OpenAI, ensuring seamless compatibility and a smooth transition between the different models.

By opting for Azure OpenAI, customers not only benefit from the security features inherent to Microsoft Azure but also run on the same models employed by OpenAI. This service offers additional advantages such as private networking, regional availability, scalability, and responsible AI content filtering, enhancing the overall experience and reliability of language AI applications.

If anyone is using GenAI models from creators, the transition to Azure-hosted version of these Models has yielded notable enhancements in the response time of the models. This shift to Azure infrastructure has led to improved efficiency and performance, resulting in more responsive and timely outputs from the models.

Rate Limiting, Batching, Parallelize API calls

Large language models are subject to rate limits, such as RPM (requests per minute) and TPM (tokens per minute), which depend on the chosen model and platform. It is important to recognize that rate limiting can introduce latency into the application. To accommodate high traffic requirements, it is recommended to select the maximum value for the max_token parameter to prevent any occurrence of a 429 error, which can lead to subsequent latency issues. Additionally, it is advisable to implement retry logic in your application to further enhance its resilience.

Effectively managing the balance between RPM and TPM allows for enhanced latency through strategies like batching or parallelizing API calls.

When you find yourself reaching the upper limit of RPM but remain comfortably within TPM bounds, consolidating multiple requests into a single batch can optimize your response times. This batching approach enables more efficient utilization of the model’s token capacity without violating rate limits.

Moreover, if your application involves multiple calls to the LLMs API, you can achieve a notable speed boost by adopting an asynchronous programming approach that allows requests to be made in parallel. This concurrent execution minimizes idle time, enhancing overall responsiveness and making the most of available resources.

If the parameters are already optimized and the application requires additional support for higher traffic and a more scalable approach, consider implementing a load balancing solution through Azure API Management layer.

Stream output and use stop sequence

Every LLM endpoint has a particular throughput capacity. As discussed, earlier GPT-3.5-turbo instance on Azure comes with a rate limit of 120,000 tokens per minute, equivalent to approximately 2 tokens per milliseconds. So, to get an output paragraph with 2000 tokens it takes 1 second and the time taken to get the output response increases as the number of tokens increase. The time taken for the output response (Latency) can be measured as the sum of time taken for the first token generation and the time taken per token from the first token onwards. That is

Latency = (Time to first token + (Time taken per token * Total tokens))

So, to improve latency we can stream the output as every token gets generated instead of waiting for the entire paragraph to finish. Both the completions and chat Azure OpenAI APIs support a stream parameter, which when set to true, streams the response back from the model via Server Sevent Events (SSE). We can use Azure Functions with FastAPI to stream the output of OpenAI models in Azure as shown in the blog here.

Designing an Efficient Workflow orchestration

Incorporating GenAI solutions into applications requires the utilization of specialized frameworks like LangChain or Semantic Kernel. These frameworks play a crucial role in orchestrating multiple Large Language Model (LLM) based tasks and grounding these models on custom datasets. However, it’s essential to address the latency introduced by these frameworks in the overall application response. To minimize the impact on application latency, a strategic approach is imperative. A highly effective strategy involves optimizing LLM usage through workflow consolidation, either by minimizing the frequency of calls to LLM APIs or simplifying the overall workflow steps. By streamlining the process, you not only enhance the overall efficiency but also ensure a smoother user experience.

For example, when the requirement is to identify the intention of the user query and based on its context get response by grounding on data from multiple sources. Most times such requirements are executed as a 3-step process –

the first step is to identify the intent using a LLM
and the next step is to get the prompt content from the knowledge base relevant to the intent
and then with the prompt content get the output derived from the LLMs.

One simple approach could be to leverage data engineering and building a consolidated knowledge base with data from all sources and using the input user text directly as the prompt to the grounded data in knowledge base to get the final LLM response in almost a single step.

Improving Latency in Ancillary AI Services

The supporting AI services like Vector DB, Azure AI Search, data pipelines, and others that complement a Language Model (LLM)-based application within the overall Retrieval-Augmented Generation (RAG) pattern are often referred to as “ancillary AI services.” These services play a crucial role in enhancing different aspects of the application, such as data ingestion, searching, and processing, to create a comprehensive and efficient AI ecosystem. For instance, in scenarios where data ingestion plays a substantial role, optimizing the ingestion process becomes paramount to minimize latency in the application.

Similarly lets look at the improvement of few other such services – 

Azure AI search

Here are some tips for better performance in Azure AI Search:

Index size and schema: Queries run faster on smaller indexes. One best practice is to periodically revisit index composition, both schema and documents, to look for content reduction opportunities. Schema complexity can also adversely affect indexing and query performance. Excessive field attribution builds in limitations and processing requirements.

Query design: Query composition and complexity are one of the most important factors for performance, and query optimization can drastically improve performance.

Service capacity: A service is overburdened when queries take too long or when the service starts dropping requests. To avoid this, you can increase capacity by adding replicas or upgrading the service tier.

For more information on the optimizations of Azure AI Search index please refer here .

For optimizing third-party vector databases, consider exploring techniques such as vector indexing, Approximate Nearest Neighbor (ANN) search (instead of KNN), optimizing data distribution, implementing parallel processing, and incorporating load balancing strategies. These approaches enhance scalability and improve overall performance significantly.

Conclusion

In conclusion, these strategies contribute significantly to mitigating latency and enhancing response in large language models. However, given the inherent complexity of these models, the optimal response time can fluctuate between milliseconds and 3-4 seconds. It is crucial to recognize that comparing the response expectations of large language models to those of traditional applications, which typically operate in milliseconds, may not be entirely equitable.

​Microsoft Tech Community – Latest Blogs –Read More 

By: Chris Kunze – Sr. Product Manager | Microsoft Intune

If you’re managing a lot of devices, you know how important it is to keep your Microsoft Entra ID dynamic group processing running smoothly and efficiently. To encourage performant dynamic group rules, the ‘contains’ and ‘not Contains’ operators were recently removed (MC705357) from the rule builder’s list of operators. While it’s still possible to use these operators if you edit the rule syntax manually, there is a reason why these operators were removed. Certain properties and operators, such as ‘contains’ and ‘match’, are significantly less efficient in group processing than others. This inefficiency can lead to significant delays in dynamic group processing. You can optimize these rules by using more performant alternatives such as ‘Equals’, ‘Not Equals’, ‘Starts With’, and ‘Not Starts With’.

In addition, some device properties that are available in the creation of a dynamic group and not indexed which also leads to inefficiencies in the processing of the group membership. It’s best to avoid using these properties until they are indexed, if possible. The deviceOwnership and enrollmentProfileName properties have recently been indexed and work is ongoing to index the following properties to improve dynamic group processing efficiency:

deviceCategory
deviceManagementAppId
deviceManufacturer
deviceModel
deviceOSType
deviceOSVersion
devicePhysicalIds
deviceTrustType
isRooted
managementType
objectId
profileType
systemLabels

Using this guidance, we saw significant improvement in group membership evaluation times in a large customer’s production environment.

Here’s a quick example. An organization wants to group all devices that were enrolled with any of these 3 enrollment profiles:

iOS devices – Teachers
iOS devices – Students
iOS devices – Admins

While “device.enrollmentProfileName -contains “iOS devices” works, the rule “device.enrollmentProfileName -startswith “iOS devices” yields the same results but is a much more efficient query.

Evaluating your dynamic group rules with PowerShell

The following is a sample script that you can use to output the displayName, id, and membershipRule for each of the dynamic groups in your organization to a CSV-based file. Using this output, you can quickly list and evaluate the membership rules for all of your Entra ID dynamic groups for inefficiencies and start improving them.

$csvPath = “C:temp”
$csvFile = “dynGroups.csv”

if (!(Get-InstalledModule Microsoft.Graph -ErrorAction SilentlyContinue)) {
Write-Host “You need to install the Microsft.Graph module to run this script.” -ForegroundColor Red
Write-Host “Run ‘Install-Module Microsoft.Graph -Scope CurrentUser’ as an administrator” -ForegroundColor Red
exit 1
}

if (!(Get-MgContext -ErrorAction SilentlyContinue)) {
Connect-MgGraph -Scopes “Directory.Read.All,Group.Read.All”
}

$results = Invoke-MgGraphRequest -Method GET -Uri “https://graph.microsoft.com/v1.0/groups?`$filter=groupTypes/any(c:c+eq+’dynamicMembership’)”
$dynamicGroups = ($results).value
do {
if ($results.’@odata.nextlink’) {
$results = Invoke-MgGraphRequest -Method GET -Uri $results.’@odata.nextlink’
$dynamicGroups += ($results).value
}
} while (
$results.’@odata.nextlink’
)

$dynamicGroups | Select-Object displayName,id,membershipRule | Export-Csv -Path $csvPath$csvFile

Conclusion

We recommend evaluating your group membership rules to see how you can write them more efficiently. Use ‘Equals’ and ‘Starts With’ wherever possible and avoid using the non-indexed properties listed above if they don’t materially change the membership of the dynamic group. You can learn more about creating efficient rules by reading this documentation: Create simpler, more efficient rules for dynamic groups in Microsoft Entra ID.

We hope this helps to improve the processing of your dynamic group memberships! If you have any questions, leave a comment below or reach out to us on X @IntuneSuppTeam.

​Microsoft Tech Community – Latest Blogs –Read More 

Using [CmdletBinding(SupportShouldProcess)] on functions using them inside the main script does work as expected: When passing -WhatIf or -Confirm:$true to the script then those parameters are passed to the advanced function too.

However when moving those function into a module (Example: Tools.psm1) the function is imported and can be called but those parameters are NOT passed to that function.

Any ideas are welcome.

Kind Regards,

Thomas

​Using [CmdletBinding(SupportShouldProcess)] on functions using them inside the main script does work as expected: When passing -WhatIf or -Confirm:$true to the script then those parameters are passed to the advanced function too. However when moving those function into a module (Example: Tools.psm1) the function is imported and can be called but those parameters are NOT passed to that function. Any ideas are welcome. Kind Regards,Thomas  Read More

​

[Special Thanks to Adam Rudell, our Sr. Support Escalation Engineer, for putting together an excellent video and tutorial]

Hello SDN Community! 

Today, when you deploy Software Defined Networking via SDNExpress or Windows Admin Center (WAC), you must provide a REST DNS name.  This is often referred to as a Dynamic DNS deployment.    The northbound API endpoint (REST DNS name) is used for all of our clients and management experience. For example, if the replica moves from one Network Controller (NC) VM to another, the API service will perform a DNS registration to update the northbound API record.  In some instances, additional security hardening or third-party integration could cause issues with the DNS registration.  

Introducing Static IP configuration for Network Controller REST! 

With a Static IP configuration, when a replica moves to another NC VM, the API service simply programs a secondary IP with no DNS registration call.  This provides fault tolerance and relaxes the DNS registration requirements of SDN. 

Static IP can be used with a new SDN deployment OR existing SDN deployments! Check out the video that Adam Rudell, our Sr. Support Escalation Engineer, for an in-depth walk through of configuring this on a new SDN deployment, an existing SDN deployment, and then even moving back to a Dynamic DNS configuration!  Also, ICYMI, we launched a new YouTube channel focused entirely on Microsoft SDN across our Edge portfolio – be sure to Like, Subscribe, and Follow as we continue to post more content!

​Microsoft Tech Community – Latest Blogs –Read More 

If you are used to traditional licensing options and pay for software assurance, you may wonder why you would ever use pay-as-you-go billing. In this episode of Data Exposed with Anna Hoffman and Sasha Nosov, we’ll cover how to use our new PayG model and understand how you can use Extended Security Updates.

Watch on Data Exposed

Resources:

SQL Server enabled by Azure Arc – SQL Server | Microsoft Learn

View/share our latest episodes on Microsoft Learn and YouTube!

​Microsoft Tech Community – Latest Blogs –Read More 

Azure Virtual Desktop for Azure Stack HCI—now in general availability—extends the capabilities of the Microsoft Cloud to your datacenters.

IT pros face a complex and challenging environment as they help their organizations move to the cloud, especially when the cloud isn’t the best option for every workload. Managing hybrid cloud migrations while meeting the needs of today’s distributed workforce takes a comprehensive approach that balances performance and accessibility with security and control. For organizations that need desktop virtualization for applications that must remain on-premises for performance, data locality, or regulatory reasons, Azure Virtual Desktop for Azure Stack HCI may be the right solution.

Organizations like Commvault, a global leader in data management, also echo these benefits:

At Commvault, we have a unique environment where thousands of employees need hyper-local acces to data with the lowest latency possible. Azure Virtual Desktop for Azure Stack HCI checked all the boxes and met our needs with its ease of deployment and management, network and storage performance, and its security integrations allowing for governed access policies. 
– Ernie Costa, Business Cloud Operations Manager, Commvault  

Azure Virtual Desktop and Azure Stack HCI each deliver their own value, and with Azure Virtual Desktop for Azure Stack HCI, organizations can experience both. Read on to learn more about each solution and how Azure Virtual Desktop for Azure Stack HCI brings both together.

What is Azure Virtual Desktop?

Azure Virtual Desktop is a cloud VDI solution designed to meet the challenges of remote and hybrid work​. It enables a secure, remote desktop experience from anywhere, providing employees the best virtualized experience with the only solution fully optimized for Windows 11 and Windows 10 multi-session capabilities. It has built-in security to help keep your organization’s applications and data secure and compliant. Azure Virtual Desktop can simplify deployment and management of your infrastructure, gives you full control over configuration and management, and reduces costs by optimizing through existing virtualization investments and skills, as well as consumption-based pricing where you only pay for what you use.

What is Azure Stack HCI?

Azure Stack HCI is a Microsoft infrastructure solution powered by hyperconverged infrastructure (HCI) that hosts Windows and Linux virtual machines (VMs) or containerized workloads and their storage. It’s a hybrid product by design that connects on-premises systems to Azure for cloud-based services, monitoring, and management. Azure Stack HCI gives organizations the agility and cost-effectiveness of public cloud infrastructure while meeting the use case and regulatory requirements for specialized workloads that can’t live in the public cloud.  

The new feature release of Azure Stack HCI is now in general availability. It brings cloud-based cluster deployment and management together with Azure Arc infrastructure, providing centralized management of workloads. Learn more about Azure Stack HCI.

What is Azure Virtual Desktop for Azure Stack HCI?

Bringing the benefits of Azure Virtual Desktop and Azure Stack HCI together, Azure Virtual Desktop for Azure Stack HCI lets organizations securely run virtualized desktops and apps on-premises at the edge or in their datacenter. For organizations with data residency requirements, latency-sensitive workloads, or those with data proximity requirements. Azure Virtual Desktop for Azure Stack HCI extends the capabilities of the Microsoft Cloud to your datacenters.

With Azure Virtual Desktop for Azure Stack HCI, you can:

Improve performance for Azure Virtual Desktop users in areas with poor connectivity to the Azure public cloud by giving them session hosts closer to their location.
Meet data locality requirements by keeping app and user data on-premises. For more information, see Data locations for Azure Virtual Desktop.
Improve access to legacy on-premises apps and data sources by keeping desktops and apps in the same location.
Deliver the full Windows experience while retaining cost efficiency with Windows 11 and Windows 10 Enterprise multi-session. 
Unify your VDI deployment and management compared to traditional on-premises VDI solutions by using the Azure portal.
Achieve the best performance by using RDP Shortpath for low-latency user access.
Deploy the latest fully patched images quickly and easily using Azure Marketplace images.

How to get started

Azure Virtual Desktop for Azure Stack HCI is generally available with the new feature release of Azure Stack HCI. After you deploy an Azure Stack HCI cluster, it will be available as a resource location for Azure Virtual Desktop host pools.

To run Azure Virtual Desktop for Azure Stack HCI, you first need to make sure you’re licensed correctly and understand the pricing model. There are three components that affect how much it costs to run Azure Virtual Desktop for Azure Stack HCI:

User access rights. The same licenses that grant access to Azure Virtual Desktop on Azure also apply to Azure Virtual Desktop for Azure Stack HCI. Learn more at Azure Virtual Desktop pricing. Note, per-user access pricing for external users is not supported on Azure Virtual Desktop for Azure Stack HCI.
Infrastructure costs. Learn more at Azure Stack HCI pricing.
Hybrid service fee. This fee requires you to pay for each active virtual CPU (vCPU) for your Azure Virtual Desktop session hosts running on Azure Stack HCI. This fee is active once the preview period ends.

Want to learn more?

Watch the Microsoft Mechanics episode to see a demo of Azure Virtual Desktop for Azure Stack HCI.

Read the Azure Stack HCI blog to learn more about its general availability details.

Your feedback makes our features better

For customers who have already experienced Azure Virtual Desktop for Azure Stack HCI in our preview phase, your feedback has been invaluable, and we thank you. We continue to integrate your requests and ideas into future releases, and your feedback helps our development teams prioritize new features, address missed opportunities, and ensure the product better meets your needs.

​Microsoft Tech Community – Latest Blogs –Read More 

​Microsoft Tech Community – Latest Blogs –Read More