Latency in speech recognition and synthesis can be a significant hurdle in creating seamless and efficient applications. Reducing latency not only improves user experience but also enhances the overall performance of real-time applications. This blog post will explore strategies to reduce latency in general transcription, real-time transcription, file transcription, and speech synthesis.

1. Network Latency: Move the Speech Resource Closer to the App
One of the primary factors contributing to latency in speech recognition is network latency. To mitigate this, it’s essential to minimize the distance between your application and the speech recognition resource. Here are some tips:

Speech Containers : It gives the flexibility of running the model within on-premise or on edge thereby eliminating the need to send over the audio data over cloud and hence reducing network latency. Link – Install and run Speech containers with Docker – Speech service – Azure AI services | Microsoft Learn

Leverage Cloud Providers: Choose cloud service providers with data centers in regions that are closer to your users. This reduces the network latency significantly.
Optimize Network Routing: Ensure that your network routing is optimized to take the shortest path between your application and the speech recognition service.

2. Real-Time Transcription:

Real-time transcription requires immediate processing of audio input to provide instant feedback. Here are some recommendations to achieve low latency in real-time transcription:

2.1 Use Real-Time Streaming
Instead of recording the entire audio and then processing it, use real-time streaming to send audio data in small chunks to the speech recognition service. This allows for immediate processing and reduces the overall latency.

def speech_recognize_continuous_async_from_microphone():
“””performs continuous speech recognition asynchronously with input from microphone”””
speech_config = speechsdk.SpeechConfig(subscription=os.getenv(“SUBSCRIPTION_KEY”), region=”centralIndia”)
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)
done = False
def recognized_cb(evt: speechsdk.SpeechRecognitionEventArgs):
print(‘RECOGNIZED: {}’.format(evt.result.text))

def stop_cb(evt: speechsdk.SessionEventArgs):
“””callback that signals to stop continuous recognition”””
print(‘CLOSING on {}’.format(evt))
nonlocal done
done = True

# Connect callbacks to the events fired by the speech recognizer
speech_recognizer.recognized.connect(recognized_cb)
speech_recognizer.session_stopped.connect(stop_cb)

# Other tasks can be performed on this thread while recognition starts…
result_future = speech_recognizer.start_continuous_recognition_async()
result_future.get() # wait for voidfuture, so we know engine initialization is done.
print(‘Continuous Recognition is now running, say something.’)
while not done:
print(‘type “stop” then enter when done’)
stop = input()
if (stop.lower() == “stop”):
print(‘Stopping async recognition.’)
speech_recognizer.stop_continuous_recognition_async()
break
print(“recognition stopped, main thread can exit now.”)
speech_recognize_continuous_async_from_microphone()

Azure Speech SDK also provides a way to stream audio into the recognizer as an alternative to microphone or file input. You can choose between PushAudioInputStream and PullAudioInputStream depending upon your requirement. For details – Speech SDK audio input stream concepts – Azure AI services | Microsoft Learn

2.2 Define the Default Language

If the default language is known, define it at the start of the transcription process. This eliminates the additional processing time required to detect the input language. If the default language is not known, use the “SpeechServiceConnection_LanguageIdMode” to detect the language at the start of the transcription and specify the list of expected languages to reduce the processing time

speech_config = speechsdk.SpeechConfig(subscription=”YourSubscriptionKey”, region=”YourServiceRegion”)
speech_config.speech_recognition_language = “en-US” # Set default language
## OR
speech_config.set_property(property_id=speechsdk.PropertyId.SpeechServiceConnection_LanguageIdMode, value = “AtStart”)
auto_detect_source_language_config = speechsdk.languageconfig.AutoDetectSourceLanguageConfig(languages=[“en-US”, “gu-In”, “bn-IN”, “mr-IN”])
speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config,auto_detect_source_language_config=auto_detect_source_language_config)

2.3 Use Asynchronous Methods
Utilize asynchronous methods like start_continuous_recognition_async instead of start_continuous_recognition and stop_continuous_recognition_async instead of stop_continuous_recognition. These methods allow for non-blocking operations and reduce latency.

speech_recognizer.start_continuous_recognition_async(); # Perform other tasks

speech_recognizer.stop_continuous_recognition_async()

2.4 Use Fast Transcription

Fast Transcription transcribes audio significantly faster than real-time streaming transcription and is apt for scenarios where immediate transcript is essential like call center analytics, meeting summarization, voice dubbing and many others. It can transcribe 30 min audio in less than a minute. Albeit this is in public preview and supports only a handful of locales. For complete list of supported languages, check out Language support – Speech service – Azure AI services | Microsoft Learn

3. File Transcription
For file transcription, processing large audio files can introduce significant latency. Here are some strategies to reduce latency:

3.1 Split the Audio into Small Chunks
Divide the audio file into smaller chunks and run the transcription for each chunk in parallel. This allows for faster processing and reduces the overall transcription time. One caveat with audio chunking is it might result a small drop in transcription quality based on the chunking strategy but if the transcription layer is followed by an intelligence layer of LLM for analytical insights, post processing etc, the drop in quality should get offset by the superior LLM intelligence.

from pydub import AudioSegment
import concurrent.futures
def transcribe_chunk(chunk):
# Transcription logic for each chunk
pass
audio = AudioSegment.from_file(“large_audio_file.wav”)
chunk_length_ms = 10000 # 10 seconds
chunks = for i in range(0, len(audio), chunk_length_ms)]

with concurrent.futures.ThreadPoolExecutor() as executor:
futures = [executor.submit(transcribe_chunk, chunk) for chunk in chunks]
results = [f.result() for f in concurrent.futures.as_completed(futures)]

3.2 Increase the Speed of Audio
Increase the playback speed of the audio file before sending it for transcription. This reduces the time taken to process the entire file with negligible compromise on the accuracy of transcription.

def increase_audio_speed(filename, output_filename = “modified_audio_file.wav”, speed_change_factor = 1.7):
# Load your audio file
audio = AudioSegment.from_file(filename) # Change to your file format
# Change speed: Speed up (e.g., 1.5 times)
speed_change_factor = speed_change_factor # Increase this to make it faster, decrease to slow down
new_audio = audio._spawn(audio.raw_data, overrides={‘frame_rate’: int(audio.frame_rate * speed_change_factor)})
# Set the frame rate to the new audio
new_audio = new_audio.set_frame_rate(audio.frame_rate)
# Export the modified audio
new_audio.export(output_filename, format=”wav”) # Change to your desired format

3.3 Compress the Input Audio
Compress the input audio before sending it for transcription. This reduces the file size for faster transmission, optimizing bandwidth usage and storage efficiency in transcription.

from pydub import AudioSegment
input_audio = ‘gujrati_tts.wav’
output_audio = ‘compressed_audio.mp3’
try:
# Load the audio file
audio = AudioSegment.from_file(input_audio)
# Export the audio file with a lower bitrate to compress it
audio.export(output_audio, format=”mp3″, bitrate=”64k”)
print(f”Compressed audio saved as {output_audio}”)
except Exception as e:
print(f”An error occurred: {e}”)

4. Speech Synthesis
Latency in speech synthesis can be a bottleneck, especially in real-time applications. Here are some recommendations to reduce latency:

4.1 Use Asynchronous Methods
Instead of using speak_text_async for speech synthesis, which blocks the streaming until the entire audio is processed, switch to the start_speaking_text_async method. This method starts streaming the audio output as soon as the first audio chunk is received, reducing latency significantly.

4.2 Text Streaming : Streaming text allows the TTS system to start processing and generating speech as soon as the initial part of the text is received, rather than waiting for the entire text to be available. This reduces the initial delay before speech output begins making it ideal for interactive applications, live events, and responsive AI-driven dialogues

# tts sentence end mark
tts_sentence_end = [ “.”, “!”, “?”, “;”, “。”, “！”, “？”, “；”, “n” ]
completion = gpt_client.chat.completions.create(
model=”gpt-4o”,
messages=[,
{“role”: “user”, “content”: <prompt>}
],
stream=True
)
collected_messages = []
last_tts_request = None

for chunk in completion:
if len(chunk.choices) > 0:
chunk_text = chunk.choices[0].delta.content
if chunk_text:
collected_messages.append(chunk_text)
if chunk_text in tts_sentence_end:
text = “”.join(collected_messages).strip() # join the received message together to build a sentence
last_tts_request = speech_synthesizer.start_speaking_text_async(text).get()
collected_messages.clear()

4.3 Optimize Audio Output Format
The payload size impacts latency. Use a compressed audio format to save network bandwidth, which is crucial when the network is unstable or has limited bandwidth. Switching to the Riff48Khz16BitMonoPcm format, which has a 384 kbps bitrate, automatically uses a compressed output format for transcription, thereby reducing latency.

By following these strategies, you can significantly reduce the latency in STT and TTS applications, providing a smoother and more efficient user experience. Implementing these techniques will ensure that your applications are responsive and performant, even in real-time scenarios.

​Microsoft Tech Community – Latest Blogs –Read More 

Hello everyone,

I am repeatedly encountering an issue in our SharePoint libraries where files are locked for shared use, preventing any metadata from being edited. The files are not checked out; they were simply opened by the user and then closed again.

After that, the user is unable to make any changes to the metadata for about 1-2 hours, I estimate.

The same thing happens when I create a new Word document.

Create, edit, and save a Word document, then close Word. After that, the metadata of the file cannot be edited because it is locked for shared use by the same user.

Is there a solution for this?

Kind regards,

Steffen

​Hello everyone, I am repeatedly encountering an issue in our SharePoint libraries where files are locked for shared use, preventing any metadata from being edited. The files are not checked out; they were simply opened by the user and then closed again.After that, the user is unable to make any changes to the metadata for about 1-2 hours, I estimate. The same thing happens when I create a new Word document.Create, edit, and save a Word document, then close Word. After that, the metadata of the file cannot be edited because it is locked for shared use by the same user. Is there a solution for this? Kind regards,Steffen  Read More

​

Hi all,

I have somewhat strange issue with Outlook. An user had access to a shared mailbox via the delegation in Exchange. With auto-mapping the mailbox was visible/accessible in Outlook.

Now the delegation for the mailbox is removed, but the mailbox is still visible in Outlook. When clicking the mailbox, the user gets an error and the mailbox isn’t accessible (= correct behaviour). In OWA the mailbox is not accessible, as expected.

What could be a reason that the mailbox is shown in Outlook? And how can we remove the mailbox in Outlook? I tried to make a new Outlook profile and even a new Windows profile. But without success so far.

Kind regards,
Arjan

​Hi all,I have somewhat strange issue with Outlook. An user had access to a shared mailbox via the delegation in Exchange. With auto-mapping the mailbox was visible/accessible in Outlook.Now the delegation for the mailbox is removed, but the mailbox is still visible in Outlook. When clicking the mailbox, the user gets an error and the mailbox isn’t accessible (= correct behaviour). In OWA the mailbox is not accessible, as expected.What could be a reason that the mailbox is shown in Outlook? And how can we remove the mailbox in Outlook? I tried to make a new Outlook profile and even a new Windows profile. But without success so far.Kind regards,Arjan  Read More

​

Good day,

We are in the process of testing and evaluating using Exchange online & Records management to mark items as a records for an x amount of years so that users cannot delete or edit the items during the retention period. Afterwards the items will be removed.

We are now seeing the following:

Mails that are labeled with a record label restricts the user to edit or delete the mail with the following notification “Outlook cannot delete one or more items because of a retention policy”

However when a user navigates to Assign policy and select Use parent folder policy the record retention label gets removed and the user can indeed remove a mail that was actually not ment to be removed.

All of the default retention tags were removed from the Default MRM policy except move to Online Archive after 2 years. 

Is this by design? Or is there a way to disable or remove “user the parent folder policy” option?

​Good day,We are in the process of testing and evaluating using Exchange online & Records management to mark items as a records for an x amount of years so that users cannot delete or edit the items during the retention period. Afterwards the items will be removed.We are now seeing the following:Mails that are labeled with a record label restricts the user to edit or delete the mail with the following notification “Outlook cannot delete one or more items because of a retention policy”However when a user navigates to Assign policy and select Use parent folder policy the record retention label gets removed and the user can indeed remove a mail that was actually not ment to be removed. All of the default retention tags were removed from the Default MRM policy except move to Online Archive after 2 years.  Is this by design? Or is there a way to disable or remove “user the parent folder policy” option?  Read More

​

Hello Esteemed Community ! 

I’m looking for Pen test approach to prove co-pilot for M365 fits for financial institutes

in the area of Security, Compliance and Risk. 

Any assistance on this regards will be great help to prepare test plan. 

Thanks 

Muniraj

​Hello Esteemed Community !  I’m looking for Pen test approach to prove co-pilot for M365 fits for financial institutesin the area of Security, Compliance and Risk. Any assistance on this regards will be great help to prepare test plan.  Thanks Muniraj  Read More

​

Windows 11 (Version 21H2) 

Hello everyone, I need assistance with a recent issue I’ve encountered. Every time I try to right-click on the desktop and open either “Display” or “Personalize,” a Windows Script Host error message pops up (refer to the screenshot attached). I’m unsure how to resolve this problem. I would greatly appreciate any suggestions or guidance. Thank you.

​Windows 11 (Version 21H2) Hello everyone, I need assistance with a recent issue I’ve encountered. Every time I try to right-click on the desktop and open either “Display” or “Personalize,” a Windows Script Host error message pops up (refer to the screenshot attached). I’m unsure how to resolve this problem. I would greatly appreciate any suggestions or guidance. Thank you.  Read More

​

Check out the specifications of my computer setup:

————————

Operating System: Windows 11 Home Edition Version 23H2 (OS Build 22631.2861)

Boot Mode: UEFI with Secure Boot successfully enabled

Processor: Intel Core i3-8100 running at 3.60 Gigahertz

Cache:

– 256 kilobytes primary memory cache

– 1 megabyte secondary memory cache

– 6 megabytes tertiary memory cache

64-bit architecture, Multi-core (4 cores), Not hyper-threaded

Storage:

– 3.00 Terabytes Total Hard Drive Capacity

– 1.99 Terabytes Free Hard Drive Space

Optical Drive: Slimtype DVD A DS8ACSH

Hard Drives:

1. ST2000DM001-1CH164 (2.00 TB) — drive 1, Serial Number: S1E18PLS, Revision: CC24, SMART Status: Healthy

2. TOSHIBA DT01ACA100 (1.00 TB) — drive 0, Serial Number: 87J9ZDPFS, Revision: MS2OA750, SMART Status: Healthy

Bus Adapters:

– Intel(R) Chipset SATA/PCIe RST Premium Controller

– Microsoft Storage Spaces Controller

– Intel(R) USB 3.0 eXtensible Host Controller – 1.0 (Microsoft)

Virus Protection: PC Matic Super Shield

– Virus Definitions Version Up To Date

– Realtime File Scanning Off

Communications:

Bluetooth Device (Personal Area Network) #2:

– Status: Cable unplugged

– Physical Address: 9C:DA:3E:DE:E7:B5

Intel(R) Dual Band Wireless-AC 3165 #2:

– Auto IP Address: 192.168.1.111 / 24

– Gateway: 192.168.1.1

– Dhcp Server: 192.168.1.1

– Physical Address: 9C:DA:3E:DE:E7:B1

Microsoft Wi-Fi Direct Virtual Adapter #2:

– Status: Not connected to a network

– Physical Address: 9C:DA:3E:DE:E7:B2

Microsoft Wi-Fi Direct Virtual Adapter #3:

– Status: Not connected to a network

– Physical Address: 9E:DA:3E:DE:E7:B1

Realtek PCIe GbE Family Controller:

– Status: Cable unplugged

– Physical Address: F4:4D:30:FC:5C:51

Network DNS Server: 192.168.1.1

​Check out the specifications of my computer setup:————————Operating System: Windows 11 Home Edition Version 23H2 (OS Build 22631.2861) Boot Mode: UEFI with Secure Boot successfully enabled Processor: Intel Core i3-8100 running at 3.60 GigahertzCache:- 256 kilobytes primary memory cache- 1 megabyte secondary memory cache- 6 megabytes tertiary memory cache64-bit architecture, Multi-core (4 cores), Not hyper-threaded Storage:- 3.00 Terabytes Total Hard Drive Capacity- 1.99 Terabytes Free Hard Drive Space Optical Drive: Slimtype DVD A DS8ACSHHard Drives:1. ST2000DM001-1CH164 (2.00 TB) — drive 1, Serial Number: S1E18PLS, Revision: CC24, SMART Status: Healthy2. TOSHIBA DT01ACA100 (1.00 TB) — drive 0, Serial Number: 87J9ZDPFS, Revision: MS2OA750, SMART Status: Healthy Bus Adapters:- Intel(R) Chipset SATA/PCIe RST Premium Controller- Microsoft Storage Spaces Controller- Intel(R) USB 3.0 eXtensible Host Controller – 1.0 (Microsoft) Virus Protection: PC Matic Super Shield- Virus Definitions Version Up To Date- Realtime File Scanning Off Communications:Bluetooth Device (Personal Area Network) #2:- Status: Cable unplugged- Physical Address: 9C:DA:3E:DE:E7:B5Intel(R) Dual Band Wireless-AC 3165 #2:- Auto IP Address: 192.168.1.111 / 24- Gateway: 192.168.1.1- Dhcp Server: 192.168.1.1- Physical Address: 9C:DA:3E:DE:E7:B1Microsoft Wi-Fi Direct Virtual Adapter #2:- Status: Not connected to a network- Physical Address: 9C:DA:3E:DE:E7:B2Microsoft Wi-Fi Direct Virtual Adapter #3:- Status: Not connected to a network- Physical Address: 9E:DA:3E:DE:E7:B1Realtek PCIe GbE Family Controller:- Status: Cable unplugged- Physical Address: F4:4D:30:FC:5C:51 Network DNS Server: 192.168.1.1  Read More

​

Our organization is currently utilizing Lenovo laptops running Windows 11 22H2 Enterprise edition. For the past five years, we have implemented a Group Policy Object (GPO) to manage our screensaver settings. However, we have recently observed a brief ‘locking’ screen appearing before the screensaver activates. We are seeking assistance to bypass the locking screen and have the screensaver start directly. Has anyone encountered and resolved this issue before?

​Our organization is currently utilizing Lenovo laptops running Windows 11 22H2 Enterprise edition. For the past five years, we have implemented a Group Policy Object (GPO) to manage our screensaver settings. However, we have recently observed a brief ‘locking’ screen appearing before the screensaver activates. We are seeking assistance to bypass the locking screen and have the screensaver start directly. Has anyone encountered and resolved this issue before?  Read More

​

Hello, I’ve observed that the layout.ini file is no longer present in the windowsprefetch directory and it is not being generated even when using the “Rundll32.exe advapi32.dll,ProcessIdleTasks” command. This change might be due to relocating the collection folders from C:/ SSD to D:/HDD. Thank you.

​Hello, I’ve observed that the layout.ini file is no longer present in the windowsprefetch directory and it is not being generated even when using the “Rundll32.exe advapi32.dll,ProcessIdleTasks” command. This change might be due to relocating the collection folders from C:/ SSD to D:/HDD. Thank you.  Read More

​

Hi all,

I am an ex-sql server dba now getting my hands dirty again and building a new system. I have been using azure sql which is working absolutely fine for the small datasets however I will be importing some timeseries data – I’ve imported around 77 million records a month. Azure SQL seems to be ok when appropriately indexed but when I imported the data into a Azure Synapse Dedicated SQL Pool it seemed much faster out of the box. This is a relatively expensive solution though and Big Query probably offers a similar experience for cheaper (as claimed by google). 

I haven’t seen much information online around Azure Synapse Dedicated SQL Pools so I’m wondering if that is an area microsoft is pushing in the future or if there a better alternative within Azure.

I want to continue to query the data via sql which i can with Azure Synapse Dedicated SQL Pools and also Big Query,

Any advice please?

​Hi all, I am an ex-sql server dba now getting my hands dirty again and building a new system. I have been using azure sql which is working absolutely fine for the small datasets however I will be importing some timeseries data – I’ve imported around 77 million records a month. Azure SQL seems to be ok when appropriately indexed but when I imported the data into a Azure Synapse Dedicated SQL Pool it seemed much faster out of the box. This is a relatively expensive solution though and Big Query probably offers a similar experience for cheaper (as claimed by google). I haven’t seen much information online around Azure Synapse Dedicated SQL Pools so I’m wondering if that is an area microsoft is pushing in the future or if there a better alternative within Azure.I want to continue to query the data via sql which i can with Azure Synapse Dedicated SQL Pools and also Big Query,Any advice please?  Read More

​