Transcribing an Audio File from a Storage Account Using the Speech Service

UseCase 1 – Azure Speech Service Outbound access not restricted

Prepare the Audio File: Upload the audio file to your storage account and note its URL. We can take sample file from here: cognitive-services-speech-sdk/sampledata/audiofiles at master · Azure-Samples/cognitive-services-speech-sdk · GitHub
Set Up the Speech Service: Obtain the API key and endpoint URL from your speech service in Azure as below reference

 3. Make the GET Request: Follow below steps in Postman to make a POST request to Speech Service

   Open Postman and create a new POST request.
   Set the URL to https://<SpeechServiceLocation>.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions.
   Add Headers:
Ocp-Apim-Subscription-Key:    <keyOfSpeechService>
Content-Type: application/json
   Set the Body to raw and select JSON format.
      Then, paste the following JSON:

{

  “contentUrls”: [

    “SASLinkToAudioFileOnStorage”

  ],

  “locale”: “en-US”,

  “displayName”: “My Transcription”,

  “model”: null,

  “properties”: {

    “wordLevelTimestampsEnabled”: true,

    “languageIdentification”: {

      “candidateLocales”: [

        “en-US”, “de-DE”, “es-ES”

      ]

    }

  }

}

The Post Request will return a status code of 201 as shown – it indicates that the request was successfully processed, and a new transcription job has been created. This status code confirms that the transcription process has been initiated

In the response body of the POST request, find the URL provided under the ‘Links’ section and make a GET request to that URL.

The response from this GET request will contain a contentUrl, which you need to use to make another request to fetch the transcribed data

Since the outbound access was not disabled, we were able to fetch the Transcribed data from speech service.

UseCase 2 – Azure Speech Service Outbound access is restricted

Repeat the steps as mentioned in UseCase1 to send POST request to Speech Service.
The request will return a status code of 403.This means that we are not allowed to access Audio File from Storage account because here we have mentioned –  restrictOutboundNetworkAccess”: true, and “allowedFqdnList”: “microsoft.com” which means we have restricted outbound access and speech service can only access “microsoft.com” only.

​Microsoft Tech Community – Latest Blogs –Read More 

This post is going to explain how to use Advanced RAG Service easily verify proper RAG tech performance for your own data, and integrate it as a service endpoint into Copilot Studio.

This time we use CSV as a sample. CSV is text structure data, when we use basic RAG to process a multiple pages CSV file as Vector Index and perform similarity search using Nature Language on it, the grounded data is always chunked and hardly make LLM to understand the whole data picture.

For example, if we have 10,000 rows in a CSV file, when we ask “how many rows does the data contain and what’s the mean value of the visits column”, usually general semantic search service cannot give exact right answers if it just handles the data as unstructured. We need to use different advanced RAG method to handle the CSV data here.

Thanks to LLamaIndex Pandas Query Engine, which provides a good idea of understanding data frame data through natural language way. However to verify its performance among others and integrate to existing Enterprise environment, such as Copilot Studio or other user facing services, it definitely needs AI service developing experience and takes certain learning curve and time efforts from POC to Production.

Advanced RAG Service supports 6 latest advanced indexing techs including CSV Query Eninge, with it developers can leverage it to shorten development POC stage, and achieve Production purpose. Here is detail step to step guideline:

a. In Docker environment, run this command to clone the dockerfile and related config sample:

b. In the AdvancedRAG folder, rename .env.sample to .env

c. In the .env file, configure necessary environment variables. In this tutorial, let’s configure:

NOTE:

d. Build your own docker image:

e. Run this docker:

f. Access http://localhost:8000/

a. Click the CSV Query Engine tab, upload a test CSV file, click Submit

b. Click the Chat Mode tab, now we can use Natural Language to test how good the CSV Query Engine at understanding CSV content:

The Advanced RAG Service is built with Gradio and FAST API. It opens necessary API Endpoints by default. We can turn off any of them in the .env settings.

The Chat endpoint can be used for different index types query/search. Since we are using “CSV Query Engine”, now it is:

The response is:

Using this method, we can easily integrate the specific RAG capability to our own service, such as Copilot Studio. Before that, let’s publish the service first.

We have different methods to release docker as an app service. Here are the generate steps when we use Azure Contain Registry and Azure Container App.

a. Create Azure Container Registry resource [ACRNAME], upload your tested docker image to it. The command is:

b. Create an Azure Container App, deploy this docker image, and deploy it. Don’t forget enable Session Affinity for the Container App.

To automate the Azure Container App deployment, I provided deploy_acr_app.sh in the repo.

To use it:

Note: for more details about this sh, can refer to this guideline.

After around 7~8 minutes, the Azure Container App will be ready. You can check the output and access it directly:

To protect your container app, can follow this guide to enable authentication on it.

Enable authentication and authorization in Azure Container Apps with Microsoft Entra ID

By default, we need to upload a CSV to the AdvRAG service before analysis. The service always saves the uploaded file to its local temp folder on server side. And then we can use temp file path to start the analysis query.

To skip this step, we can save common files in subfolder rules of the AdvancedRAG folder, and then build your docker image. The files will be copy to the docker itself. As a demo, I can put a CSV file in AdvancedRAG/rules/files, and then pubish the docker to Azure.

a. Open Copilot Studio, create a new Topic, use “CSV Query” to trigger it.

b. For demo purpose, I upload a test CSV file and got its path, then put it into a variable:

c. Now let’s add a Question step to ask what question the user want to ask:

d. Click “+”, “Add an Action”, “Create a flow”. We will use this new flow to call AdvancedRAG service endpoint.

e. We need Query, File_Path, System_Message as input variables.

e. In the flow Editor, let’s add an HTTP step. In the step, post the request to the AdvancedRAG endpoint as below:

Save the flow as ADVRAGSVC_CSV, and publish it.

f. Back to Copilot Studio topic, we will add the action as below, and set input variables as need:

g. Publish and open this Custom Copilot in Teams Channel based on this guide.

h. Now we can test this topic lit this, as we see, even I used gpt-4o-mini here, the response accuracy is very good:

From above, it shows how to quickly verify potential useful RAG techs (Pandas Query Engine) in the AdvancedRAG service studio, expose and publish it as REST API endpoint which can be used by other service, such as Copilot Studio.

The AdvancedRAG service focuses on key logic and stability of different important index types, the efficiency to be landed into M365 AI use cases. For any feature improvement ideas, feel free to visit below repos to create issues, fork projects and create PRs.

Docker Deploy Repo: https://github.com/freistli/AdvancedRAG

Source Code Repo: https://github.com/freistli/AdvancedRAG_SVC

Exploring the Advanced RAG (Retrieval Augmented Generation) Service

​Microsoft Tech Community – Latest Blogs –Read More 

Hi, 

My apologies as this is a very basic VLOOKUP question but I can’t figure it out. I have data where every person has a research partner. I’m trying to create a Partner_ID variable that contains the ID of the person the participant is partnered with. I’m pretty sure VLOOKUP is the best way to do this but it isn’t working for me. I have put an example with fake data in the screenshot below. Essentially, I want VLOOKUP to search for the name “Cox” in the Table and find the name under the LastName column, then print the ID that is next to it (in this case, cell D2 should print a 3, indicating that Perry’s Partner is ID#3). As you can see, however, Excel is returning error messages that it can’t find the expected value. I’ve looked at various tutorials and I can’t figure out what I’m doing wrong. Thank you! 

The formula I’m using is =VLOOKUP(C2,$A$2:$B$7,1,FALSE)

​Hi,  My apologies as this is a very basic VLOOKUP question but I can’t figure it out. I have data where every person has a research partner. I’m trying to create a Partner_ID variable that contains the ID of the person the participant is partnered with. I’m pretty sure VLOOKUP is the best way to do this but it isn’t working for me. I have put an example with fake data in the screenshot below. Essentially, I want VLOOKUP to search for the name “Cox” in the Table and find the name under the LastName column, then print the ID that is next to it (in this case, cell D2 should print a 3, indicating that Perry’s Partner is ID#3). As you can see, however, Excel is returning error messages that it can’t find the expected value. I’ve looked at various tutorials and I can’t figure out what I’m doing wrong. Thank you!  The formula I’m using is =VLOOKUP(C2,$A$2:$B$7,1,FALSE)   Read More

​