Hi 

I am using Azure OpenAI with Azure Cognitive services as grounded data. And the LLM is GPT 3.5

My understanding of the flow is as follows.

 User sends the prompt questionLLM looks in to the grounded data (Azure cognitive service)Generates the response and return to user.

While validating the implementation, I found irrespective of the prompt question length, the number of prompt tokens it is generating looks to be on higher side. Now my question is, is the prompt token count generated is as expected? Is there a scope of improvement to reduce the number of tokens generated?

​Hi I am using Azure OpenAI with Azure Cognitive services as grounded data. And the LLM is GPT 3.5My understanding of the flow is as follows. User sends the prompt questionLLM looks in to the grounded data (Azure cognitive service)Generates the response and return to user.While validating the implementation, I found irrespective of the prompt question length, the number of prompt tokens it is generating looks to be on higher side. Now my question is, is the prompt token count generated is as expected? Is there a scope of improvement to reduce the number of tokens generated?     Read More

​