The new retention banners added in web outlook are horribly abrasive and distracting, your eyes keep going to them and not the content of the message. We can’t find a way to disable these unless we change the polocy. Curious if there is a way to, or hopefully MS changes this graphic. 

​The new retention banners added in web outlook are horribly abrasive and distracting, your eyes keep going to them and not the content of the message. We can’t find a way to disable these unless we change the polocy. Curious if there is a way to, or hopefully MS changes this graphic.   Read More

​

Earlier posts in this series:

Part 1: Is my Chatbot Ready for Production? – A 10,000 foot overview to LLMOps

Part 2: How do I Evaluate my LLM Chatbot? – A guide to different LLM chatbot evaluation techniques and how to implement them

Part 3: How do I Monitor my LLM Chatbot? –  A guide to monitoring LLM chatbot security and performance

Feedback is a critical, often overlooked, component of the LLMOps lifecycle. User feedback is a great way to gauge application success and guide new features or enhancements on the roadmap. Although feedback can be captured in many ways, the ultimate goal is application improvement.

In the age of LLMs, feedback is more important than ever. The probabilistic nature of LLMs is a paradigm shift away from standard ML models, where results are deterministic. Although there are robust evaluation methods, there is still no substitute for human analysis. Once an application is live, you can “crowdsource” this human analysis by soliciting feedback – both positive and negative – from users.

Types of feedback can be classified into two categories:

Direct Feedback: Soliciting feedback directly from users
Indirect Feedback: Analysis of user behavior and interactions

Both are valuable and provide different flavors of insight into your application. Together, they provide a complete picture of application success.

Direct Feedback

Direct user feedback is the most straightforward method, but it has some limitations and biases. Methods of getting direct feedback from a user include surveys, reviews, in-application prompts/pop-ups, or support tickets, among others. 

Sample Direct Feedback High Level Architecture 

With all these methods of direct feedback intake, we are left with a lot of messy and unstructured text data… a perfect problem for an LLM to solve. After aggregating all user feedback, you can use an LLM to summarize and categorize feedback so that the team quickly knows which application area(s) to focus on. 

Keep in mind, direct feedback sometimes comes with challenges like selection bias and small sample sizes. Often, the most likely users to reply to a survey or write a review are users that are not satisfied – which skews feedback in a negative direction. When users are satisfied, they are less likely to reply to provide feedback – which limits sample size.

Indirect Feedback

Indirect feedback involves analyzing user behavior and application interaction to infer how users feel about the application. This flavor is less straightforward than direct feedback and takes more investment to implement, but it is not as susceptible to some of the biases or skews in the resulting data.

Remember our data collection process from our monitoring system in part 3? We can use collected production data and the same evaluation framework from part 2 to assess how well our app is actually performing in Production. If scores are different in Production than they were during testing the test dataset should be adjusted to better match actual user behavior. 

A key metric to infer response effectiveness based on user behavior is edit distance between user prompts, which can indicate how much users are tweaking their inputs to get desired responses. The smaller the edit distance between prompts, the more likely a user is dissatisfied with a response and repeating the same or a similar question.

Example of Edit Distance. Source: Wikipedia

Other metrics to consider include tracking the average time between a user’s prompt and the LLM’s response, as well as the average time users take to write their prompts, which are valuable details on how the application is being used. User retention metrics, such as daily or weekly active usage, are also crucial indicators of long-term user engagement and overall application success.

Put it All Together

Let’s look at the whole picture: A complete monitoring system collects data on user interactions and performance and a feedback system collects data on user interactions. These datasets are analyzed and used to drive changes to improve the system. Finally, the changes are tested and validated before being deployed to production and repeating the process.

This sample repository is a good reference to get started on implementation. 

Both direct and indirect feedback provide unique and complementary insights that are essential for comprehensive understanding and app improvement. Direct feedback offers immediate user opinions, while indirect feedback reveals user behavior patterns. Together, they enable developers to address issues, optimize performance, and ultimately drive application success.

​Microsoft Tech Community – Latest Blogs –Read More 

Hello Everyone,

I will admit that I have spent a considerable amount of time on the issue that I will describe below with any success.

I try to built an pivot graph. On x-axis I have time and dates. When I expand the timeline my x-axis is a mess. Please see the picture below.

Ideally, I would like to create a graph with only the dates which I want on the graph and without the time. For example the second graph is exactly what I need. Any idea how to edit my axis?

​Hello Everyone, I will admit that I have spent a considerable amount of time on the issue that I will describe below with any success. I try to built an pivot graph. On x-axis I have time and dates. When I expand the timeline my x-axis is a mess. Please see the picture below. Ideally, I would like to create a graph with only the dates which I want on the graph and without the time. For example the second graph is exactly what I need. Any idea how to edit my axis?     Read More

​

We are excited to announce that the UNISTR intrinsic function and ANSI SQL concatenation operator (||) are now available in public preview in Azure SQL Database. The UNISTR function allows you to escape Unicode characters, making it easier to work with international text. The ANSI SQL concatenation operator (||) provides a simple and intuitive way to combine characters or binary strings. These new features will enhance your ability to manipulate and work with text data. 

What is UNISTR function?

The UNISTR function takes a text literal or an expression of characters and Unicode values, that resolves to character data and returns it as a UTF-8 or UTF-16 encoded string. This function allows you to use Unicode codepoint escape sequences with other characters in the string. The escape sequence for a Unicode character can be specified in the form of xxxx or +xxxxxx, where xxxx is a valid UTF-16 codepoint value, and xxxxxx is a valid Unicode codepoint value. This is especially useful for inserting data into NCHAR columns. 

The syntax of the UNISTR function is as follows:

UNISTR ( ‘character_expression’ [ , ‘unicode_escape_character’ ] )

The data type of character_expression could be char, nchar, varchar, or nvarchar. For char and varchar data types, the collation should be a valid UTF-8 collation only.
A single character representing a user-defined Unicode escape sequence. If not supplied, the default value is .

Examples

Example #1:

For example, the following query returns the Unicode character for the specified value:

— All the queries below will produce the same output.

SELECT UNISTR(N’Hello D83DDE00′);
SELECT UNISTR(N’048ello +01F603′);
SELECT UNISTR(N’04806506C06C06F +01F603′);

Results:

——————————-

Hello

Example #2: 

In this example, the UNISTR function is used with a user-defined escape character ($) and a VARCHAR data type with UTF-8 collation.

SELECT UNISTR (‘I $2665 Azure SQL.’ COLLATE Latin1_General_100_CI_AS_KS_SC_UTF8, ‘$’);

Results:

——————————-

I ♥ Azure SQL.

What is ANSI SQL concatenation operator (||)?

The ANSI SQL concatenation operator (||) concatenates two or more characters or binary strings, columns, or a combination of strings and column names into one expression. The || operator does not honor the SET CONCAT_NULL_YIELDS_NULL option and always behaves as if the ANSI SQL behavior is enabled. This operator will work with character strings or binary data of any supported SQL Server collation. The || operator supports compound assignment ||= similar to +=. If the operands are of incompatible collation, then an error will be thrown. The collation behavior is identical to the CONCAT function of character string data.

The syntax of the string concatenation operator is as follows:

expression || expression

The expression is a character or binary expression. Both expressions must be of the same data type, or one expression must be able to be implicitly converted to the data type of the other expression. If one operand is of binary type, then an unsupported operand type error will be thrown.

Examples

Example #1: 

For example, the following query concatenates two strings and returns the result:

SELECT ‘Hello ‘ || ‘ World!’;

Results:

——————————-

Hello World!

Example #2:

In this example, multiple character strings are concatenated. If at least one input is a character string, non-character strings will be implicitly converted to character strings.

SELECT ‘Josè’ || ‘ Doe’ AS full_name,
‘Order-‘ || CAST(1001 AS VARCHAR) || ‘~TS~’ || current_timestamp || ‘~’ || NEWID() AS order_details,
‘Item-‘ || NULL AS item_desc;

Results:

——————————-

full_name order_details                                                                                                       item_desc

Josè Doe Order-1001~TS~Jun 1 2024 6:25AM~442A4706-0002-48EC-84FC-8AF27XXXX NULL

Example #3: 

In the example below, concatenating two or more binary strings and also compounding with T-SQL assignment operator.

DECLARE @v1 VARBINARY(10) = 0x1a;
SET @v1 ||= 0x2b;
SELECT @v1 as V1, 0x|| 0x4E as B1, CAST(NEWID() AS VARBINARY) || 0xa5 as B2

Results:

——————————-

V1          B1    B2

0x1A2B       0x4E  0xAE8C602E951AC245ADE767A23C834704A5

Example #4: 

As shown in the example below, using the || operator with only non-character types or combining binary data with other types is not supported.

SELECT 1|| 2;
SELECT 1|| ‘a’ || 0x4e;

Above queries will fail with error messages as below –  

The data types int and int are incompatible in the concat operator.
The data types varchar and varbinary are incompatible in the concat operator. 

Conclusion

In this blog post, we have introduced the UNISTR function and ANSI SQL concatenation operator (||) in Azure SQL Database.  The UNISTR function allows you to escape Unicode characters, making it easier to work with international text. ANSI SQL concatenation operator (||) provides a simple and intuitive way to combine characters or binary data. These new features will enhance your ability to manipulate and work with text data efficiently. 

We hope you will explore these enhancements, apply them in your projects, and share your feedback with us to help us continue improving. Thank you!

​Microsoft Tech Community – Latest Blogs –Read More 

Bom dia,
Utilizo Office 365 (online) e tenho página de reservas no Booking, tenho disponibilidade de reservas para terças e quintas entre as 14h00 e as 16h00, com período mínimo de reserva de 24 horas e máximo de 365 dias. Porém, ao tentarem marcar consultas, descobrem que não há horário disponível. Um ponto a ressaltar é que funcionou por aproximadamente 1 mês, após esse período começamos a ter esse problema. Você poderia nos ajudar?

Conforme imagem anexa do integrante da equipe, o horário de trabalho está de acordo com o horário disponível na agenda:

​Bom dia,
Utilizo Office 365 (online) e tenho página de reservas no Booking, tenho disponibilidade de reservas para terças e quintas entre as 14h00 e as 16h00, com período mínimo de reserva de 24 horas e máximo de 365 dias. Porém, ao tentarem marcar consultas, descobrem que não há horário disponível. Um ponto a ressaltar é que funcionou por aproximadamente 1 mês, após esse período começamos a ter esse problema. Você poderia nos ajudar?Conforme imagem anexa do integrante da equipe, o horário de trabalho está de acordo com o horário disponível na agenda:    Read More

​