Welcome to the next installment in our AI Futures blog series!

I’m Rob Chambers, and in this post, I’ll walk you through a fascinating journey supercharging a customer’s interactive visualizations solution with Chart.js and Azure AI.

Introduction

Recently, a customer contacted us with an interesting challenge: How to generate charts from data using AI? They had initially tried using Azure OpenAI Assistants with Code Interpreter, which can generate static images by building and executing Python code in real time. Although the static image approach works out of the box, the customer wanted a more interactive solution that could create more lively and dynamic charts.

Here’s an example input prompt from the customer:

Display this data using a bar-chart:

Label Brown Pink Yellow Sum
A 339 433 126 898
B 48 421 222 691

Our Approach

To address the need for interactive and dynamic charts, I had a few interactive conversations with GPT-4o, exploring different options. Eventually, we turned to Chart.js – a popular JavaScript library for creating vibrant and responsive charts. Chart.js perfectly matched the customer’s needs, allowing us to create interactive charts seamlessly integrated into web applications.

Once we selected Chart.js, we needed to determine how we could combine its features with our AI chatbot. We knew we could use the Azure AI CLI to generate a base webpage that already supports markdown and code highlighting using markd and highlight.js. We’d just need to extend this sample to include Chart.js. We decided to hook into the markdown rendering process to support a new language, ‘chartjs’, that would contain JSON objects representing the key parameters for Chart.js charts. To make this work, we had to update our OpenAI Assistant to understand how to emit this new markdown language. Here’s a snippet from our .ai/data/assistant.instructions file:

You are a helpful assistant.

One of your special skills is that you can make charts.
You can do that because you now support a new markdown language called ‘chartjs’ which contains a JSON object that represents a Chart.js chart.

Here’s an example of what the markdown for a Chart.js chart looks like:
“`chartjs
{
“type”: “bar”,
…
}

We used the Azure AI CLI to update our OpenAI Assistant with the new instructions, and to create a simple locally runnable HTML/JavaScript web application:

ai chat assistant update –instructions “@assistant.instructions”
ai dev new openai-asst-webpage-with-functions –javascript

Modifying our web application

Now that our OpenAI Assistant is ready to go, we needed to update our base web application to support the new ‘chartjs’ markdown language.

In the AI Futures team, we always try to use AI as much as possible in our development process, either by using GitHub Copilot inside VS Code or by running the Azure AI CLI interactively. This time, we used the AI CLI to read our existing code, make suggestions and changes, and then save the files back to disk. We reviewed the updates along the way using VS Code’s built-in diff viewer, accepting or rejecting changes as needed. You can take a look at a prompt that is a conglomerate of the changes we made in our GitHub repo here.

Here’s how we invoked the AI CLI to read, understand, and modify the web application:

cd openai-asst-webpage-with-functions-js
ai chat –interactive –built-in-functions –system @SYSTEMPROMPT.md –user @USERPROMPT.md

The ‘–built-in-functions’ option is the key to giving the AI CLI and the LLM access to read and write files on disk.

Integrating Chart.js

Here’s what we (the AI CLI and me) did to integrate Chart.js into the web application:

Extending marked to Handle chartjs “language” Blocks

We extended the markd library to handle the new language block. If the code block’s language is ‘chartjs’, we return a new div with a class of ‘chartjs’ where we’ll render the chart using Chart.js:

marked.setOptions({
highlight: function (code, lang) {
if (lang === ‘chartjs’) {
let chart = createNewChartJsChart(code);
return `<div class=”chartjs”>${chart}</div>`;
} let hl = lang === undefined || lang === ” ? hljs.highlightAuto(code).value : hljs.highlight(lang, code).value;
return `<div class=”hljs”>${hl}</div>`;
}
});

Rendering the chart

The createNewChartJsChart function above parses the JSON, generates the required HTML canvas elements, and defers the chart population until the DOM elements are available. Once the DOM elements are ready, the populateNewChartNow function is called to render the chart using Chart.js:

function populateNewChartNow(chart) {
let ctx = document.getElementById(chart.id).getContext(‘2d’);
new Chart(ctx, {
type: chart.config.type,
data: chart.config.data,
options: chart.config.options
});
}

Conclusion and Call to Action

The end result is a dynamic and interactive chatbot that uses Chart.js to generate charts from data, making the customer’s experience more engaging and insightful. The customer can now simply provide data in their prompts and receive interactive visualizations in response.

The journey to integrate Chart.js and enhance our AI chatbot showcases the power of Azure AI, both in the form of innovative and engaging end-user solutions, as well as in the development of such experiences.

Ready to transform your AI applications with dynamic charts?

Check out the following links for more information:

https://github.com/robch/openai-asst-webpage-with-functions-and-chart-js

https://github.com/Azure/azure-ai-cli

https://thebookof.ai

https://ai.azure.com

​Microsoft Tech Community – Latest Blogs –Read More