Introducing Semantic Workbench: Your Gateway to Agentic AI Development

In the rapidly evolving landscape of artificial intelligence (AI), the ability to quickly prototype and integrate intelligent assistants is becoming increasingly crucial. Whether you’re developing a new agent from scratch or integrating an existing one, having a versatile and user-friendly tool can make all the difference. Enter Semantic Workbench — a powerful tool designed to streamline the creation and management of intelligent agents.

The Semantic Workbench comes from our own efforts inside Microsoft to explore these ideas, it is a platform in active use. We decided to make this available as multiple teams inside Microsoft are finding value in it and we believe the broader community will as well. Separating it out as an independent project will enable teams that want to use this to add capabilities for their scenarios which will benefit everyone using it.

What is Semantic Workbench?

Semantic Workbench is a versatile tool designed to help you prototype intelligent agents quickly and efficiently. It supports the creation of new assistants or the integration of existing ones, all within a cohesive and intuitive interface. The workbench provides a user-friendly UI for creating conversations with one or more assistants, configuring settings, and exposing various behaviors.

Semantic Workbench is composed of three main components, each playing a crucial role in its functionality:

Workbench Service: The backend service that handles core functionalities, such as agents’ interaction, conversations, file uploads, authentication, and more.
Workbench App: The frontend web user interface that allows users to interact with the workbench and assistants seamlessly.
Agent Services: Any number of agent services that implement the workbench service protocol API, developed using any framework and programming language of your choice.

Why Use Semantic Workbench?

Simplified AI Agents Development

Writing a new agent and multi-agent solutions with Semantic Workbench is very simple. Developers can focus on the most important aspects, such as messaging, handling events, and executing commands. The integration with the workbench is a non-intrusive thin layer that can be removed when agents are ready for production, ensuring a smooth transition from development to deployment.

Versatility and Flexibility

Designed to be agnostic of any agent framework, language, or platform, Semantic Workbench facilitates experimentation, development, testing, and measurement of agent behaviors and workflows. This flexibility allows developers to integrate agents via a RESTful API, making it broadly applicable in various development environments. Agents can be written in any programming language and can interact with the user and each other via the Semantic Workbench web service.

User-Friendly Interface with Real-Time Insights and Debugging

Semantic Workbench provides a cohesive and intuitive interface for creating and managing conversations with intelligent assistants. This user-friendly UI simplifies the process of configuring settings and exposing various behaviors, making it accessible even to those who may not have extensive technical expertise. The interface allows users to chat with agents and offers other utilities to aid development, such as attaching debugging information to each message, including details about agent behavior, costs, and performance.

Agents can publish insights, called “states”, that are visible on side panels. These insights can be HTML/Markdown pages, allowing for the publication of complex information. Agents can update these insights in real-time, providing continuous feedback and debugging information. This feature is invaluable for developers looking to fine-tune agent behaviors and performance.

Customizable Configuration

Agents can have custom configuration settings, which are completely customizable. This includes text areas, radio buttons, checkboxes, and more, providing developers with the flexibility to tailor the settings to their specific needs. Agents can also expose configuration options such as prompts and temperature into assistant options making end user customization possible to improve iteration without having to update code.

File Management, Persistence and Cloning

The workbench also supports file management, allowing both users and agents to upload and manage files. This means AI agents can generate output as files that can be downloaded or shared with other agents, facilitating a more integrated and collaborative development environment.

Conversations and agents persist on disk, so you can develop an agent, restart it, and continue without losing state. Conversations can also be downloaded, making it easy to share demos and test results. Additionally, agents can be cloned to work with different configurations, allowing for parallel development and testing.

Conversation management

There are options within the conversation interface to delete messages from the conversation, as well as the option to rewind the conversation to a specific point which removes all the messages past that. You also have the option to add files to the conversation via a side panel or as part of a message. The side panel also provides you the option to export a conversation as markdown. There is an option to import past conversations from the dashboard screen as well.

The chat interface also informs you as to how many tokens your message is going to use.

Responses from generative assistants provide an info button which allows you to access the underlying messages that were sent to generate that response and examine details such as how many tokens were used.

Support for Multiple Users

The workbench supports multiple users. Conversations with assistants support multiple participants. Links to a conversation can be shared with other users to join. The workbench is configured by default to work with Microsoft organization or personal accounts.

Getting Started with Semantic Workbench

To get started with Semantic Workbench, you can follow these steps:

Clone the Repository: start by cloning the Semantic Workbench repository from GitHub at https://github.com/microsoft/semanticworkbench
Set Up the Workbench Service: Install and configure the backend service to handle core functionalities.
Launch the Workbench App: Use the frontend web interface to interact with the workbench and assistants.
Integrate your Agent Services: Develop and integrate agent services using any framework and programming language of your choice, following the service protocols/APIs. Agents can be developed with any editor, from VS Code and Visual Studio, to Rider, PyCharm, and any IDE.

You can get started even more quickly by opening the repository locally in VS Code, and letting it open the project as a dev container. In this mode all the required dependencies are provided automatically and there are launch configurations provided to start the Workbench Service and App. You can also open the repository directly in GitHub Codespaces for editing, please see this readme on an additional required step for running it in Codespaces.

Using the Semantic Workbench

On your first run of the Semantic Workbench, you’ll need to agree to the terms and sign in. This step is necessary even for local use because the workbench supports multiple users. Once signed in, you’ll be greeted by the Semantic Workbench dashboard.

The dashboard allows you to manage assistants. You can select an assistant from a dropdown menu if you’ve already created one or start a new one. Creating a new assistant involves selecting it from the dropdown and providing a name. The repository includes a Canonical Assistant that echoes input, which can be used as a starting point for your projects.

After saving your assistant you can select it and change any of its default setting if you like. Assistants can surface settings such as prompts, temperature, etc. here which makes it very convenient to iterate ideas without having to go back to code. This allows users who are not developers to experiment and provide feedback at earlier stages of development of generative assistants.

You can start a new conversation from either the assistant screen or the main dashboard. Opening a conversation lets you interact with your agent using a basic chat interface. You can see a response from the Echo assistant here:

If multiple assistants are involved, you can direct messages to a specific one using the “Directed to” box. The message box also reports how many tokens the message you are sending will use.

The side panel lets you see which participants across users and assistants are in the conversation. You can also download a transcript of the conversation to use it in other contexts. If there are files in the conversation they will be displayed here, and more can be added.

Explore the GitHub Repository

Semantic Workbench is publicly available on GitHub at https://github.com/microsoft/semanticworkbench. The repository contains some examples in Python and .NET, demonstrating how agents can be written in different languages. These examples also show how to leverage Azure AI Content Safety for responsible AI bots, how to render complex content types such as Mermaid graphs, Markdown, HTML, and more.

Semantic Workbench is a game-changer for anyone looking to prototype and integrate Agentic AI solutions quickly and efficiently. Its versatility, user-friendly interface, real-time insights, and seamless integration capabilities make it an invaluable tool in the AI development toolkit. Whether you’re a seasoned developer or just starting, Semantic Workbench provides the flexibility and functionality you need to bring your intelligent assistant projects to life.

​Microsoft Tech Community – Latest Blogs –Read More