Month: October 2024
The Future of AI: Supercharge Your AI Chatbot with Interactive Visualizations Using Chart
Welcome to the next installment in our AI Futures blog series!
I’m Rob Chambers, and in this post, I’ll walk you through a fascinating journey supercharging a customer’s interactive visualizations solution with Chart.js and Azure AI.
Introduction
Recently, a customer contacted us with an interesting challenge: How to generate charts from data using AI? They had initially tried using Azure OpenAI Assistants with Code Interpreter, which can generate static images by building and executing Python code in real time. Although the static image approach works out of the box, the customer wanted a more interactive solution that could create more lively and dynamic charts.
Here’s an example input prompt from the customer:
Display this data using a bar-chart:
Label Brown Pink Yellow Sum
A 339 433 126 898
B 48 421 222 691
Our Approach
To address the need for interactive and dynamic charts, I had a few interactive conversations with GPT-4o, exploring different options. Eventually, we turned to Chart.js – a popular JavaScript library for creating vibrant and responsive charts. Chart.js perfectly matched the customer’s needs, allowing us to create interactive charts seamlessly integrated into web applications.
Once we selected Chart.js, we needed to determine how we could combine its features with our AI chatbot. We knew we could use the Azure AI CLI to generate a base webpage that already supports markdown and code highlighting using markd and highlight.js. We’d just need to extend this sample to include Chart.js. We decided to hook into the markdown rendering process to support a new language, ‘chartjs’, that would contain JSON objects representing the key parameters for Chart.js charts. To make this work, we had to update our OpenAI Assistant to understand how to emit this new markdown language. Here’s a snippet from our .ai/data/assistant.instructions file:
You are a helpful assistant.
One of your special skills is that you can make charts.
You can do that because you now support a new markdown language called ‘chartjs’ which contains a JSON object that represents a Chart.js chart.
Here’s an example of what the markdown for a Chart.js chart looks like:
“`chartjs
{
“type”: “bar”,
…
}
We used the Azure AI CLI to update our OpenAI Assistant with the new instructions, and to create a simple locally runnable HTML/JavaScript web application:
ai chat assistant update –instructions “@assistant.instructions”
ai dev new openai-asst-webpage-with-functions –javascript
Modifying our web application
Now that our OpenAI Assistant is ready to go, we needed to update our base web application to support the new ‘chartjs’ markdown language.
In the AI Futures team, we always try to use AI as much as possible in our development process, either by using GitHub Copilot inside VS Code or by running the Azure AI CLI interactively. This time, we used the AI CLI to read our existing code, make suggestions and changes, and then save the files back to disk. We reviewed the updates along the way using VS Code’s built-in diff viewer, accepting or rejecting changes as needed. You can take a look at a prompt that is a conglomerate of the changes we made in our GitHub repo here.
Here’s how we invoked the AI CLI to read, understand, and modify the web application:
cd openai-asst-webpage-with-functions-js
ai chat –interactive –built-in-functions –system @SYSTEMPROMPT.md –user @USERPROMPT.md
The ‘–built-in-functions’ option is the key to giving the AI CLI and the LLM access to read and write files on disk.
Integrating Chart.js
Here’s what we (the AI CLI and me) did to integrate Chart.js into the web application:
Extending marked to Handle chartjs “language” Blocks
We extended the markd library to handle the new language block. If the code block’s language is ‘chartjs’, we return a new div with a class of ‘chartjs’ where we’ll render the chart using Chart.js:
marked.setOptions({
highlight: function (code, lang) {
if (lang === ‘chartjs’) {
let chart = createNewChartJsChart(code);
return `<div class=”chartjs”>${chart}</div>`;
} let hl = lang === undefined || lang === ” ? hljs.highlightAuto(code).value : hljs.highlight(lang, code).value;
return `<div class=”hljs”>${hl}</div>`;
}
});
Rendering the chart
The createNewChartJsChart function above parses the JSON, generates the required HTML canvas elements, and defers the chart population until the DOM elements are available. Once the DOM elements are ready, the populateNewChartNow function is called to render the chart using Chart.js:
function populateNewChartNow(chart) {
let ctx = document.getElementById(chart.id).getContext(‘2d’);
new Chart(ctx, {
type: chart.config.type,
data: chart.config.data,
options: chart.config.options
});
}
Conclusion and Call to Action
The end result is a dynamic and interactive chatbot that uses Chart.js to generate charts from data, making the customer’s experience more engaging and insightful. The customer can now simply provide data in their prompts and receive interactive visualizations in response.
The journey to integrate Chart.js and enhance our AI chatbot showcases the power of Azure AI, both in the form of innovative and engaging end-user solutions, as well as in the development of such experiences.
Ready to transform your AI applications with dynamic charts?
Check out the following links for more information:
https://github.com/robch/openai-asst-webpage-with-functions-and-chart-js
https://github.com/Azure/azure-ai-cli
Microsoft Tech Community – Latest Blogs –Read More
SharePoint content processing updates – translation and simple prebuilt models
As we introduce some of the latest SharePoint innovations, features, and capabilities coming this Fall, a clear theme emerges: Cloud computing and AI continues to drive massive digital transformations and change the way we approach tasks and workflows and tasks.
These days, our closest coworkers, partners, and customers are operating globally, and our roadmap focuses on helping dispersed organizations boost efficiency, maximize productivity, and forge deeper, more meaningful connections with their customers wherever they are. Today, we’ll cover the latest SharePoint product updates around content processing and enrichment: translation, custom glossaries with translation, and the new simple document processing prebuilt model. Also, we’re excited to share opportunities for you to learn more at upcoming events.
In a global marketplace where we regularly engage with diverse populations, it’s critical to offer translation features that break down barriers and promote accessibility, inclusivity, and cultural relevance. And with the pace of business in this digital era, to be able to do this efficiently.
Beyond translation enhancements, we’re fully embracing AI’s power to empower SharePoint admins, allowing them to monitor their sites more effectively and ensure consistency in settings, policies, and content governance.
Here are some of the key roadmap features from SharePoint AI-powered metered processing services we’re excited to share with you:
Multi-language translation selection
Today, in your SharePoint document library, you can easily create translated copies of selected files. We’ve enhanced this feature, allowing you to translate a document into multiple languages simultaneously. Simply right click on the demand translation or in the command bar or automated translation via a rule, select the languages you need (up to 10 per request) and the translated documents are generated. As always, translation is available for all supported languages and dialects. This multi-language translation feature supports various file types, including Word documents and closed captioning text files.
Through June 2025, you can try out document translation and other selected Syntex services at no cost with free monthly capacity if you have pay-as-you-go billing set up.
Feature ID: 410213
Learn more about document translation
Enhanced glossary support
Every sector, industry, and business have their own specialized terminology. Whether you’re a pharmacist, engineer, or mechanic, the unique terminology and acronyms are integral to how your business operates. To support this need, you can now create a custom glossary that includes specific translations. For instance, if a pharmaceutical company is developing labeling and packaging materials for a new drug, Xyraphen, you can define in the glossary that whenever ‘Xyraphen’ appears, it should be translated in a specific way in French or German.
It’s as simple as adding your glossary file (.csv or .tsv format) to your document translation request. While you can translate up to 10 languages at a time, when including a custom glossary you will only be able to make one translation request. The glossary will be available at the SharePoint site level; users can point to the same custom glossary in multiple libraries from within the same site.
Simple document processing prebuilt model
You’ll recall that SharePoint offers a variety of document processing models that fall into two categories: custom models and pre-built models. The key distinction is that pre-built models are pretrained; they have been designed around specific schemas, such as business file types. For example, a pre-built schema for contracts is trained to recognize a defined set of common fields typically found in contract documents.
Pretraining involves a range of document types but is ultimately constrained by the defined model. As such, pre-built models are limited to recognizing what they have been trained on. This is advantageous when the pre-trained model aligns perfectly with your document parameters, but it can pose challenges when you need to capture fields outside of those parameters.
Custom models on the other hand, offer greater flexibility to address unique business scenarios. but require more upfront development time.
Seeing the need for both flexibility and time to value, we created the simple document processing pre-built model, which uses Optical Character Recognition (OCR) and deep learning AI. This is a good option if you need to extract to extract key-value pairs, tables, and selection marks from documents – such as barcodes or checkboxes – that aren’t recognized in our other prebuilt model schemas.
This approach is particularly useful if your business uses a standard form tailored to capture common attributes in your content, such as identifying various barcode types. The configuration process is similar, but the main difference is that simple document processing detects common elements in documents without a set schema. This feature can be invaluable for extracting necessary information from unique documents, helping to automate the detection process.
Additionally, simple document processing can detect the primary language of the document. When applied to a specific document library, it can extract language information based on your configuration, returning an ISO language code once the model runs.
Pre-built models offer ease of configuration, but due to their pre-trained nature, they may not always capture all the key information you need. Simple document processing serves as a valuable intermediary between custom and pre-built models, providing maximum flexibility in analyzing your content.
Learn more using a prebuilt model to extract information from simple documents
SharePoint eSignature updates
Have you tried SharePoint eSignature yet? We’re excited to that eSignature has expanded availability. Click here to check out our blog outlining the latest eSignature updates, including a new feature and expansion to new markets.
Join this online event – Microsoft Digital: AI tagging makes finding content easier
Last but not least, on 10/29 you are invited to an online event where we will be discussing how Microsoft is leveraging SharePoint’s Taxonomy Tagging feature to drive efficiency in content management. For more event details, please click here.
We hope you find the updates above useful. As always, we value your feedback! Please leave your comments below.
For additional SharePoint information, please read our technical documentation at aka.ms/SharePointDocs
Connect with us at a variety of events and webinars coming this fall, including Microsoft Ignite November 19-22 and ESPC Conference happening December 2 – 5.
For a list of all upcoming events please refer to Microsoft Tech Community Events
Not a SharePoint customer yet? Check out our latest features, demos, and pricing.
Microsoft Tech Community – Latest Blogs –Read More
SharePoint eSignature product updates!
The pace of modern work is rapid, and our digital workplaces naturally accelerate how tasks get done. As we continue to develop SharePoint innovations, we constantly challenge ourselves to find and build efficiencies for our customers. This includes simplifying the process of signing and sharing business-critical documents while maintaining enterprise-grade security and compliance.
Earlier this year, we began rolling out SharePoint eSignature in the U.S. market. This feature allows you to request, sign, and store signed documents seamlessly within your workflow, ensuring your documents remain secure and compliant. Throughout the review, signing, and record-keeping processes, your content stays within the Microsoft 365 trust boundary.
Our ongoing innovation around SharePoint eSignature focuses on enhancing efficiency, saving time and money, and ensuring security and compliance for both your organization and your partners.
We’re excited to share the following updates to SharePoint eSignature:
Expanding eSignature to Canada and UK
When we first launched SharePoint eSignature, it was only available in the U.S., so we’re excited to announce that availability has now expanded to the UK and Canada. We’re also targeting November for availability in the rest of Europe and in a set of countries within APAC, with global availability planned for 2025.
Sequential signing
In addition to the rollout to new markets, creators of eSignature requests can now specify an order for signing. The document is first signed by the initial recipient in the sequence, then automatically forwarded to the subsequent recipient for their signature. This process continues in the designated order until every recipient has added their signature. Once everyone has signed, the signed document is made available to all involved. This feature unlocks numerous scenarios where sequential signing is necessary and makes this workflow significantly more efficient. With sequential signing, requesters can control the signing order of the workflow.
Adobe Acrobat Sign and Docusign eSignature integration with SharePoint eSignature is generally available (US, UK, Canada)
SharePoint users can now initiate Adobe Acrobat Sign or DocuSign eSignatures directly from PDF documents. The eSignature request will be processed by Adobe or DocuSign, and a copy of the signed document will automatically save to SharePoint. This integration ensures business-critical documents are saved back to Microsoft 365. Users will not need to manually download or upload documents to initiate a request or save signed documents to SharePoint for long-term storage.
This Adobe Acrobat Sign and Docusign integration is currently available to customers in the US, UK and Canada with a global rollout expected in November 2024.
To use Adobe or Docusign integration within SharePoint eSignature, an IT admin will need to activate it from the Microsoft 365 Admin Center and specify the SharePoint sites where it can be used. You can learn more about the setup here.
Using the third-party offering of eSignature from Microsoft is no additional charge. Pay-as-you-go billing must be set up, customers will not be charged. However, customers must ensure that they have valid licenses with Adobe or Docusign.
Feature ID: 382128 and Feature ID: 382129
From the Select a provider drop-down, you can choose from SharePoint eSignature, Adobe Acrobat Sign, or Docusign
Learn more about SharePoint eSignature
We hope you find the updates above useful. As always, we value your feedback! Please leave your comments below.
For additional SharePoint information, please read our technical documentation at aka.ms/SharePointDocs
Stay connected! We have a variety of events coming this fall, including Microsoft Ignite November 19-22 and ESPC Conference happening December 2–5! For a list of all upcoming events please refer to Microsoft Tech Community Events.
Not a SharePoint customer yet? Check out our latest features, demos, and pricing.
Microsoft Tech Community – Latest Blogs –Read More
OpenUSD: Bridging the gap between the physical and digital worlds
In today’s rapidly evolving digital landscape, the need for standardized data models and interoperability has never been more critical. OpenUSD, or Universal Scene Description, is a powerful framework designed to address these challenges by providing a unified data model for constructing and packaging complex 3D scenes. Originally developed by Pixar, OpenUSD has gained significant traction across various industries, including manufacturing, energy, media, and entertainment.
What is OpenUSD?
OpenUSD is a versatile and extensible framework that allows for the interchange and augmentation of 3D scenes composed of many component assets. Its unique composition ability provides rich and varied ways to combine assets into larger assemblies, enabling collaborative workflows where multiple developers can work together seamlessly. This makes OpenUSD a powerful tool for many industries that rely heavily on 3D data and complex simulations.
Although OpenUSD got its start in media and entertainment, it has steadily gained traction in other industries as well including manufacturing and energy. It’s unique ability to provide a unified data model and facilitate collaborative workflows makes it an invaluable tool for many industries. The Alliance for OpenUSD (AOUSD) plays a crucial role in accelerating the adoption and use of OpenUSD, ensuring that many industries can leverage its full potential for digital transformation, innovation, and efficiency.
Manufacturing: In the manufacturing sector, OpenUSD facilitates the creation of highly detailed, interactive 3D models of facilities, such as manufacturing plants. It also provides data interoperability and reduces the costs associated with existing proprietary systems, allowing smaller companies to compete and innovate faster. Integrating operational data (OT) from physical assets and IT data from other systems of record, manufacturers can streamline complex workflows, enhance predictive maintenance, and improve overall operational efficiency.
Energy: The energy industry benefits from OpenUSD’s ability to provide semantic meaning from non-3D enterprise data alongside 3D data for digital twins exploration and simulation. This capability enables energy companies to create accurate virtual models of their infrastructure, facilitating better planning, monitoring, and maintenance of energy assets. The integration of real-time telemetry from physical assets further enhances the efficiency and reliability of energy operations.
Media and Entertainment: OpenUSD has its roots in the media and entertainment industry, where it has driven innovation and collaboration. By providing a standardized framework for 3D content creation, OpenUSD allows artists and developers to work together more effectively, creating immersive experiences and high-quality visual effects. The framework’s flexibility and extensibility make it ideal for managing large-scale datasets and complex data dependencies, essential for film and animation pipelines.
The Role of the Alliance for OpenUSD
The Alliance for OpenUSD (AOUSD) was formed in 2023 to drive the adoption and development of OpenUSD standards across various industries. Comprising major players like Pixar, Adobe, Apple, Autodesk, and NVIDIA, the AOUSD aims to foster global collaboration and improve the interoperability of 3D content. Microsoft’s involvement in the AOUSD represents a pivotal step in our commitment to open standards and digital transformation.
By joining the AOUSD, Microsoft aims to strengthen the OpenUSD ecosystem and make 3D data more accessible. This collaboration enables customers and partners to take advantage of optimized workflows, empowering them to create immersive experiences. The AOUSD continues to expand its membership and form new interest groups to drive innovation and inclusivity.
Future Potential of OpenUSD
The future potential of OpenUSD is vast and promising. As industries continue to embrace digital transformation, the demand for standardized and interoperable 3D data models will only grow. OpenUSD’s ability to integrate with emerging technologies like AI and robotics positions it as a key driver in the next wave of innovation. For instance, OpenUSD’s ability to describe the physics for simulation capabilities make it an ideal framework for creating operational digital twins and immersive simulations. This can revolutionize industries by providing more accurate and dynamic models for planning, monitoring, and maintenance.
OpenUSD’s role in AI development is particularly noteworthy with a growing belief that the next wave of AI will be powered by incorporating more information about how our world operates from a physics point-of-view. OpenUSD, with its expressive way to codify knowledge and build “playgrounds” for AI to gather experiences, is poised to play a crucial role in this evolution. As AI systems become more sophisticated, the ability to create and manipulate complex 3D scenes will be essential for training and development.
Microsoft’s Involvement with the Alliance for OpenUSD
Microsoft officially joined the Alliance for OpenUSD in July 2024. This collaboration marks a significant milestone in our commitment to driving digital transformation across various industries. Microsoft aims to enhance its support for industrial customers by ensuring that operations data is effectively integrated into the OpenUSD format and to better accommodate equipment IDs and attributes within USD, facilitating seamless interoperation with non-3D enterprise data.
Microsoft plans to contribute to the work being done by the AOUSD by participating in working groups and driving the development of USD schemas for industrial operations. This involvement will help bridge the gap between 3D and non-3D data, improving overall efficiency and data consistency in industrial operations. Additionally, Microsoft aims to leverage OpenUSD in its products, such as Azure and Omniverse, to provide powerful operational digital twins and immersive simulations.
OpenUSD is a game-changer for industries that rely on 3D data and complex simulations and its ability to provide a unified data model and facilitate collaborative workflows makes it invaluable tool industrial customers, as well as media and entertainment. The Alliance for OpenUSD plays a crucial role in accelerating the adoption and use of OpenUSD, ensuring that industries can leverage its full potential for digital transformation. The future of OpenUSD is bright, with its potential to drive innovation and efficiency across various sectors, making it a cornerstone of the digital future.
Microsoft Tech Community – Latest Blogs –Read More
Analytics solution options for SharePoint Online
Hi SharePoint Brain trust!
We use Cardiolog analytics for our SharePoint Online environment. We’re exploring options to get rid of this and potentially use native tooling in SPO or an economical option. Any ideas are greatly appreciated !
Thank you!
Hi SharePoint Brain trust! We use Cardiolog analytics for our SharePoint Online environment. We’re exploring options to get rid of this and potentially use native tooling in SPO or an economical option. Any ideas are greatly appreciated ! Thank you! Read More
can’t use create_from_edge_environment with proxy options – client doesn’t connect to azure iot hub
Hello,
I am trying to configure a client connection towards azure iot hub from a module. I’ve tried creating the client with create_from_edge_environment method and I get the following error:
Traceback (most recent call last): File “/usr/local/lib/python3.10/site-packages/azure/iot/device/common/mqtt_transport.py”, line 391, in connect rc = self._mqtt_client.connect( File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 914, in connect return self.reconnect() File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 1044, in reconnect sock = self._create_socket_connection() File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 3683, in _create_socket_connection return socks.create_connection(addr, timeout=self._connect_timeout, source_address=source, **proxy) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 209, in create_connection raise err File “/usr/local/lib/python3.10/site-packages/socks.py”, line 199, in create_connection sock.connect((remote_host, remote_port)) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 47, in wrapper return function(*args, **kwargs) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 814, in connect raise GeneralProxyError(“Socket error”, error) socks.GeneralProxyError: Socket error: 407: Proxy Authentication Required
This problem doesn’t appear if I create the client using create_from_connection_string method. This adds complexity to my script and I would like to use create_from_edge_environment method.
Does anybody have any insights on why one method works and the other doesn’t?
Thanks!
Hello, I am trying to configure a client connection towards azure iot hub from a module. I’ve tried creating the client with create_from_edge_environment method and I get the following error: Traceback (most recent call last): File “/usr/local/lib/python3.10/site-packages/azure/iot/device/common/mqtt_transport.py”, line 391, in connect rc = self._mqtt_client.connect( File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 914, in connect return self.reconnect() File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 1044, in reconnect sock = self._create_socket_connection() File “/usr/local/lib/python3.10/site-packages/paho/mqtt/client.py”, line 3683, in _create_socket_connection return socks.create_connection(addr, timeout=self._connect_timeout, source_address=source, **proxy) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 209, in create_connection raise err File “/usr/local/lib/python3.10/site-packages/socks.py”, line 199, in create_connection sock.connect((remote_host, remote_port)) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 47, in wrapper return function(*args, **kwargs) File “/usr/local/lib/python3.10/site-packages/socks.py”, line 814, in connect raise GeneralProxyError(“Socket error”, error) socks.GeneralProxyError: Socket error: 407: Proxy Authentication Required This problem doesn’t appear if I create the client using create_from_connection_string method. This adds complexity to my script and I would like to use create_from_edge_environment method. Does anybody have any insights on why one method works and the other doesn’t? Thanks! Read More
Excel does not scroll on the main screen while duplicating screens
Hey y’all,
a coworker of mine has this problem on his laptop when he is duplicating his laptop screen on a monitor. does someone know a fix? it is only with Excel and not with other programs
The Specs:
Intel i7 8th gen pro verison
an ssd
windows 10
Hey y’all, a coworker of mine has this problem on his laptop when he is duplicating his laptop screen on a monitor. does someone know a fix? it is only with Excel and not with other programs The Specs:Intel i7 8th gen pro verisonan ssdwindows 10 Read More
Group Company and Azure
Hi,
Apologies if this has already been asked and answered in a previous post but I’m wondering what people’s thoughts are on how to organise your Azure platform for a group company?
Do you recommend splitting them out into different subscriptions or same subscription different databases etc? how is the best way to structure?
Thanks
Hi, Apologies if this has already been asked and answered in a previous post but I’m wondering what people’s thoughts are on how to organise your Azure platform for a group company? Do you recommend splitting them out into different subscriptions or same subscription different databases etc? how is the best way to structure? Thanks Read More
Can not create subsite in Sharepoint Hub anymore (SP Online)
I used to be able to go to site contents -> New -> Subsite
Option is no longer there anymore. Is there a fix to this?
I used to be able to go to site contents -> New -> SubsiteOption is no longer there anymore. Is there a fix to this? Read More
some modules are not displayed correctly
Why this and other pages are not visualized in the correct way? Button “next” is missing and the points too.
https://learn.microsoft.com/it-it/training/modules/m365-compliance-information-prevent-data-loss/
Why this and other pages are not visualized in the correct way? Button “next” is missing and the points too.https://learn.microsoft.com/it-it/training/modules/m365-compliance-information-prevent-data-loss/ https://learn.microsoft.com/it-it/training/modules/purview-exercise-information-protection-data-loss-prevention/configure-environment Read More
Old .NET versions automatic uninstallation/removal
Hello,
How are you removing old versions of .NET from your devices? Is there a way to automate this?
I appreciate your response and help.
Thanks,
Mark
Hello, How are you removing old versions of .NET from your devices? Is there a way to automate this? I appreciate your response and help. Thanks,Mark Read More
Celebrating MVPs Achieving their 10 Year Milestone
As we commemorate over 30 years of the Microsoft Most Valuable Professionals (MVP) Program, we want to sincerely acknowledge the efforts those who have achieved award milestones of 10, 15, 20 years, and more. This journey wouldn’t be possible without your support and dedication to community leadership – thank you!
This blog features a few MVPs achieving their 10 year milestone. Read on to find out what these MVPs have to say about their experience and time in the program.
What has motivated you to remain committed to the MVP program for the past 10 years?
Prajwal Desai Security/Windows and Devices MVP (India): “Becoming an MVP was a dream come true for me because helping others is my passion. Every year, I work hard to maintain this status, and being part of the Microsoft community continues to inspire and motivate me. It pushes me to keep learning, publishing, and embracing new challenges. I’m eager to remain in this role, as the MVP program provides me with incredible knowledge and the opportunity to learn from other professionals. It’s a truly rewarding experience!”
Stefan Malter M365 MVP (Germany): “As an author, media trainer and Microsoft MVP, I appreciate the exchange with the product groups. They hear me when it comes to the perspective and needs of teachers and lecturers. Technical developments like AI and cloud computing are a big challenge for the education sector worldwide. Being part of the MVP program also gives me important insights that allow me to translate the digital progress for my less tech-savvy target group.”
Josh Garverick Microsoft Azure/Developer Technology MVP (US): “There are two things that continue to draw me into the community, being able to see my impact on those who I talk with and the ability to help shape the many different products that Microsoft has through conversations with the Product Groups. I love talking with people about all sorts of technologies and it makes me feel great if I can help them out in their understanding of a service or product!”
Chris Gomez Developer Technology MVP (US): “My goals align well with the MVP program, focusing on educational outreach and technical content. I aim to help developers solve problems through videos, presentations, and articles, often inspired by challenges I’ve faced. I was mentored by great teachers and senior developers, and I feel a responsibility to pass that knowledge forward, especially to underrepresented communities. After ten years as an MVP, I’m honored and grateful to continue helping others save time and build on what I’ve learned.”
What impact has the MVP community had on your personal and professional growth?
Prajwal Desai Security/Windows and Devices MVP (India): “Being an MVP and being part of the Microsoft community, my knowledge and skills have been greatly enhanced. The MVP community is a great source of learning and inspiration for me. Each MVP possesses distinct abilities, and collectively, we exchange a wealth of knowledge and intelligence. Personally, the MVP community inspires me to learn and contribute, and professionally, I have noticed that my knowledge at work and the way I handle things have improved.”
Stefan Malter M365 MVP (Germany): “The MVP program has changed my life in so many ways. Getting to other dedicated enthusiasts has led to many helpful conversations and special connections over the years. But I also see the MVP award as a confirmation for my community work. It has become a yearly boost for my self-confidence, and the associated benefits allow me to discover new technical possibilities that otherwise would not be part of my learning content.”
Josh Garverick Microsoft Azure/Developer Technology MVP (US): “It’s difficult to verbalize, honestly. When I first joined the MVP program, I was very reserved and just seeing folks like Scott Hanselman and Scott Guthrie blew my mind. I’ve since become much more comfortable with myself as well as more outgoing to other members in the community. Professionally it helped me learn a ton and apply what I’ve learned, advancing myself to where I am today. Personally, I’m more capable of speaking to groups of people and have a better level of self-confidence.”
Chris Gomez Developer Technology MVP (US): “Fellow MVPs are a tremendous source of knowledge, both through their online content and local involvement. Early in my journey, I learned from their articles, talks, and now through videos, projects, and courses across a wide range of technologies. Locally, I was inspired by the MVPs in the Philly.NET community, which welcomed me and gave me the platform for my first technical talk. This community, led by MVPs, became the foundation for my career growth, helping me evolve from a developer to a software architect.”
What advice would you give to new MVPs just starting their journey?
Prajwal Desai Security/Windows and Devices MVP (India): “I would tell all the new MVPs that technology is developing far more quickly than it did in the past. In addition to improving your abilities with certifications, you also need to keep up with these advancements. I always recommend subscribing to Microsoft blogs, MVP newsletters, following your MVP gurus on social platforms, and attending free training provided by Microsoft. This will enhance the knowledge and skills that are required for career progression. Lastly, I would say to all the new MVP’s, You’re awesome and you are in the perfect community. Keep learning, stay motivated and help others, the rewards will follow you.”
Stefan Malter M365 MVP (Germany): “At first, I found it hard to understand how extensive and valuable the MVP program really is. Take your time to discover all the possibilities, but do not feel obliged to embrace everything at once. There is also no need to be shy when it comes to contacting the product groups or visit one of the Microsoft events. All people are really open-minded and nice and – as we say in Germany – also only cook with water.”
Josh Garverick Microsoft Azure/Developer Technology MVP (US): “Ask LOTS of questions, don’t be afraid to reach out to other MVPs either in person or via LinkedIn, and most of all, don’t be afraid to say hi to people you recognize from conference talks, online videos, or social media!”
Chris Gomez Developer Technology MVP (US): “As a new MVP, stay true to yourself and take advantage of the great opportunities available, including engaging with product teams. While imposter syndrome is real, your feedback is valuable and represents many voices. Don’t hesitate to explore other tech areas that interest you, and use the resources available, like distribution lists and video libraries. Always pay attention to NDAs when sharing information and ask if you’re unsure. Most importantly, enjoy the journey and maintain a healthy work-life balance—you’re recognized for your passion to help others.”
How do you balance your MVP activities with your professional & personal commitments?
Prajwal Desai Security/Windows and Devices MVP (India): “Everyone goes through ups and downs and balancing personal life with professional commitments can be difficult. In the past 10 years, I have learnt how to prioritize things so that I can balance my MVP responsibilities with professional commitments. For example, every day I spent at least 3–4 hours on personal learning after my job shift. In another instance, I would mark a meeting for this coming Monday on my calendar and make sure no other business took priority during that time. At times during my office hours when I couldn’t attend a MVP meeting, I made sure to watch the recorded meeting after my work hours.”
Stefan Malter M365 MVP (Germany): ”I am always aware of my versatile role as a community leader and have found my way to balance all interests. I can be an independent author and media trainer with critical views on technological developments and media competency. At the same time, I can appreciate the constructive exchange with professionals at Microsoft and discuss the challenges we all face in this fast-changing world. This is how this program – to me – has become THE Most Valuable Puzzle.”
Josh Garverick Microsoft Azure/Developer Technology MVP (US): “It’s certainly not the easiest thing to do, but I am fortunate to have such a great employer who allows me to work with folks at Microsoft and GitHub. I am encouraged to submit talks to and attend conferences, and community accomplishments are celebrated amongst our colleagues. I also guard my free time, making sure I prioritize family events and other non-technical activities. Sometimes I just need to hack on something for a couple of hours to scratch that itch, though.”
Chris Gomez Developer Technology MVP (US): “Balancing professional commitments and community involvement requires respect for both. While it’s exciting to work on personal projects for the community, maintaining a healthy work/MVP/life balance is essential. Just as you protect confidential work information, it’s important to keep MVP program content and job responsibilities separate. What you learn over time can benefit both your career and the community, but it’s crucial to maintain clarity between the two to respect both your employer and the MVP program.”
Thank you MVPs!
Thanks to everyone who shared their experiences, and congratulations once more on reaching this 10-year milestone. If you are interested in becoming a Microsoft MVP, please visit our website to learn more.
Microsoft Tech Community – Latest Blogs –Read More
Enhancing Server and Container Risk Score Analysis in Power BI
Microsoft Defender for Cloud provides vulnerability assessments for both virtual machines (servers) and container images, identifying vulnerabilities as Common Vulnerabilities and Exposures (CVEs). The risk posed by each CVE is assessed using the Common Vulnerability Scoring System (CVSS), providing a standardized numerical score that ranges from 0.0 to 10.0, translated into severity ratings like Low, Medium, High, or Critical. While Microsoft Defender for Cloud provides a robust risk level assessment for each resource, there is an opportunity to enhance this by integrating additional factors such as the exploitability of each CVE, the age since it was made public, and whether the CVE is a zero-day vulnerability. Additionally, resources themselves have contextual elements such as the number of attack paths, which can significantly impact their overall risk. The Power BI solution builds Defender for Cloud’s capabilities by integrating these multiple factors, providing a more comprehensive risk score for each resource and enhancing the prioritization of vulnerabilities requiring urgent remediation. This combined approach allows users to generate a more accurate top-down list of resources needing attention.
This model is based on a Deterministic Approach, meaning that the conditions and assumptions used for calculating the final Risk Score use finite values defined and fixed by the consumer of the report. While it is not perfect science, it provides more depth to understand how to rank the resources. The end user is free to change the fixed values (weights) as needed, allowing flexibility in adapting the model to specific needs.
Download the report file here – aka.ms/pbiSrvCntRisk
Key Factors in Risk Correlation for Servers and Containers
In the Power BI solution, a scoring model was developed to correlate the CVE severity, exploit information, and the contextual risk level of each resource to produce a risk score. This helps generate a prioritized list of which servers or container images should be remediated first. Here’s how the model achieves this:
CVE Severity (CVSS Score Weight)
Each CVE is assigned a score based on its severity:
Critical: 40 points
High: 30 points
Medium: 20 points
Low: 10 points
Exploit Information (Exploitability Weight)
If a CVE has characteristics that make it more likely to be exploited, additional points are added:
Remote Exploit: +10 points
Exploit Kit Exists: +15 points
Exploit Kit Verified: +20 points
Public Exploit Available: +25 points
Contextual Resource Risk Level
The risk level of each resource is calculated based on several factors from Defender for Cloud, such as exposure to the internet and the presence of sensitive data:
Critical Resource: +40 points
High Risk: +30 points
Medium Risk: +20 points
Low Risk: +10 points
Attack Path Weight (Server Exposure)
For servers, the number of attack vectors is also included:
6 or more vectors: +30 points
3 to 5 vectors: +20 points
1 to 2 vectors: +10 points
Age of Vulnerabilities (CVE Age)
The age of a CVE (time since it was made public) plays a significant role in prioritization. Older CVEs are considered riskier if they remain unpatched, as attackers have had more time to develop exploits. The TotalVulnerabilityScore is adjusted by a multiplier based on the CVE’s age:
Less than 30 days: 1.1 multiplier
30 to 180 days: 1.3 multiplier
More than 180 days: 1.5 multiplier
Handling Multiple CVEs and Risk Prioritization
Servers and container images often have multiple CVEs, sometimes hundreds, which significantly impacts the risk score calculation and prioritization. Here’s how the scoring model adapts to this situation:
Aggregate CVE Scores: The scores of all CVEs for each resource are aggregated to derive a comprehensive risk score. This helps emphasize resources with a larger number of vulnerabilities.
Logarithmic Scaling for Excessive CVEs: A logarithmic scaling factor is introduced to avoid disproportionately high scores for resources with a large number of CVEs. For instance, the logarithmic function (e.g., log10) scales the score growth to keep it manageable while still reflecting the increased risk.
Dynamic Classification Using Percentiles
To make the CombinedRiskScore more comprehensible, the scores are dynamically classified into risk levels using percentile-based thresholds:
Critical Risk: Top 25% of scores.
High Risk: Between 50th and 75th percentiles.
Medium Risk: Between 25th and 50th percentiles.
Low Risk: Bottom 25%.
Scaling for Usability
To make the CombinedRiskScore user-friendly and easier to interpret, a percentile-based approach was used for categorization. The scores are also scaled down (e.g., divided by 1000) to avoid presenting overly large numbers to end users, making the information more digestible.
Visualization in Power BI
The Power BI dashboard allows users to easily view and prioritize remediation efforts across servers and container images. Here’s how the scoring model is visualized:
Risk Levels by Category: Each server and container image are categorized into risk levels (e.g., Critical, High, Medium, Low) based on its aggregated risk score. This classification is made dynamic by using percentile thresholds that adjust according to the data set, ensuring adaptability as new vulnerabilities are discovered.
Conditional Formatting and Interactive Filtering: Conditional formatting is applied to highlight critical resources, and slicers are added to allow users to filter by Entra ID, Subscriptions, and Cloud Environment (for Servers) and Registry Repositories (for Containers).
Score Rank for Risk: The numeric rank of each resource’s risk score is displayed to provide the top-down context on how it compares to others, allowing for a relative understanding of each server or container’s risk level.
Example of Combined Risk Score Calculation
For a given server:
CVE Severity: High (30 points)
Exploit Information: Remote Exploit (10 points) + Exploit Kit Exists (15 points) + Public Exploit (25 points)
Resource Risk Level: High Risk (30 points)
Attack Path Weight: 4 attack vectors (20 points)
CVE Age: More than 180 days (1.5 multiplier)
Total Score: (30 (CVE Severity) + 10 (Remote Exploit) + 15 (Exploit Kit) + 25 (Public Exploit) + 30 (Resource Risk) + 20 (Attack Path)) * 1.5 (CVE Age multiplier) = 195 points
Conclusion
This Power BI solution is built for server and container risk score analysis leverages both standardized vulnerability scoring and contextual risk factors to provide a holistic view of risk across cloud resources. By integrating CVE severity, exploitability, resource risk context, and CVE age, security teams can effectively prioritize their remediation efforts and focus on the most critical vulnerabilities first.
The combination of detailed numerical risk scores, percentile rankings, and user-friendly categorizations ensures that the information presented is both actionable and comprehensible, empowering security teams to enhance their cloud security posture efficiently.
Microsoft Defender for Cloud Additional Resources
Download the new Microsoft CNAPP eBook at aka.ms/MSCNAPP
Become a Defender for Cloud Ninja by taking the assessment at aka.ms/MDCNinja
Reviewers
Yuri Diogenes, Principal PM Manager, CxE Defender for Cloud
Microsoft Tech Community – Latest Blogs –Read More
Building Success Together: Driving Customer Satisfaction and Growth
Imagine you’ve just onboarded a new cybersecurity partner. After months of careful selection, a significant portion of your security budget, and a considerable investment, you’re confident that this partnership will shield your organization from the relentless and ever-evolving threats that jeopardize your environments and productivity.
Imagine you’ve just onboarded a new cybersecurity partner. After months of careful selection, a significant portion of your security budget, and a considerable investment, you’re confident that this partnership will shield your organization from the relentless and ever-evolving threats that jeopardize your environments and productivity. Read More
“Modified Date” and file changes
Does the “Modified Date” in OneDrive reflect any changes to a file, or are there ways a file could be modified without the date changing? I’d like to know if my photos are originals from the modified date listed.
Does the “Modified Date” in OneDrive reflect any changes to a file, or are there ways a file could be modified without the date changing? I’d like to know if my photos are originals from the modified date listed. Read More
FastTrack in a Classic site collection in my tenant
I have a very brief question for which I have not been able to get any input or answers so, I am putting it out to anyone that might have some info to offer.
I have a Classic SharePoint FastTrack site collection in my tenant containing Classic Add-ins. May I delete it and the Add-ins along with it without causing problems with the use of FastTrack programs and services?
https://XXXXXX.sharepoint.com/sites/fasttrack
https://XXXXXX.sharepoint.com/sites/apps
Thank you for any insight you can offer.
I have a very brief question for which I have not been able to get any input or answers so, I am putting it out to anyone that might have some info to offer. I have a Classic SharePoint FastTrack site collection in my tenant containing Classic Add-ins. May I delete it and the Add-ins along with it without causing problems with the use of FastTrack programs and services? https://XXXXXX.sharepoint.com/sites/fasttrackhttps://XXXXXX.sharepoint.com/sites/apps Thank you for any insight you can offer. Read More
Supercharge Your AI Chatbot with Interactive Visualizations Using Chart.js
Supercharge Your AI Chatbot with Interactive Visualizations Using Chart.js
Welcome to the next installment in our AI Futures blog series!
I’m Rob Chambers, and in this post, I’ll walk you through a fascinating journey supercharging a customer’s interactive visualizations solution with Chart.js and Azure AI.
Introduction
Recently, a customer contacted us with an interesting challenge: How to generate charts from data using AI? They had initially tried using Azure OpenAI Assistants with Code Interpreter, which can generate static images by building and executing Python code in real time. Although the static image approach works out of the box, the customer wanted a more interactive solution that could create more lively and dynamic charts.
Here’s an example input prompt from the customer:
Display this data using a bar-chart:
Label Brown Pink Yellow Sum
A 339 433 126 898
B 48 421 222 691
Our Approach
To address the need for interactive and dynamic charts, I had a few interactive conversations with GPT-4o, exploring different options. Eventually, we turned to Chart.js – a popular JavaScript library for creating vibrant and responsive charts. Chart.js perfectly matched the customer’s needs, allowing us to create interactive charts seamlessly integrated into web applications.
Once we selected Chart.js, we needed to determine how we could combine its features with our AI chatbot. We knew we could use the Azure AI CLI to generate a base webpage that already supports markdown and code highlighting using markd and highlight.js. We’d just need to extend this sample to include Chart.js. We decided to hook into the markdown rendering process to support a new language, ‘chartjs’, that would contain JSON objects representing the key parameters for Chart.js charts. To make this work, we had to update our OpenAI Assistant to understand how to emit this new markdown language. Here’s a snippet from our .ai/data/assistant.instructions file:
You are a helpful assistant.
One of your special skills is that you can make charts.
You can do that because you now support a new markdown language called ‘chartjs’ which contains a JSON object that represents a Chart.js chart.
Here’s an example of what the markdown for a Chart.js chart looks like:
“`chartjs
{
“type”: “bar”,
…
}
We used the Azure AI CLI to update our OpenAI Assistant with the new instructions, and to create a simple locally runnable HTML/JavaScript web application:
ai chat assistant update –instructions “@assistant.instructions”
ai dev new openai-asst-webpage-with-functions –javascript
Modifying our web application
Now that our OpenAI Assistant is ready to go, we needed to update our base web application to support the new ‘chartjs’ markdown language.
In the AI Futures team, we always try to use AI as much as possible in our development process, either by using GitHub Copilot inside VS Code or by running the Azure AI CLI interactively. This time, we used the AI CLI to read our existing code, make suggestions and changes, and then save the files back to disk. We reviewed the updates along the way using VS Code’s built-in diff viewer, accepting or rejecting changes as needed. You can take a look at a prompt that is a conglomerate of the changes we made in our GitHub repo here.
Here’s how we invoked the AI CLI to read, understand, and modify the web application:
cd openai-asst-webpage-with-functions-js
ai chat –interactive –built-in-functions –system @SYSTEMPROMPT.md –user @USERPROMPT.md
The ‘–built-in-functions’ option is the key to giving the AI CLI and the LLM access to read and write files on disk.
Integrating Chart.js
Here’s what we (the AI CLI and me) did to integrate Chart.js into the web application:
Extending marked to Handle chartjs “language” Blocks
We extended the markd library to handle the new language block. If the code block’s language is ‘chartjs’, we return a new div with a class of ‘chartjs’ where we’ll render the chart using Chart.js:
marked.setOptions({
highlight: function (code, lang) {
if (lang === ‘chartjs’) {
let chart = createNewChartJsChart(code);
return `<div class=”chartjs”>${chart}</div>`;
} let hl = lang === undefined || lang === ” ? hljs.highlightAuto(code).value : hljs.highlight(lang, code).value;
return `<div class=”hljs”>${hl}</div>`;
}
});
Rendering the chart
The createNewChartJsChart function above parses the JSON, generates the required HTML canvas elements, and defers the chart population until the DOM elements are available. Once the DOM elements are ready, the populateNewChartNow function is called to render the chart using Chart.js:
function populateNewChartNow(chart) {
let ctx = document.getElementById(chart.id).getContext(‘2d’);
new Chart(ctx, {
type: chart.config.type,
data: chart.config.data,
options: chart.config.options
});
}
Conclusion and Call to Action
The end result is a dynamic and interactive chatbot that uses Chart.js to generate charts from data, making the customer’s experience more engaging and insightful. The customer can now simply provide data in their prompts and receive interactive visualizations in response.
The journey to integrate Chart.js and enhance our AI chatbot showcases the power of Azure AI, both in the form of innovative and engaging end-user solutions, as well as in the development of such experiences.
Ready to transform your AI applications with dynamic charts?
Check out the following links for more information:
https://github.com/robch/openai-asst-webpage-with-functions-and-chart-js
https://github.com/Azure/azure-ai-cli
Microsoft Tech Community – Latest Blogs –Read More
From Pixels to Intelligence: Introduction to OCR Free Vision RAG using Colpali For Complex Documents
In the rapidly evolving landscape of artificial intelligence, the ability to understand and process complex documents is becoming increasingly vital. Traditional Optical Character Recognition (OCR) systems have served us well in extracting text from images, but they often fall short when it comes to interpreting the intricate visual elements that accompany textual information. In this blog we will use ColPali, a groundbreaking approach that leverages multi-vector retrieval through late interaction mechanisms and Vision Language Models (VLMs) to enhance Retrieval-Augmented Generation (RAG) processes. This blog post will take you on a deep dive into how ColPali is revolutionizing document understanding and retrieval.
In practice, these retrieval pipelines for PDF documents have a huge impact on performance but are non-trivial
The Limitations of Traditional OCR
What is OCR?
Optical Character Recognition (OCR) is a technology that converts different types of documents—such as scanned paper documents, PDFs, or images captured by a digital camera—into editable and searchable data. While OCR has made significant strides in accuracy, it primarily focuses on text extraction, often overlooking the contextual and visual elements present in complex documents.
Challenges with Complex Documents
Complex documents, such as financial reports, legal contracts, and academic papers, often contain:
Tables and Charts: These elements convey critical information that cannot be captured through text alone.
Images and Diagrams: Visual aids play a significant role in understanding the content but are often ignored by traditional OCR systems.
Layout and Formatting: The arrangement of text and visuals can significantly impact meaning, yet OCR typically treats each element in isolation.
Due to these limitations, traditional OCR can lead to incomplete or misleading interpretations of complex documents.
What is ColPali?
ColPali builds upon recent developments in VLMs, which combine the power of Large Language Models (LLMs) with Vision Transformers (ViTs). By inputting image patch embeddings through a language model, ColPali maps visual features into a latent space aligned with textual content. This alignment is crucial for effective retrieval because it ensures that the visual elements of a document contribute meaningfully to the matching process with user queries.
Key Features of ColPali
Integrated Vision Language Models (VLMs):
ColPali utilizes VLMs like PaliGemma to interpret document images effectively. These models are trained on vast datasets that include not just text but also images, diagrams, and layouts.
By understanding the relationship between visual elements and text, ColPali can provide richer insights into complex documents.
Enhanced Contextual Understanding:
Unlike traditional OCR systems that treat text as isolated data points, ColPali analyzes the entire layout of a document.
This means it can recognize how tables relate to surrounding text or how diagrams illustrate key concepts, leading to more accurate interpretations.
Dynamic Retrieval-Augmented Generation (RAG):
ColPali seamlessly integrates into RAG frameworks, allowing for real-time information retrieval based on user queries.
This dynamic approach ensures that responses are not only relevant but also contextually rich, providing users with comprehensive insights.
Beyond improved accuracy, ColPali also offers significant efficiency gains:
Simplified Indexing: By eliminating the need for complex preprocessing steps, ColPali accelerates the document indexing process. Traditional methods can be time-consuming due to the multiple stages involved in parsing and chunking documents.
Low Query Latency: ColPali maintains low latency during querying, a critical requirement for real-time applications. Its end-to-end trainable architecture optimizes the retrieval process, ensuring swift responses to user queries.
Ok now lets implement the same using Azure AI services.
Lets load the libraries.
import torch
from torch.utils.data import DataLoader
from tqdm import tqdm
from transformers import AutoProcessor
from PIL import Image
from io import BytesIO
from colpali_engine.models.paligemma_colbert_architecture import ColPali
from colpali_engine.utils.colpali_processing_utils import process_images, process_queries
from colpali_engine.utils.image_utils import scale_image, get_base64_image
import os
from dotenv import load_dotenv
load_dotenv(‘azure.env’, override=True)
Load the ColPali model
if torch.cuda.is_available():
device = torch.device(“cuda”)
if torch.cuda.is_bf16_supported():
type = torch.bfloat16
elif torch.backends.mps.is_available():
device = torch.device(“mps”)
type = torch.float32
else:
device = torch.device(“cpu”)
type = torch.float32
Lets store the model in the device selected above.
model_name = “vidore/colpali-v1.2”
model = ColPali.from_pretrained(“vidore/colpaligemma-3b-pt-448-base”, torch_dtype=type).eval()
model.load_adapter(model_name)
model = model.eval()
model.to(device)
processor = AutoProcessor.from_pretrained(model_name)
Once we have loaded the model , the first step in the process is get images from PDF.
import requests
from pdf2image import convert_from_path
from pypdf import PdfReader
def download_pdf(url):
response = requests.get(url)
if response.status_code == 200:
return BytesIO(response.content)
else:
raise Exception(f”Failed to download PDF: Status code {response.status_code}”)
def get_pdf_images(pdf_url):
# Download the PDF
pdf_file = download_pdf(pdf_url)
# Save the PDF temporarily to disk (pdf2image requires a file path)
temp_file = “temp.pdf”
with open(temp_file, “wb”) as f:
f.write(pdf_file.read())
reader = PdfReader(temp_file)
page_texts = []
for page_number in range(len(reader.pages)):
page = reader.pages[page_number]
text = page.extract_text()
page_texts.append(text)
images = convert_from_path(temp_file)
assert len(images) == len(page_texts)
return (images, page_texts)
Lets go ahead and download the PDF. Once pdf is downloaded lets fetch the PDF images.
sample_pdfs = [
{
“title”: “Attention Is All You Need”,
“url”: “https://arxiv.org/pdf/1706.03762”
}
]
We once loaded into the PDF images , texts
for pdf in sample_pdfs:
page_images, page_texts = get_pdf_images(pdf[‘url’])
pdf[‘images’] = page_images
pdf[‘texts’] = page_texts
Now lets go ahead create the embedding for each page image.
for pdf in sample_pdfs:
page_embeddings = []
dataloader = DataLoader(
pdf[‘images’],
batch_size=2,
shuffle=False,
collate_fn=lambda x: process_images(processor, x),
)
for batch_doc in tqdm(dataloader):
with torch.no_grad():
batch_doc = {k: v.to(model.device) for k, v in batch_doc.items()}
embeddings = model(**batch_doc)
mean_embedding = torch.mean(embeddings, dim=1).float().cpu().numpy()
#page_embeddings.extend(list(torch.unbind(embeddings.to(“cpu”))))
page_embeddings.extend(mean_embedding)
pdf[’embeddings’] = page_embeddings
ColPali During indexing, we aim to strip away a lot of the complexity by using images (“screenshots”) of the document pages directly.
A Vision LLM (PaliGemma-3B) encodes the image by splitting it into a series of patches, which are fed to a vision transformer.
During runtime querying, a user query is embedded by the language model, to obtain token embeddings. ColBERT-style “late interaction” (LI) operation to efficiently match query tokens to document patches. To compute a LI(query, document) score, for each term in the query, we search for the document patch that has the most similar ColPali representation. We then sum the scores of the most similar patches for all terms of the query, to obtain the final query-document score. Intuitively, this late-interaction operation allows for a rich interaction between all terms of the query and document patches, all the while benefiting from the fast matching and offline computation offloading that more standard (bi-encoder) embedding models enable.
import numpy as np
lst_feed = []
for pdf in sample_pdfs:
url = pdf[‘url’]
title = pdf[‘title’]
for page_number, (page_text, embedding, image) in enumerate(zip(pdf[‘texts’], pdf[’embeddings’], pdf[‘images’])):
base_64_image = get_base64_image(scale_image(image,640),add_url_prefix=False)
page = {
“id”: str(hash(url + str(page_number))),
“url”: url,
“title”: title,
“page_number”: page_number,
“image”: base_64_image,
“text”: page_text,
“embedding”: embedding.tolist()
}
lst_feed.append(page)
Now once we have embedding , we need to store these embedding into a vector Store. We will use AI search as our vector store. Lets create an AI index.
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
SimpleField,
SearchFieldDataType,
SearchableField,
SearchField,
VectorSearch,
HnswAlgorithmConfiguration,
VectorSearchProfile,
SemanticConfiguration,
SemanticPrioritizedFields,
SemanticField,
SemanticSearch,
SearchIndex
)
def create_pdf_search_index(endpoint: str, key: str, index_name: str) -> SearchIndex:
# Initialize the search index client
credential = AzureKeyCredential(key)
index_client = SearchIndexClient(endpoint=endpoint, credential=credential)
# Define vector search configuration
vector_search = VectorSearch(
algorithms=[
HnswAlgorithmConfiguration(
name=”myHnsw”,
parameters={
“m”: 4, # Default HNSW parameter
“efConstruction”: 400, # Default HNSW parameter
“metric”: “cosine”
}
)
],
profiles=[
VectorSearchProfile(
name=”myHnswProfile”,
algorithm_configuration_name=”myHnsw”,
vectorizer=”myVectorizer”
)
]
)
# Define the fields
fields = [
SimpleField(
name=”id”,
type=SearchFieldDataType.String,
key=True,
filterable=True
),
SimpleField(
name=”url”,
type=SearchFieldDataType.String,
filterable=True
),
SearchableField(
name=”title”,
type=SearchFieldDataType.String,
searchable=True,
retrievable=True
),
SimpleField(
name=”page_number”,
type=SearchFieldDataType.Int32,
filterable=True,
sortable=True
),
SimpleField(
name=”image”,
type=SearchFieldDataType.String,
retrievable=True
),
SearchableField(
name=”text”,
type=SearchFieldDataType.String,
searchable=True,
retrievable=True
),
SearchField(
name=”embedding”,
type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
searchable=True,
vector_search_dimensions=128,
vector_search_profile_name=”myHnswProfile”
)
]
# Create the index definition
index = SearchIndex(
name=index_name,
fields=fields,
vector_search=vector_search
)
# Create the index in Azure Cognitive Search
result = index_client.create_or_update_index(index)
return result
Once the index is created, we should upload the documents.
from azure.search.documents import SearchClient
credential = AzureKeyCredential(SEARCH_KEY)
index_client = SearchClient(endpoint=SEARCH_ENDPOINT, credential=credential, index_name = INDEX_NAME)
index_client.upload_documents(documents=lst_feed)
Once the document ingestion is completed , next is handling user query. As you can see in the code we create embedding for input query.
def process_query(query: str, processor: AutoProcessor, model: ColPali) -> np.ndarray:
mock_image = Image.new(‘RGB’, (224, 224), color=’white’)
inputs = processor(text=query, images=mock_image, return_tensors=”pt”)
inputs = {k: v.to(model.device) for k, v in inputs.items()}
with torch.no_grad():
embeddings = model(**inputs)
return torch.mean(embeddings, dim=1).float().cpu().numpy().tolist()[0]
Now lets create a search client.
from IPython.display import display, HTML
from openai import AzureOpenAI
client = AzureOpenAI(api_key=os.environ[‘AZURE_OPENAI_API_KEY’],
azure_endpoint=os.environ[‘AZURE_OPENAI_ENDPOINT’],
api_version=os.environ[‘OPENAI_API_VERSION’])
search_client = SearchClient(
SEARCH_ENDPOINT,
index_name=INDEX_NAME,
credential=credential,
)
def display_query_results(query, response, hits=5):
html_content = f”<h3>Query text: ‘{query}’, top results:</h3>”
for i, hit in enumerate(response):
title = hit[“title”]
url = hit[“url”]
page = hit[“page_number”]
image = hit[“image”]
score = hit[“@search.score”]
html_content += f”<h4>PDF Result {i + 1}</h4>”
html_content += f'<p><strong>Title:</strong> <a href=”{url}”>{title}</a>, page {page+1} with score {score:.2f}</p>’
html_content += (
f'<img src=”data:image/png;base64,{image}” style=”max-width:100%;”>’
)
display(HTML(html_content))
Now once we have retrieved the relevant images , we can use any VLM to pass in these images to ask question to user.
query = “What is the projected global energy related co2 emission in 2030?”
vector_query = VectorizedQuery(
vector=process_query(query, processor, model),
k_nearest_neighbors=3,
fields=”embedding”,
)
results = search_client.search(search_text=None, vector_queries=[vector_query])
#display_query_results(query, results)
response = client.chat.completions.create(
model=”gpt-4o”,
messages=[
{
“role”: “system”,
“content”: “””You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don’t know the answer, just say that you don’t know. Use three sentences maximum and keep the answer concise.You will be given a mixed of text, tables, and image(s) usually of charts or graphs.”””
},
{
“role”: “user”,
“content”: [
{“type”: “text”, “text”: query},
*map(lambda x: {“type”: “image_url”, “image_url”: {“url”: f’data:image/jpg;base64,{x[“image”]}’, “detail”: “low”}}, results),
],
}
],
max_tokens=300,
)
print(“Answer:” + response.choices[0].message.content)
Conclusion
As we move further into an era where data is increasingly complex and multifaceted, tools like Multi Modal LLM are essential for unlocking valuable insights from our documents. By integrating advanced Vision Language Models with Retrieval-Augmented Generation techniques, this sets a new standard for document understanding that transcends traditional OCR limitations. Whether you’re a researcher looking to streamline your workflow or a developer interested in AI advancements, embracing technologies like VLM and ColPali will undoubtedly enhance your ability to navigate complex information landscapes. Stay tuned for more updates as we continue to explore the fascinating intersection of AI and document processing!
** Do check the license of the opensource models before using the same
Learn more about AI Search: https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search
Model: https://huggingface.co/vidore/colpali-v1.2
Thanks
Manoranjan Rajguru
https://www.linkedin.com/in/manoranjan-rajguru/
Microsoft Tech Community – Latest Blogs –Read More
New on Azure Marketplace: October 1-9, 2024
We continue to expand the Azure Marketplace ecosystem. For this volume, 217 new offers successfully met the onboarding criteria and went live. See details of the new offers below:
Get it now in our marketplace
AccuFlow.ai: AccuFlow.ai simplifies AI application development for businesses by creating knowledge bases, conducting tests, setting up prompts, and analyzing data. Its customizable AI assistants can align with your corporate image. This offer is available only in Chinese.
Advanced Hardened Image for Windows Server 2019 – Level 2: This offer from Madarson IT provides an Azure-based virtual machine with Windows Server 2019. It’s optimized for security, with advanced hardening measures and stringent access controls. The image aligns with many standards, including NIST CSF, ISO 27000, PCI DSS, and HIPAA.
AlmaLinux 8.3: This offer from Bansir Cloud provides AlmaLinux 8.3, an enterprise Linux distribution that’s compatible with CentOS and Red Hat Enterprise Linux. Ideal for mission-critical workloads in datacenters and cloud environments, it offers an extensive software repository and active community support.
AlmaLinux 8.4: This offer from Bansir Cloud provides AlmaLinux 8.4, an enterprise Linux distribution that’s compatible with CentOS and Red Hat Enterprise Linux. Ideal for mission-critical workloads in datacenters and cloud environments, it offers an extensive software repository and active community support.
AlmaLinux 8.6: This offer from Bansir Cloud provides AlmaLinux 8.6, an enterprise Linux distribution that’s compatible with CentOS and Red Hat Enterprise Linux. Ideal for mission-critical workloads in datacenters and cloud environments, it offers an extensive software repository and active community support.
AlmaLinux 8.7: This offer from Bansir Cloud provides AlmaLinux 8.7, an enterprise Linux distribution that’s compatible with CentOS and Red Hat Enterprise Linux. Ideal for mission-critical workloads in datacenters and cloud environments, it offers an extensive software repository and active community support.
AShield Digital Authentication and Fraud Management: AShield delivers phishing-proof authentication and bot management using quantum-immune cryptography. AShield ensures user authenticity without interference and reduces false positives and false negatives in fraud detection.
Autoscriber Standalone: Autoscriber B.V. offers AI software that automatically captures and converts doctor-patient conversations into structured clinical notes, including medical codes. This reduces administrative time, improves data quality, and can lead to better patient outcomes. The software is GDPR-compliant and certified for data security.
AVA+ GenAI Core Azure Application: Ascendion’s AVA+ GenAI Core platform centralizes generative AI management for enterprises. It supports foundational models, prompt crafting, RAG setups, and complex workflows with robust safety features. Seamlessly integrate and scale AI initiatives within your Azure environment, utilizing unlimited prompts, knowledge bases, use cases, pipelines, and agents.
azShareExplorer: ROMAWO GmbH’s platform offers secure, direct access to Microsoft Azure file share data. Users can browse directories, view file structures, and securely download files via an intuitive web interface. Storage credentials are encrypted, ensuring only authorized access. It’s ideal for businesses seeking streamlined cloud file management.
BeautifulSoup4 v4.12.3 on Ubuntu v20: This offer from Anarion Technologies provides BeautifulSoup4 along with Ubuntu on a Microsoft Azure virtual machine. BeautifulSoup4 is a Python library for parsing and navigating HTML or XML documents. It simplifies web scraping by converting web pages into parse trees, allowing easy data extraction.
BissantzService: Business Intelligence Platform: DeltaApp by Bissantz enables cloud-based business intelligence applications for data navigation, variance analysis, and key performance indicator comparison on browsers and smartphones. It includes an administration interface for managing users, roles, and portals. Offline data upload and automatic data provision via DeltaMaster Publisher are supported.
Board – The Enterprise Planning Platform: The Board Enterprise Planning Platform enables more than 2,000 organizations to enhance financial and operational planning using AI and analytics. Trusted by global leaders like H&M and Coca-Cola, it offers cost-effective, end-to-end solutions integrated with Microsoft Azure.
Browser-Based Ubuntu 22.04 GUI Desktop Accessible Via HTTPS: This offer from TechLatest lets you experience Ubuntu 22.04 GUI Desktop in a secure, browser-based virtual environment via HTTPS. Ideal for development, testing, education, and remote work, this virtual machine offers seamless access without local installations. Key benefits include secure access, embeddable iFrame, and flexibility across devices.
Browser-Based Ubuntu 24.04 GUI Desktop Accessible Via HTTPS: This offer from TechLatest lets you experience Ubuntu 24.04 GUI Desktop in a secure, browser-based virtual environment via HTTPS. Ideal for development, testing, education, and remote work, this virtual machine offers seamless access without local installations. Key benefits include secure access, embeddable iFrame, and flexibility across devices.
Call Center AI: Call Center AI revolutionizes customer support by analyzing call recordings to provide insights on customer sentiment, agent performance, and operational trends. It aids in improving agent training, understanding customer feedback, and streamlining operations, making it essential for unlocking actionable insights.
Callsign Platform: Callsign offers seamless and secure digital identity solutions that protect against account takeovers and online scams. Using AI and fraud models, Callsign’s platform creates a digital DNA profile for each user. Features include behavior biometrics, device fingerprinting, and multifactor authentication. Reduce fraud, save money, and improve your user experience with Callsign.
Code Llama 7B on Ubuntu v20: This offer from Anarion Technologies provides Code Llama 7B along with Ubuntu on a Microsoft Azure virtual machine. Code Llama 7B is a cutting-edge language model designed for code generation and comprehension. It offers intelligent code completion, syntax highlighting, and error detection, enhancing coding efficiency and quality.
Cognite Data Fusion – NL: Cognite is a global industrial SaaS company that empowers asset-heavy industries with contextualized, real-time data through its platform, Cognite Data Fusion. Leveraging AI, big data, and 3D modeling, it enhances decision-making in maintenance, production, and safety, driving digital transformation and operational efficiency across sectors like manufacturing, oil and gas, and renewable energy.
CogVoice Agentic Network IVR: Norwood Systems’ AI-powered CogVoice solutions enhance mobile and fixed-line operator services with advanced Agentic IVR technology. Key features include natural, context-aware conversations; intelligent memory for continuity; real-time interruptible interactions; and multi-lingual support. These innovations improve subscriber engagement, acquisition, and retention while elevating the customer experience.
CogVoice Call Screener for Operators: Norwood Systems’ AI-powered CogVoice solutions enhance mobile operators’ subscriber engagement, acquisition, and retention. Flexible delivery options integrate seamlessly into existing networks, offering scalable AI-powered voice service orchestration through the Open Media Services Gateway (OMSG).
Debian 12 Minimal with NGINX: This offer from Virtual Pulse S. R. O. provides a minimal installation of Debian 12 along with NGINX on a Microsoft Azure virtual machine. NGINX handles static and dynamic content, scales easily, and integrates with Apache and other platforms.
Debian 12 Minimal with OpenVPN: This offer from Virtual Pulse S. R. O. provides a minimal installation of Debian 12 along with OpenVPN on a Microsoft Azure virtual machine. OpenVPN is a lightweight, efficient operating system designed for secure networking. It combines Debian’s stability with OpenVPN’s encrypted connections, ensuring privacy and resource optimization.
Debian 12 Minimal with Pure-FTPd: This offer from Virtual Pulse S. R. O. provides a minimal installation of Debian 12 along with Pure-FTPd on a Microsoft Azure virtual machine. This user-friendly FTP server solution is ideal for beginners in server administration. It offers secure data transfer, customizable specifications, and easy user management.
Debian 12 Minimal with Rocket.Chat: This offer from Virtual Pulse S. R. O. provides a minimal installation of Debian 12 along with Rocket.Chat on a Microsoft Azure virtual machine. Rocket.Chat is an enterprise messaging platform that supports real-time chat, group discussions, file sharing, and voice/video calls. It integrates with GitHub and Trello.
Debian 12 Minimal with SoftEther VPN Server: This offer from Virtual Pulse S. R. O. provides a minimal installation of Debian 12 along with SoftEther VPN on a Microsoft Azure virtual machine. SoftEther VPN offers advanced VPN technology with high security, fast performance, and versatility. It supports multiple protocols and is open-source, ensuring constant improvements.
Debian 12 with NGINX: This offer from Virtual Pulse S. R. O. provides Debian 12 along with NGINX on a Microsoft Azure virtual machine. NGINX offers enhanced reliability and versatility for web hosting. With robust security and easy installation, it’s effective for both small blogs and large-scale platforms.
Debian 12 with OpenVPN: This offer from Virtual Pulse S. R. O. provides Debian 12 along with OpenVPN on a Microsoft Azure virtual machine. OpenVPN offers robust digital security through encrypted tunnels using SSL/TLS and AES-256 encryption. OpenVPN’s open-source nature ensures transparency and reliability.
Debian 12 with Pure-FTPd: This offer from Virtual Pulse S. R. O. provides Debian 12 along with Pure-FTPd on a Microsoft Azure virtual machine. Pure-FTPd offers robust security and efficiency for FTP servers. It features SSL/TLS support, privilege separation, and attack protection.
Debian 12 with Rocket.Chat: This offer from Virtual Pulse S. R. O. provides Debian 12 along with Rocket.Chat on a Microsoft Azure virtual machine. Rocket.Chat enhances teamwork with secure, instant communication, customizable themes, and extensive integrations.
Debian 12 with SoftEther VPN: This offer from Virtual Pulse S. R. O. provides Debian 12 along with SoftEther VPN on a Microsoft Azure virtual machine. SoftEther VPN delivers exceptional network performance, high security with AES 256-bit and RSA 4096-bit encryption, and extensive features like SSL-VPN, HTTPS tunneling, and support for various VPN protocols.
DisplayPro: Real-Time Displays with Power BI: DisplayPro from L Squared is a digital signage solution for manufacturing and IT data visualization. It includes a 44-inch TV, a SaaS platform license, professional installation, and training. DisplayPro seamlessly integrates with Microsoft Power BI to display reports on TV screens.
docuAI by Abstracta: Abstracta’s docuAI automates data extraction from financial documents, reducing manual labor and errors. It supports various financial institutions and integrates with Xero and Memory Conty. Users simply upload documents and docuAI converts them to spreadsheets.
EGroupware, Secured and Supported by Hossted: Hossted offers a streamlined deployment of EGroupware with enhanced security, a control dashboard, and continuous security scans. EGroupware is a comprehensive collaboration platform designed to streamline project management, document sharing, and team communication.
EncompaaS Intelligent Information Management Version 2: EncompaaS empowers highly regulated enterprises to mitigate compliance and privacy risks using AI to manage structured and unstructured data. It automates governance, ensuring data quality and regulatory adherence. Trusted by Fortune 500 companies, EncompaaS optimizes information across on-premises and multi-cloud environments, enabling real-time risk management.
EncompaaS Search: EncompaaS Search is a module of EncompaaS, designed for enterprise data discovery. It offers fast, accurate, and secure access to data across various sources and formats. Utilizing AI and natural language processing, it enhances data quality and delivers relevant results while respecting data governance and privacy policies.
erwin Data Intelligence by Quest: erwin Data Intelligence by Quest integrates data catalog, quality, literacy, and marketplace features to manage and share trusted data and AI models. It offers automation, multi-platform visibility, and support for Microsoft platforms. With more than 120 data connectors, it ensures reliable data pipelines and AI readiness.
GenAI Scan & OCR (1 Million Scans): GenAI Scan & OCR from Sahara Watira is an AI-powered tool that enhances document scanning, recognition, and data extraction using generative AI and optical character recognition technologies. It digitizes physical documents by converting images, PDFs, or scanned files into editable and searchable text formats, streamlining document management.
GenAI Scan & OCR (500,000 Scans): GenAI Scan & OCR from Sahara Watira is an AI-powered tool that enhances document scanning, recognition, and data extraction using generative AI and optical character recognition technologies. It digitizes physical documents by converting images, PDFs, or scanned files into editable and searchable text formats, streamlining document management.
Geneyx Analysis: Geneyx Analysis is a next-generation sequencing solution for hospitals and labs that integrates AI and machine learning to enhance diagnostic yields and improve turnaround times. It offers customizable protocols for gene panels, exomes, and genomes, supporting all genetic variants. The platform automates diagnostics and simplifies bioinformatics pipeline regulation.
Gitea Server: This offer from Cloud Infrastructure Services provides Gitea, a lightweight, self-hosted Git service for managing repositories, project collaboration, and code review. Gitea has a web-based interface and integrates with CI/CD tools. Running on Microsoft Azure, Gitea ensures secure, scalable hosting. Features include issue tracking, user access control, and two-factor authentication.
Hardened AlmaLinux 8 Image – Level 1: Foundational: This Azure-based virtual machine with AlmaLinux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened AlmaLinux 8 Image – Level 2: Advanced: This Azure-based virtual machine with AlmaLinux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are up-to-date, secure, and compliant with industry best practices.
Hardened AlmaLinux 9 Image – Level 1: Foundational: This Azure-based virtual machine with AlmaLinux 9 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened AlmaLinux 9 Image – Level 2: Advanced: This Azure-based virtual machine with AlmaLinux 9 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Oracle Linux 8 Image – Level 1: Foundational: This Azure-based virtual machine with Oracle Linux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Oracle Linux 8 Image – Level 2: Advanced: This Azure-based virtual machine with Oracle Linux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Rocky Linux 8 Image – Level 1: Foundational: This Azure-based virtual machine with Rocky Linux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Rocky Linux 8 Image – Level 2: Advanced: This Azure-based virtual machine with Rocky Linux 8 is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Ubuntu 22.04 LTS Image – Level 1: Foundational: This Azure-based virtual machine with Ubuntu 22.04 LTS is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Ubuntu 22.04 LTS Image – Level 2: Advanced: This Azure-based virtual machine with Ubuntu 22.04 LTS is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Ubuntu 24.04 LTS Image – Level 1: Foundational: This Azure-based virtual machine with Ubuntu 24.04 LTS is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Hardened Ubuntu 24.04 LTS Image – Level 2: Advanced: This Azure-based virtual machine with Ubuntu 24.04 LTS is optimized for security, featuring advanced hardening measures, stringent access controls, and regular updates. It aligns with standards like NIST CSF, ISO 27000, PCI DSS, and HIPAA. Madarson IT ensures these images are secure, up-to-date, and compliant with industry best practices.
Holobox as a Service: Verofax’s Holobox uses augmented reality and AI to create hyperrealistic avatars for enhanced customer engagement in retail, tourism, and hospitality. Boost brand loyalty with personalized recommendations and innovative, interactive experiences.
IBM Event Automation: IBM Event Automation leverages Apache Kafka and Flink to manage, process, and analyze real-time business events. It offers capabilities for event streams, endpoint management, and processing, enabling businesses to detect trends and issues.
Idenprotect Passport: Idenprotect Passport offers password-free authentication using biometrics, enhancing security and mitigating phishing risks. Idenprotect Passport can simplify compliance, improves productivity, and reduce IT costs.
Imperium Hourly (SaaS): Imperium Hourly is a timesheet management app designed for easy hour reporting, especially for multi-project employees. It allows managers to adjust, approve, and provide feedback on timesheets. Features include real-time analytics, mobile compatibility, and integration with Imperium365 Starter CRM, enhancing business management from lead nurturing to customer billing.
inFlow: inFlow is a SaaS solution leveraging generative AI to enhance customer engagement across WhatsApp, Facebook, Instagram, and 40+ channels. It offers industry-specific solutions, boosts conversions, and provides instant scalability. The platform automates customer interactions, appointments, orders, and inquiries, thereby lowering acquisition costs and optimizing efficiency.
KeeWeb, Secured and Supported by Hossted: Hossted offers a streamlined deployment of KeeWeb with enhanced security, a control dashboard, and continuous security scans. KeeWeb is an open-source password manager that enables users to securely store, manage, and access their passwords across multiple devices.
MongoDB on Oracle 9: This offer from Tidal Media provides MongoDB along with Oracle 9 on a Microsoft Azure virtual machine. MongoDB is a NoSQL database offering flexibility, scalability, and performance for big-data applications. It supports various data models, automatic sharding, and indexing, ensuring high availability and fast query execution.
MongoDB on Red Hat 9: This offer from Tidal Media provides MongoDB along with Red Hat 9 on a Microsoft Azure virtual machine. MongoDB is an open-source, document-oriented database designed for efficient data management at any scale.
OAuthlib v3.2.2 on Ubuntu v20: This offer from Anarion Technologies provides OAuthlib along with Ubuntu on a Microsoft Azure virtual machine. OAuthlib is a Python library for implementing OAuth 1.0 and 2.0 protocols, simplifying secure authentication and authorization for applications.
Oracle 9 with iPerf3: This offer from Virtual Pulse S. R. O. provides Oracle 9 along with iPerf3 on a Microsoft Azure virtual machine. iPerf is a powerful tool for optimizing network performance. It supports TCP, UDP, and SCTP protocols with IPv4 and IPv6, offering customizable parameters and dynamic server capabilities.
Oracle Linux 7: This offer from Bansir Cloud provides Oracle Linux 7 on a Microsoft Azure virtual machine. Oracle Linux is an enterprise-grade Linux distribution offering high performance, scalability, and security. Fully compatible with Red Hat Enterprise Linux, it supports mission-critical applications and is cost-effective for businesses.
Oracle Linux 7.8: This offer from Bansir Cloud provides Oracle Linux 7.8 on a Microsoft Azure virtual machine. Oracle Linux is an enterprise-grade Linux distribution offering high performance, scalability, and security. Fully compatible with Red Hat Enterprise Linux, it supports mission-critical applications and is cost-effective for businesses.
Pachito Digital Assistant: Boost team efficiency with Pachito, a generative AI-powered digital assistant from Information Workers. Pachito integrates with Microsoft Teams to enhance communication, personalize experiences, and streamline workflows. The assistant automates routine tasks and scales with your organization.
PaddlePaddle v2.6.2 on Ubuntu v20: This offer from Anarion Technologies provides PaddlePaddle along with Ubuntu on a Microsoft Azure virtual machine. PaddlePaddle is an open-source deep learning platform that supports neural network architectures and offers tools for data preprocessing, model training, and evaluation. Known for its scalability across GPUs and distributed systems, it includes pretrained models and user-friendly APIs.
Pebble Flow Workflow Server: The Pebble Stream Workflow Server converts Microsoft Excel spreadsheet models into scalable back-end processes accessible via secure web APIs. Business analysts can run multiple models simultaneously, while developers can access these models via APIs or embed them in custom applications.
Prometheus on Kubernetes, Secured and Supported by Hossted: Deploy Prometheus on Azure Kubernetes Service with this offer from Hossted. Prometheus is a monitoring and alerting toolkit that provides metrics collection and querying for apps. Hossted’s offer involves repackaged open-source software with charges for supplementary services, including troubleshooting and premium support.
Purchase Plant-a-Tree Service: NetGreen: NetGreen, powered by Verofax, enables consumers to convert loyalty points from retail purchases into tree-planting programs, then share their positive impact on social media. Thusly, consumers and brands can effectively work together to scale up reforestation projects.
Rocky 8.10 HPC-ai for Ansys by CIQ: Experience enhanced engineering simulations with Rocky Linux and Ansys Axis from CIQ. This solution offers optimized performance, scalable infrastructure, enterprise-grade security, and cost efficiency. Designed for engineering, R&D, and educational use, it includes expert support to maximize simulation capabilities.
RuckZuck: Private RuckZuck Core infrastructure as a managed application allows customers to operate independently of the public RuckZuck service, eliminating rate limits for automated tasks. Optional features include caching package content files with static URLs and enabling a CDN for improved performance.
SapienceS2P Satori Bot for Business: The Satori bot by SapienceS2P is an AI-driven chatbot for procurement, integrating with SAP Ariba and S/4HANA. It automates tasks like supplier onboarding and contract management using natural language processing and machine learning.
SATT TRVST: Vaccine Track and Trace: SATT TRVST is a cloud-based vaccine traceability solution by SoftGroup that offers real-time data monitoring, advanced security, and a user-friendly interface. SATT TRVST enables batch data upload and publishing of serialization reports following the latest published regulations and technical specifications set by regulatory authorities.
Simcorp SaaS Canada: Simcorp SaaS is a comprehensive application lifecycle management service delivered with a continuous development road map to enhance your organization’s resilience and efficiency. The platform offers a simplified, scalable, secure operating model with end-to-end ownership.
Targetty Statement: Targetty Statement is an all-in-one financial reporting tool that integrates with Microsoft Power BI and aligns with International Financial Reporting Standards (IFRS) and the United States’ Generally Accepted Accounting Principles standard. Targetty Statement’s intuitive interface allows users to define and generate reports without the need for programming or formula knowledge.
Temporal, Secured and Supported by Hossted: Hossted offers a repackaged Temporal deployment with enhanced security, instant setup, and a control dashboard. Temporal is a workflow orchestration engine that simplifies building and managing scalable, reliable, and long-running processes on Microsoft Azure.
TEPlay – Video Streaming Platform: TEPlay by Tata Elxsi is a scalable, secure video delivery platform with preintegration for faster market entry. It supports multi-device streaming, personalized content recommendations, and detailed analytics. Features include automated workflows, a unified back end, and various monetization options.
TerraProvider – Terraform Provider for Microsoft 365: TerraProvider from glueckkanja AG automates Microsoft 365 tenant configurations using Terraform, ensuring consistency, compliance, and disaster recovery. TerraProvider manages settings as version-controlled code, using Microsoft Graph API for updates. This tool simplifies multi-tenant management, conditional access, and change management.
Text (SMS) Messaging for Dynamics 365 Field Service (SaaS): Imperium Dynamics’ Text Messaging for Microsoft Dynamics 365 Field Service enhances field operations with real-time SMS/MMS communication via Power Textor and Microsoft Power Automate. The solution provides automated updates, direct and bulk messaging, case timelines, and compliance with the A2P 10 DLC messaging standard.
Theano v1.0.5 on Ubuntu v20: This offer from Anarion Technologies provides Theano along with Ubuntu on a Microsoft Azure virtual machine. Theano is an open-source Python library for efficient numerical computation, particularly with multi-dimensional arrays (tensors). It supports CPU and GPU processing, crucial for deep learning.
TiDB with Grafana & Prometheus: Real-Time Data & Analytics: This offer from TechLatest delivers a virtual machine with TiDB, a distributed SQL database that supports hybrid transactional and analytical processing (HTAP) workloads. E-commerce, finance, IoT, gaming, and media applications can use it for horizontal and vertical scalability, ACID compliance, and advanced monitoring with Grafana and Prometheus.
Unleash: The Unleash platform from Unleash Technologies uses advanced large language models like GPT and integrates with 70+ SaaS tools to provide employees with instant, data-driven answers to work-related questions. Accessible on the web, on Slack, on Microsoft Teams, on Zendesk, on Salesforce, or on a desktop machine, it delivers precise answers, saving time and effort.
WinGLUE: WinGLUE is comprehensive gas-lift well surveillance and optimization software by Shell and AppSmiths. WinGLUE integrates various well data types for efficient gas-lift diagnostics and optimization. It also features an API for project data visualization and comparison.
WordPress: This offer from Cloud Infrastructure Services provides WordPress along with Ubuntu 22.04 on a Microsoft Azure virtual machine. The preconfigured setup also includes Apache, MariaDB, Let’s Encrypt, Pro FTPd, and Webmin.
WordPress on Ubuntu 24.04: This offer from Cloud Infrastructure Services provides WordPress along with Ubuntu 24.04 on a Microsoft Azure virtual machine. The preconfigured setup also includes Apache, MariaDB, Let’s Encrypt, Pro FTPd, and Webmin.
Writesonic: Writesonic is an AI-powered content marketing platform. Key features include AI Article Writer, a tool for producing pieces up to 5,000 words long; Chatsonic, a powerful ChatGPT alternative with artifacts, AI agents, and real-time web search capabilities; and Photosonic and Audiosonic, tools for AI-generated images and human-like voice-overs.
Xillio Content Migration Solution for Microsoft 365: Xillio specializes in efficient, hassle-free content migrations for various industries, supporting more than 60 enterprise content management systems. Clients praise Xillio’s expertise, flexibility, and timely project completion.
Zabbix Proxy 7.0 (MySQL): Zabbix is an open-source, enterprise-class monitoring solution for IT services, cloud, and IoT. It offers automated monitoring, smart thresholds, and machine learning. With robust performance, it processes up to 100,000 metrics per second.
Zabbix Proxy 7.0 (PostgreSQL): Zabbix is an open-source, enterprise-class monitoring solution for IT services, cloud, and IoT. It offers automated monitoring, smart thresholds, and machine learning. With robust performance, it processes up to 100,000 metrics per second.
Zabbix Proxy 7.0 (SQLite): Zabbix is an open-source, enterprise-class monitoring solution for IT services, cloud, and IoT. It offers automated monitoring, smart thresholds, and machine learning. With low TCO and extensive integrations, it efficiently processes up to 100,000 metrics per second.
Zabbix Server 7.0 (MySQL): Zabbix is an open-source, enterprise-class monitoring solution for IT services, cloud, and IoT. It offers automated monitoring, smart thresholds, and machine learning. With robust performance, it processes up to 100,000 metrics per second.
Zabbix Server 7.0 (PostgreSQL): Zabbix is an open-source, enterprise-class monitoring solution for IT services, cloud, and IoT. It offers automated monitoring, smart thresholds, and machine learning. With robust performance, it processes up to 100,000 metrics per second.
Go further with workshops, proofs of concept, and implementations
Analytics for Efficient Trade Term Management: In this service targeting consumer packaged goods companies, Sigmoid will implement a solution using Microsoft Azure tools to address challenges like data silos and inefficient decision-making. Sigmoid will analyze trade investment performance, channel growth, and more, utilizing Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, and Microsoft Power BI.
Atea Copilot for Security: Atea will implement Copilot for Security, a comprehensive AI-driven security management solution. Atea will ensure that your organization is well-equipped to manage and optimize security infrastructure and all aspects where the AI platform can bring benefits.
Azure OpenAI Custom Development: SHI can help your business safely and effectively adopt generative AI. SHI’s experts will assist in aligning AI solutions with your business objectives, defining infrastructure needs, developing capabilities in Azure OpenAI, and supporting deployed solutions.
Azure Stack HCI Deployment and Migration Services: SHI offers comprehensive services for a seamless transition to Azure Stack HCI: unboxing, racking, advanced engineering, integration, configuration, workload migrations, and testing. Enhance your IT scalability and flexibility with SHI’s expert support.
Beyondsoft Azure VMware Solution: Beyondsoft offers Azure infrastructure design and implementation for Azure VMware Solution, setting up a landing zone tailored to customer needs. Beyondsoft will provide cost control, governance, and tools to evaluate application dependencies and architecture.
Boltrics Data Connector: In this service, 2foqus will extract and integrate comprehensive data from Boltrics to Azure using Python scripting and the Boltrics API. Designed for logistics companies using Boltrics ERP, WMS, and TMS software, the service provides full data access for enhanced analytics and decision-making.
Cognizant’s Quick Start Offering for Microsoft Purview: Cognizant offers a rapid, industry-specific data governance assessment and proof of concept using Microsoft Purview for Fortune 500 companies. The four-week program includes industry requirements analysis, data governance framework setup, a Microsoft Purview proof of concept, and business glossary creation.
Copilot for Connected Products and Machines: Cluster Reply’s 12-week proof of concept will accelerate the implementation of complex generative AI architectures. The solution accelerator will provide a unified data layer, a copilot core, and conversational interfaces, enhancing decision-making with data-driven insights.
CT Operational Technology Data Liberator: Celebal Technologies will implement its OT Data Liberator to efficiently extract, process, and integrate operational technology data into a centralized lakehouse. This will ensure metadata synchronization, real-time data streaming, historical data retrieval, and robust connectivity to legacy and modern systems.
Data Governance: Softchoice’s data governance services provide a tailored framework to ensure data integrity and security across on-premises, multi-cloud, or hybrid environments. Services include data readiness assessments, scalable data foundations, and governance solutions featuring Microsoft Purview.
Data Platform to Fabric Migration: Cognizant’s migration service helps organizations transition from legacy platforms to Microsoft Fabric, enhancing performance, scalability, and security. Such a migration optimizes resource allocation, centralizes data management, and reduces migration costs by up to 60% using automation.
Data Services: 6-Week Implementation: Softchoice offers comprehensive services for managing, optimizing, and deriving insights from data assets, focusing on Microsoft environments. Softchoice provides data readiness assessments, governance, estate management, and analytics services to enhance decision-making, drive innovation, improve security, and leverage AI.
Datacenter Cloud Migration | Azure Adoption Workshop: In this workshop, Cluster Reply will plan your cloud migration. This will involve an initial call, an infrastructure inventory, and an assessment results discussion. Cluster Reply will determine your current infrastructure, evaluate your adoption readiness, and draft a high-level cloud adoption road map.
Easyflex Data Platform: In this service for companies in the staffing industry, 2foqus will extract and integrate data from Easyflex to Azure using Python scripting and the Easyflex API. This will enable detailed data access for deeper analysis, improving operational efficiency and decision-making. This offer is available only in Dutch.
Fabric Enablement: Microsoft Fabric is a unified analytics platform requiring strategic implementation and governance. SHI will help customers make a smooth transition to Microsoft Fabric by assessing data culture, creating governance frameworks, and providing training and support.
GitHub Implementation: SGA’s migration service for GitHub will ensure a smooth transition for your repositories and CI/CD pipelines, enhancing collaboration, workflow automation, and security. Testing and team training will be included. This offer is available only in Portuguese.
Information Search Platform on Azure Cosmos DB and Azure AI: Tiger Analytics will provide a comprehensive solution for information retrieval, interaction, and collaboration using Azure OpenAI Service, Azure App Service, Azure Functions, Azure Cosmos DB, and Delta Lake. Azure OpenAI and Cosmos DB will enable intelligent querying and summarization of unstructured and structured documents.
Microsoft Fabric Real-Time Intelligence: 4-Week PoC: WinWire will deliver a proof of concept of Microsoft Fabric, showcasing real-time data processing and analytics. Microsoft Fabric offers cost savings opportunities and reduced downtime. WinWire’s service includes assessment, implementation, and a road map for scaling real-time analytics.
OpenAI Azure Assistant: In this engagement, exe will implement assistants using Azure OpenAI Service to aid communication and request resolution, analyze your organizational needs, and provide answers using existing data. The assistants will integrate with Azure Virtual Machines, PostgreSQL, and Microsoft Teams.
Purview for Data Security and Governance: Maximize your data governance with Zure’s implementation of Microsoft Purview. Zure’s experts will optimize your Purview licenses, ensuring you understand and protect your organization’s sensitive data across all locations. A senior expert will lead the kickoff and targeted workshops, involving technical setup of Purview and development road map documentation.
Simplify and Optimize Data Management with Microsoft Fabric: 54cuatro Mexico’s workshop will explore Microsoft Fabric, an advanced platform that allows you to centralize and optimize business data management. During this session, you’ll discover how shortcuts in OneLake can transform the way your organization manages and analyzes data. This service is available only in Spanish.
Smart Production Planning System: Celebal Technologies will implement its Smart Production Planning System, an Industry 4.0-ready solution built on Microsoft Azure to transform manufacturing operations for production planners, shop floor operators, sales managers, and procurement managers. With seamless integration into Azure IoT systems, it provides real-time visibility into production plans, capacity constraints, actual throughput, and resource utilization.
ThoughtsWin Systems: Data Services: ThoughtsWin Systems offers comprehensive data services, including business intelligence strategy for actionable insights, data quality and governance for reliable data, and data engineering for efficient processing. ThoughtsWin Systems also provides IoT data integration for real-time intelligence and enterprise information management for streamlined information handling.
Transform Your Data Strategy with Noventiq’s Workshop on Microsoft Fabric: Microsoft Fabric is a SaaS solution for comprehensive data management, from ingestion to actionable insights. Noventiq’s one-week workshop will cover high-level architecture, data integration, governance, real-time analytics, and implementation scenarios. It aims to align Microsoft Fabric’s capabilities with your organizational goals, providing a tailored adoption road map.
Voice of the Customer Analytics Platform: SNP will implement its AI-powered Voice of the Customer (VOC) analytics platform, which uses Azure technologies to automate customer feedback processing. The platform extracts insights, identifies sentiment, and validates interactions against business criteria, and it consolidates feedback into a central knowledge base.
Workshop: Master Azure Synapse for Data Management and Transformation: In this workshop, 54cuatro Mexico will guide you step by step through the main Azure Synapse tools and services to manage, transform, and analyze data at scale. You will learn how to integrate key services, such as Azure Data Lake, Azure Data Factory, and Azure Key Vault, within a secure environment. This service is available only in Spanish.
Xoriant DevOps Tooling Infrastructure Modernization: Accelerate your DevOps journey with Xoriant’s migration services. Seamlessly transition from SVN to GitHub Enterprise, automate CI/CD pipelines with Azure DevOps, and enhance security with advanced GitHub measures. Optimize workflows with tailored repository segmentation and secure your code. Modernize your development and gain a competitive edge today.
Contact our partners
.NET Application Modernization: 1-Hour Briefing
.NET MAUI Mobile Application: 1-Hour Briefing
1956 Individuals: AI for Customer Engagement
App and Infrastructure: 1-Hour Briefing
Azure Analytics Modernization Assessment
Azure Cost Optimization Assessment
Azure Database Modernization Assessment
Azure Infrastructure Readiness: 7-Day Assessment
Azure Well-Architected Framework Evaluation
BonBon Generative AI Accelerator
Business Process Automation: 1-Hour Briefing
Cloud Cost Optimization Service: 4-Week Assessment
ConsoleWorks Cyber Security Platform
Copilot for Microsoft 365 Readiness & Data Security Analysis
Custom Training: 1-Hour Briefing
Cyber Essentials: CISO Operation
Cyber Premium: Security Operation
Data Buddy: Data Tech Modernization
Data Buddy: Knowledge Curation
Data Resilience Assessment: Veeam Data Cloud with Microsoft 365 Backup Storage
Devart ODBC Driver for Zoho Projects
DIGITALL Copilot Readiness: 7-Day Assessment
Digna: Modern Data Quality Platform for Data Warehouses
Dynamics 365 Sales: 1-Hour Briefing
Enola AI: AI Governance and Evaluation
Enterprise Search using AI Agents
EVA Unified Commerce: Store Front Consulting Sprint
Golf Tournament Management with ScoreCapture
HCLTech’s Caddie: A Calling Solution
Hexagon Computer-Aided Engineering Solutions
HFT-AlgoCompli: Algorithmic Trading Governance, Risk, and Compliance Management
Hy-Tek IntraOne and NetSuite No-Code Integration
Kosai Secure Open-Source Software
Microsoft Fabric: 4 Week Assessment
OpenFOAM v2406 on ARM for NEXTFOAM HPC
OpenFOAM v2406 with Intel oneAPI for NEXTFOAM HPC
PageSpeed Insights to Power BI
Power Platform: 1-Hour Briefing
Reporting Suite for Utility Companies Using Power BI
SAP Ariba No-Code Integration with Dynamics and NetSuite
ShareMy.Health Offline Maternal and Child Health
SharePoint Intranet: 1-Hour Briefing
Software Audit: 1-Hour Briefing
Stibo Systems Solution Governance Service
SUSE Long Term Service Pack Support (LTSS)
Synapse Link Integration with Microsoft Fabric
Tableau to Power BI Migration Utility Plus Assessment
Ubuntu 22.04 with Intel oneAPI for NEXTFOAM HPC
Unisys Attack Surface Discovery
VMware to Azure Stack HCI: 2-Week Assessment
Web Performance Monitoring Service
Website Chatbot: 1-Hour Briefing
Wipro’s Offering on AI Design Win Framework: 3-Hour Briefing
Yardi Dashboard and Analytics Platform by RentViewer
This content was generated by Microsoft Azure OpenAI and then revised by human editors.
Microsoft Tech Community – Latest Blogs –Read More
Exchange Online Ressources are not displayed in Teams
I recently tried to create resources in Exchange Online so that our users can book rooms through Teams. Unfortunately, the resources are only visible in the Outlook client and not in the Teams client.
I already created a new distribution group, added the resources, and set the building for each room.
New-DistributionGroup -Name “Company Rooms” -RoomList
Add-DistributionGroupMember -Identity “Company Rooms” -Member “Room A300”
Add-DistributionGroupMember -Identity “Company Rooms” -Member “Room A301”
Set-Place -Identity “Room A300” -Building “Germany Office”
Set-Place -Identity “Room A301” -Building “Germany Office”
I waited 24 hours after making the changes, but unfortunately, it’s still not working.
I recently tried to create resources in Exchange Online so that our users can book rooms through Teams. Unfortunately, the resources are only visible in the Outlook client and not in the Teams client.I already created a new distribution group, added the resources, and set the building for each room. New-DistributionGroup -Name “Company Rooms” -RoomList
Add-DistributionGroupMember -Identity “Company Rooms” -Member “Room A300”
Add-DistributionGroupMember -Identity “Company Rooms” -Member “Room A301”
Set-Place -Identity “Room A300” -Building “Germany Office”
Set-Place -Identity “Room A301” -Building “Germany Office” I waited 24 hours after making the changes, but unfortunately, it’s still not working. Read More
