Tag Archives: microsoft
Intune issue on Ubuntu 24.04
Hello,
I installed Intune on Ubuntu 24.04 after a distro upgrade. The install went fine but the identity provider agent report the “Unable to derive key material as unable to find the derivation key for alias” error.
Full trace:
journalctl -u microsoft-identity-device-broker.service
[…]
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: E/DeviceBrokerServiceOperation:generateDerivedKey: [2024-11-01 13:49:33 – thread_id: 17, correlation_id: ************************* – ] Unable to derive key material as unable to find the derivation key for alias: <redacted>
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: com.microsoft.identity.common.java.exception.ClientException: Unable to derive key material as unable to find the derivation key for alias: <redacted>
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at com.microsoft.identity.device.broker.operation.DeviceBrokerServiceOperation.generateDerivedKey(DeviceBrokerServiceOperation.java:173)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at com.microsoft.identity.broker.dbus.services.impl.DeviceBrokerDBusV1Impl.generateDerivedKey(DeviceBrokerDBusV1Impl.java:212)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.lang.reflect. Method.invoke(Method.java:566)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at org.freedesktop.dbus.connections.AbstractConnection$2.run(AbstractConnection.java:816)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.lang.Thread.run(Thread.java:829)
I know that in theory its not supported on 24.04 but I have some contact who managed the app to work, they just didn’t had that error.
Does anyone know what the error mean?
Ive already tried to uninstall all intune component + wiped the folders I found on my machine without success…
Hello, I installed Intune on Ubuntu 24.04 after a distro upgrade. The install went fine but the identity provider agent report the “Unable to derive key material as unable to find the derivation key for alias” error. Full trace: journalctl -u microsoft-identity-device-broker.service
[…]
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: E/DeviceBrokerServiceOperation:generateDerivedKey: [2024-11-01 13:49:33 – thread_id: 17, correlation_id: ************************* – ] Unable to derive key material as unable to find the derivation key for alias: <redacted>
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: com.microsoft.identity.common.java.exception.ClientException: Unable to derive key material as unable to find the derivation key for alias: <redacted>
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at com.microsoft.identity.device.broker.operation.DeviceBrokerServiceOperation.generateDerivedKey(DeviceBrokerServiceOperation.java:173)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at com.microsoft.identity.broker.dbus.services.impl.DeviceBrokerDBusV1Impl.generateDerivedKey(DeviceBrokerDBusV1Impl.java:212)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.lang.reflect. Method.invoke(Method.java:566)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at org.freedesktop.dbus.connections.AbstractConnection$2.run(AbstractConnection.java:816)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
nov 01 14:49:33 my-laptop microsoft-identity-device-broker[21822]: at java.base/java.lang.Thread.run(Thread.java:829) I know that in theory its not supported on 24.04 but I have some contact who managed the app to work, they just didn’t had that error.Does anyone know what the error mean? Ive already tried to uninstall all intune component + wiped the folders I found on my machine without success… Read More
How to Use the Graph SDK to Manage Group-Based Licensing
Group-based licensing is a mechanism to make it easier to assign and mange product licenses for large sets of user accounts. In this article, we discuss how to use Microsoft Graph PowerShell SDK cmdlets to manage group-based license assignments in a Microsoft 365 tenant. Assigning licenses to groups is very much like direct assignments, but some differences exist.
https://office365itpros.com/2024/11/04/group-based-licensing-sdk/
Group-based licensing is a mechanism to make it easier to assign and mange product licenses for large sets of user accounts. In this article, we discuss how to use Microsoft Graph PowerShell SDK cmdlets to manage group-based license assignments in a Microsoft 365 tenant. Assigning licenses to groups is very much like direct assignments, but some differences exist.
https://office365itpros.com/2024/11/04/group-based-licensing-sdk/ Read More
एयरटेल में गलत ट्रांजेक्शन का रिफंड कैसे करें ?
एयरटेल में गलत ट्रांजेक्शन का रिफंड कैसे पाएं? कस्टमर सपोर्ट से संपर्क करें डायल करें: – +91 88-61,11,16-24 या 0886-111-624 और अपना अकाउंट विवरण दें …8861111624
एयरटेल में गलत ट्रांजेक्शन का रिफंड कैसे पाएं? कस्टमर सपोर्ट से संपर्क करें डायल करें: – +91 88-61,11,16-24 या 0886-111-624 और अपना अकाउंट विवरण दें …8861111624 Read More
टाटा प्ले में शिकायत कैसे करें?
टाटा प्ले के खिलाफ शिकायत दर्ज करने के लिए, आप इन चरणों का पालन कर सकते हैं:198 टाटा प्ले ग्राहक सहायता से संपर्क करें:(0983✓122✓4914 ) एयरटेल वेबसाइट पर जाएँ या अपने मोबाइल डिवाइस पर टाटा प्ले खोलें।
टाटा प्ले के खिलाफ शिकायत दर्ज करने के लिए, आप इन चरणों का पालन कर सकते हैं:198 टाटा प्ले ग्राहक सहायता से संपर्क करें:(0983✓122✓4914 ) एयरटेल वेबसाइट पर जाएँ या अपने मोबाइल डिवाइस पर टाटा प्ले खोलें। Read More
एयरटेल पेमेंट बैंक से पैसा कट जाए तो क्या करें?
एयरटेल में गलत ट्रांजेक्शन का रिफंड कैसे पाएं? कस्टमर सपोर्ट से संपर्क करें डायल करें: – +91 88-61,11,16-24 या 0886-111-624 और अपना अकाउंट विवरण दें …8861111624
एयरटेल में गलत ट्रांजेक्शन का रिफंड कैसे पाएं? कस्टमर सपोर्ट से संपर्क करें डायल करें: – +91 88-61,11,16-24 या 0886-111-624 और अपना अकाउंट विवरण दें …8861111624 Read More
Transform array of cell addresses into hyperlinks
I need some help here… I have an array of cell addresses as input, which I would like to transform into an array of hyperlinks. Like this:
I tried HYPERLINK(A1#), but that doesn’t seem to work. Any suggestions?
I need some help here… I have an array of cell addresses as input, which I would like to transform into an array of hyperlinks. Like this: I tried HYPERLINK(A1#), but that doesn’t seem to work. Any suggestions? Read More
果博东方上分电话微70411382华为终端
2024年11月3日,华为为其鸿蒙HarmonyOS NEXT系统发布了5.0.0.102新版本更新,以及针对该版本的SP6子更新。此次更新主要面向华为MatePad Pro系列平板电脑、Mate60与Pura70系列智能手机,旨在通过新增功能和性能优化,提升用户体验。
新版本中最为引人注目的功能是“碰一碰分享”。这一功能允许用户通过将两部手机的顶部轻轻接触,快速传输视频、图片等内容,无需事先添加对方为好友。此外,SP6更新进一步优化了设备的信号稳定性和部分场景下的网络速度,同时改善了蓝牙连接的稳定性。在个性化设置方面,用户现在可以将本地音乐或在线铃声设置为来电铃声,并且增加了“响铃时振动”、“拨号按键音”等实用功能选项。
华为常务董事、终端BG董事长及智能汽车解决方案BU董事长余承东曾表示,原生鸿蒙系统代表着一个全新的开始,团队正全力以赴以确保系统的持续优化与完善,以期提供更加流畅、安全、便捷的使用体验给广大用户。原生鸿蒙操作系统在内核、数据库、编程语言以及AI大模型等方面实现了全面自主研发,标志着中国在操作系统领域取得了重要进展,实现了技术上的自主可控。
此次更新不仅体现了华为对于技术创新的追求,也展现了其致力于打造更加开放、兼容的操作系统生态的决心。随着更多功能的加入和优化,鸿蒙系统正逐渐成长为一个能够满足多样化需求的成熟平台。
2024年11月3日,华为为其鸿蒙HarmonyOS NEXT系统发布了5.0.0.102新版本更新,以及针对该版本的SP6子更新。此次更新主要面向华为MatePad Pro系列平板电脑、Mate60与Pura70系列智能手机,旨在通过新增功能和性能优化,提升用户体验。新版本中最为引人注目的功能是“碰一碰分享”。这一功能允许用户通过将两部手机的顶部轻轻接触,快速传输视频、图片等内容,无需事先添加对方为好友。此外,SP6更新进一步优化了设备的信号稳定性和部分场景下的网络速度,同时改善了蓝牙连接的稳定性。在个性化设置方面,用户现在可以将本地音乐或在线铃声设置为来电铃声,并且增加了“响铃时振动”、“拨号按键音”等实用功能选项。 华为常务董事、终端BG董事长及智能汽车解决方案BU董事长余承东曾表示,原生鸿蒙系统代表着一个全新的开始,团队正全力以赴以确保系统的持续优化与完善,以期提供更加流畅、安全、便捷的使用体验给广大用户。原生鸿蒙操作系统在内核、数据库、编程语言以及AI大模型等方面实现了全面自主研发,标志着中国在操作系统领域取得了重要进展,实现了技术上的自主可控。 此次更新不仅体现了华为对于技术创新的追求,也展现了其致力于打造更加开放、兼容的操作系统生态的决心。随着更多功能的加入和优化,鸿蒙系统正逐渐成长为一个能够满足多样化需求的成熟平台。 Read More
फोनपे से पैसे काटने पर क्या करें
फोनपे से पैसे काटने पर क्या करें में ” संपर्क करें.((,+91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन-डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। अगर पुराना नहीं है अर्थात, तो अपने बैंक से संपर्क करें.
फोनपे से पैसे काटने पर क्या करें में ” संपर्क करें.((,+91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन-डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। अगर पुराना नहीं है अर्थात, तो अपने बैंक से संपर्क करें. Read More
Data Flow textbox size increase
Hi Community,
Is there a way to increase the textbox size to Data Flows Select mapping fields (or request a change)? The Destination field (Name As) is way smaller than the source field, and therefore can’t see the complete text – there is plenty of space to the right to increase this size…
Hi Community, Is there a way to increase the textbox size to Data Flows Select mapping fields (or request a change)? The Destination field (Name As) is way smaller than the source field, and therefore can’t see the complete text – there is plenty of space to the right to increase this size… Read More
Disable mouse over text / hover tooltips
Hi Community,
How can I disable the annoying tooltips / hover text when working in Azure Synpase Studio. When navigating around the page, the tooltip will appear multiple times, getting in the way of actually clicking where I want to go.
Example below
Hi Community, How can I disable the annoying tooltips / hover text when working in Azure Synpase Studio. When navigating around the page, the tooltip will appear multiple times, getting in the way of actually clicking where I want to go.Example below Read More
How do I contact Qantas frequent flyers? Qantas Regional
To contact Qantas Frequent Flyers, you can:
Call: Call the Qantas Frequent Flyer Service Centre at +61 1800 005 886 within Australia, or +61 1800 005 886 for other countries. The service center is open 24/7,(AET).
Email: Send an email to email address removed for privacy reasons.
Online form: Complete the online contact form.
If you think your Qantas Frequent Flyer account has been compromised, you should call +61 1800 005 886 the Qantas Frequent Flyer Service Centre immediately. You can also report a scam to Scamwatch.
If you want to provide feedback or make a complaint about Qantas’s scheduled services, +61 1800 005 886 you can contact Customer Care in writing at Qantas Airways Limited, 10 Bourke Road, Mascot NSW 2020, Australia. In the US, you can also call Qantas on +61 1800 005 886.
To contact Qantas Frequent Flyers, you can:Call: Call the Qantas Frequent Flyer Service Centre at +61 1800 005 886 within Australia, or +61 1800 005 886 for other countries. The service center is open 24/7,(AET).Email: Send an email to email address removed for privacy reasons.Online form: Complete the online contact form.If you think your Qantas Frequent Flyer account has been compromised, you should call +61 1800 005 886 the Qantas Frequent Flyer Service Centre immediately. You can also report a scam to Scamwatch.If you want to provide feedback or make a complaint about Qantas’s scheduled services, +61 1800 005 886 you can contact Customer Care in writing at Qantas Airways Limited, 10 Bourke Road, Mascot NSW 2020, Australia. In the US, you can also call Qantas on +61 1800 005 886. Read More
क्या मोबिक्विक पैसा वापस करेगा?
क्या मोबिक्विक पैसा वापस करेगा?”ट्रांज़ेक्शन” (91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन- डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। यदि भुगतान नहीं मिलता है, तो अपने बैंक से संपर्क करें,,,
क्या मोबिक्विक पैसा वापस करेगा?”ट्रांज़ेक्शन” (91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन- डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। यदि भुगतान नहीं मिलता है, तो अपने बैंक से संपर्क करें,,, Read More
क्या पेटीएम विफल लेनदेन पर पैसे वापस करता है?
क्या पेटीएम विफल लेनदेन पर पैसे वापस करता है? व्यापारी में “ट्रांजेक्शन” ((,+91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन-डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। अगर पुराना नहीं है अर्थात, तो अपने बैंक से संपर्क करें.
क्या पेटीएम विफल लेनदेन पर पैसे वापस करता है? व्यापारी में “ट्रांजेक्शन” ((,+91-629O-348-172) या “इतिहास” सेक्शन में जाएं। रिले लेन-डेन चुनें। “वापस लें” या “वापस लेने के लिए भुगतान करें” विकल्प चुनें। अगर पुराना नहीं है अर्थात, तो अपने बैंक से संपर्क करें. Read More
Has the SQL Server Bulk Insert Assertion Error (KB 2700641) Been Resolved in SQL Server 2019?
Hello SQL Community,
I’m reaching out to confirm whether the specific issue described in KB 2700641 has been resolved in SQL Server 2019. The issue in question, which occurred in SQL Server 2008 and 2008 R2, involved an assertion error during BULK INSERT or BCP operations without the CHECK_CONSTRAINTS option enabled. Although our tables do not have constraints to check during insert operations, we had to use the CHECK_CONSTRAINTS option as a workaround in SQL Server 2008.
I have reviewed the release notes for SQL Server 2012 – 2019 but couldn’t find an explicit mention of this particular fix. Before we remove the workaround from our code, could anyone confirm if this issue has indeed been addressed in SQL Server 2019? Or is there any documentation available that confirms this bug was fixed in a specific SQL Server version?
Thank you for any insights or official references you could provide!
Hello SQL Community,I’m reaching out to confirm whether the specific issue described in KB 2700641 has been resolved in SQL Server 2019. The issue in question, which occurred in SQL Server 2008 and 2008 R2, involved an assertion error during BULK INSERT or BCP operations without the CHECK_CONSTRAINTS option enabled. Although our tables do not have constraints to check during insert operations, we had to use the CHECK_CONSTRAINTS option as a workaround in SQL Server 2008.I have reviewed the release notes for SQL Server 2012 – 2019 but couldn’t find an explicit mention of this particular fix. Before we remove the workaround from our code, could anyone confirm if this issue has indeed been addressed in SQL Server 2019? Or is there any documentation available that confirms this bug was fixed in a specific SQL Server version?Thank you for any insights or official references you could provide! Read More
I have idea get schedule from web
I have idea get schedule from web
I have id and password. How setup this
Thanks you
I have idea get schedule from web I have id and password. How setup this Thanks you Read More
Enhancing E-Commerce Product Search with Vector Similarity in Azure Cosmos DB
In the competitive landscape of e-commerce, delivering precise and relevant search results is crucial for enhancing user experience and driving sales. Traditional keyword-based search methods often struggle to grasp the nuanced intentions behind user queries. This is where vector similarity search comes into play, leveraging advanced embedding techniques to provide more accurate and context-aware search results. In this blog, we’ll walk you through implementing vector similarity search in an e-commerce products API using Azure Cosmos DB for NoSQL and TypeScript (JavaScript).
Introduction to Vector Similarity Search
Vector similarity search is a powerful technique that transforms data into high-dimensional vectors (embeddings) and enables the comparison of these vectors to find similarities. In the context of e-commerce, this allows for more intelligent product searches, recommendations, and categorization by understanding the semantic meaning behind product descriptions, tags, and features.
Embeddings
Modern machine learning models can be trained to convert raw data into embeddings, represented as arrays (or vectors) of floating-point numbers of fixed dimensionality. What makes embeddings useful in practice is that the position of an embedding in vector space captures some of the semantics of the data, depending on the type of model and how it was trained. Points that are close to each other in vector space are considered similar (or appear in similar contexts), and points that are far away are considered dissimilar.
Large datasets of multi-modal data (text, audio, images, etc.) can be converted into embeddings with the appropriate model. Projecting the vectors’ principal components in 2D space results in groups of vectors that represent similar concepts clustering together, as shown below.
Indexes
Embeddings for a given dataset are made searchable via an index. The index is constructed by using data structures that store the embeddings such that it’s very efficient to perform scans and lookups on them
Brute force search
The simplest way to perform vector search is to perform a brute force search, without an index, where the distance between the query vector and all the vectors in the database are computed, with the top-k closest vectors returned. This is equivalent to a k-nearest neighbours (kNN) search in vector space.
As you can imagine, the brute force approach is not scalable for datasets larger than a few hundred thousand vectors, as the latency of the search grows linearly with the size of the dataset. This is where approximate nearest Neighbour (ANN) algorithms come in.
Overview of Azure Cosmos DB Vector Search
Azure Cosmos DB for NoSQL now offers a vector search feature in preview, designed to handle high-dimensional vectors efficiently and accurately at any scale. This feature allows you to store vectors directly within your documents alongside traditional schema-free data. By colocating data and vectors, Azure Cosmos DB ensures efficient indexing and searching, simplifying data management and enhancing AI application architectures.
Key capabilities include:
- Vector Embedding Policies: Define how vectors are generated and stored within your documents.
- Vector Indexing: Optimize data retrieval based on vector similarities using various indexing methods.
- Vector Distance Functions: Utilize metrics like Cosine, Dot Product, and Euclidean to measure similarity.
By integrating vector similarity search into your e-commerce API, you can significantly enhance the relevance and personalization of search results.
What is a Vector Store?
A vector store or vector database is designed to store and manage vector embeddings—mathematical representations of data in a high-dimensional space. Each dimension corresponds to a feature of the data, and a vector’s position represents its characteristics. This enables efficient similarity comparisons for various data types, including text, images, audio, and more.
How Does a Vector Store Work?
Vector stores use vector search algorithms to index and query embeddings. These algorithms help find similar items based on data characteristics rather than exact matches. This technique is invaluable for applications like searching similar text, finding related images, making recommendations, and detecting anomalies. Vector search measures the distance between data vectors and a query vector, retrieving the most semantically similar results.
Azure Cosmos DB’s integrated vector database allows embeddings to be stored, indexed, and queried alongside original data, eliminating the need for separate vector databases and enhancing data consistency, scalability, and performance.
Prerequisites
Before diving into the implementation, ensure you have the following:
- Azure Cosmos DB for NoSQL Account: If you don’t have one, read how to create one here or use an existing Azure subscription.
- Azure Cosmos DB JavaScript SDK: Version 4.1.0 or later.
- Node.js and TypeScript: Familiarity with these technologies is essential.
- Vector Search Preview Registration: Required to access vector search features. Read more here
- Azure Account: Try Azure for free, if student Try azure for Students
Registering for Azure Cosmos DB Vector Search Preview
Vector search in Azure Cosmos DB is currently in preview and requires explicit registration. Follow these steps to enroll:
Navigate to Azure Portal:
- Search for Azure Cosmos DB
- Click create
- Create Azure Cosmos DB for NoSQL
- Please complete the process of provisioning, if you have any issue check this blog
- go to your Azure Cosmos DB for NoSQL resource page.
Access Features Pane:
- Select the “Features” pane under the “Settings” menu.
Enable Vector Search:
- Find “Vector Search in Azure Cosmos DB for NoSQL.”
- Read the feature description to ensure it meets your requirements.
- Click “Enable” to enroll in the preview. After it is enabled, it should look like the below image
Note: The vector search feature is only supported on new containers. Existing containers cannot be retrofitted with vector search capabilities.
The registration request will be auto-approved, but it may take several minutes to take effect.
Keys we shall need in our env file
- Visit Data Explorer, select New database and then provide your database id ie. productsdb. This will be the name we shall use in our .env file example AZURE_COSMOS_DB = productsdb
- To obtain AZURE_COSMOS_DB_KEY, click Settings, go to Keys and copy the PRIMARY KEY. To get AZURE_COSMOS_DB_ENDPOINT, copy URI.
Provisioning Azure OpenAI
- On Azure portal, search Azure OpenAI
- Create and complete the provisioning process.
- After you’re done, go to overview and click Explore Azure AI Studio
- This will redirect you to Azure AI Studio, sign in with the same email account you signed in Azure Portal
- In the Azure AI Studio, Click Deployments, select Deploy base model from Deploy model.
- Select and check embeddings from Inference tasks menu.
- Select text-embedding-3-large and confirm to provision the model.
- Give your model a deployment name or go with the default and click deploy.
- After the deployment is complete copy the Target URI this will be the value of AZURE_OPENAI_ENDPOINT and Key will be the value of AZURE_OPENAI_API_KEY in our .env file.
- With that you should have below values
AZURE_COSMOS_DB_ENDPOINT=
AZURE_COSMOS_DB_KEY=
AZURE_COSMOS_DB=productsdb
AZURE_OPENAI_API_KEY=
AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT=
Setting Up Azure Cosmos DB Container for Vector Search
Creating the Database and Container
Assume we’re building an internet-based store for smart home devices. Each product has properties like Name
, Brand
, SKU
, Category
, and Description
. We’ll also define vector embeddings for descriptionVector
, tagsVector
, and featuresVector
to enable vector-based searches.
Example JSON Structure:
{
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
"category": "Home Automation",
"price": 19.99,
"currency": "USD",
"stock": 500,
"description": "Illuminate your home with BrightHome Smart LED Light Bulbs. Control brightness and color temperature via smartphone or voice commands.",
"features": "Wi-Fi Enabled, Voice Control Compatible, Energy Efficient, Adjustable Brightness, Long Lifespan, Easy Installation",
"rating": 4.3,
"reviewsCount": 850,
"tags": [
"lighting",
"smart home",
"LED",
"energy-efficient",
"voice-control"
],
"imageUrl": "https://example.com/images/products/SKU-1002.jpg",
"manufacturer": "BrightHome Technologies",
"model": "BH-SMART-LB",
"releaseDate": "2023-07-20",
"warranty": "1 year",
"dimensions": {
"weight": "75g",
"width": "6.5cm",
"height": "11cm",
"depth": "6.5cm"
},
"color": "White",
"material": "Glass and Plastic",
"origin": "Germany",
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f"
}
Defining Vector Embedding Policy
The vector embedding policy informs the Cosmos DB query engine how to handle vector properties. It specifies the path, data type, dimensions, and distance function for each vector.
Typescript Example:
const vectorEmbeddingPolicy = {
vectorEmbeddings: [
{
path: "/descriptionVector",
dataType: "float32",
dimensions: 1536,
distanceFunction: "cosine",
},
{
path: "/tagsVector",
dataType: "float32",
dimensions: 1536,
distanceFunction: "cosine",
},
{
path: "/featuresVector",
dataType: "float32",
dimensions: 1536,
distanceFunction: "cosine",
},
],
};
Adding Vector Indexes to the Indexing Policy
Vector indexes optimize the storage and retrieval of vector data. You must define these indexes during container creation.
Typescript Example:
const indexingPolicy = {
vectorIndexes: [
{ path: "/descriptionVector", type: "diskANN" },
{ path: "/tagsVector", type: "diskANN" },
{ path: "/featuresVector", type: "diskANN" },
],
includedPaths: [
{
path: "/*",
},
],
excludedPaths: [
{
path: "/descriptionVector/*",
},
{
path: "/tagsVector/*",
},
{
path: "/featuresVector/*",
},
],
};
Note: During early preview, vector indexes can’t be modified once created. If changes are needed, create a new container with the updated vector index policy.
Creating the Container
With the embedding and indexing policies defined, create the container:
Typescript Example:
const containerName = "vector-embedding-container";
// Create container
const { resource: containerdef } = await database.containers.createIfNotExists({
id: containerName,
vectorEmbeddingPolicy: vectorEmbeddingPolicy,
indexingPolicy: indexingPolicy,
});
Important: Vector search is only supported on new containers. Ensure both the vector embedding policy and vector indexing policy are set during container creation.
Building the Products API with Vector Search
We’ll build a TypeScript-based API using Express that interacts with Azure Cosmos DB to perform vector similarity searches.
Folder Structure
Project Setup
package.json
:
{
"name": "products-api-cosmosdb-typescript-vector-search",
"version": "1.0.0",
"description": "a Products API using Express and TypeScript, and integrate Azure Cosmos DB (with vector search capabilities) for managing product data. We'll also implement vector search, enabling us to query items based on vector similarity.",
"scripts": {
"dev": "tsx watch src/server.ts",
"build": "rimraf dist && tsc",
"start": "node dist/server.js"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"@types/cors": "^2.8.17",
"@types/express": "^5.0.0",
"@types/morgan": "^1.9.9",
"@types/multer": "^1.4.12",
"@types/node": "^22.7.9",
"@types/uuid": "^10.0.0",
"rimraf": "^6.0.1",
"tsx": "^4.19.1",
"typescript": "^5.6.3"
},
"dependencies": {
"@azure/cosmos": "^4.1.1",
"axios": "^1.7.7",
"cors": "^2.8.5",
"dotenv": "^16.4.5",
"express": "^4.21.1",
"express-rate-limit": "^7.4.1",
"helmet": "^8.0.0",
"morgan": "^1.10.0",
"multer": "1.4.5-lts.1",
"pino": "^9.5.0",
"pino-pretty": "^11.3.0",
"rotating-file-stream": "^3.2.5",
"uuid": "^10.0.0",
"zod": "^3.23.8"
}
}
tsconfig.json
:
{
"compilerOptions": {
/* Language and Environment */
"target": "ES2022",
/* Modules */
"module": "CommonJS",
"rootDir": "./src",
"outDir": "./dist",
"moduleResolution": "node",
/* Interop Constraints */
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"allowSyntheticDefaultImports": true,
/* Type Checking */
"strict": true,
/* Completeness */
"skipLibCheck": true
},
"exclude": [
"node_modules",
"dist"
],
"include": [
"src/**/*.ts"
]
}
Configuring Environment Variables
Create a .env
file to manage sensitive configurations:
.env
:
# Azure Cosmos DB Configuration
AZURE_COSMOS_DB_ENDPOINT=https://<your-cosmosdb-account>.documents.azure.com:443/
AZURE_COSMOS_DB_KEY=<your-cosmosdb-key>
AZURE_COSMOS_DB=<your-database-name>
# Azure OpenAI Configuration
AZURE_OPENAI_API_KEY=<your-openai-api-key>
# Azure OpenAI Text Embedding Model Configuration
AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT=https://<your-openai-endpoint>.openai.azure.com/openai/deployments/<your-embedding-model-deployment>/embeddings?api-version=2023-05-15
Ensure you replace placeholder values with your actual credentials.
Setting Up the Server
src/server.ts
:
//server/src/server.ts
import config from './config/env';
import logger from './utils/logger';
import CosmosDB from './services/cosmosdbService';
import initializeServices from './services/index';
import app from './app';
// Start server
const startServer = async () => {
try {
// Initialize CosmosDB
const CosmosInstance = CosmosDB.getInstance();
await CosmosInstance.initialize();
// Initialize services after CosmosDB is initialized
const services = initializeServices();
// Attach services to app.locals for access in controllers
app.locals.services = services;
const PORT = config.port;
app.listen(PORT, () => {
logger.info(`Server is running on port ${PORT}`);
});
} catch (error) {
logger.error('Failed to initialize Cosmos DB:', error);
process.exit(1);
}
};
startServer();
Implementing Logging
src/utils/logger.ts
:
// src/utils/logger.ts
import pino from 'pino'
const logger = pino({
base: {
pid: false
},
transport: {
target: 'pino-pretty',
options: {
colorize: true
}
}
})
export default logger;
Using Pino for efficient and structured logging.
Defining Routes
src/routes/productsRoutes.ts
:
//src/routes/productsRoutes.ts
import express, { Request, Response } from "express";
import {
createProduct,
getProductById,
getProducts,
searchProductsByDescriptionVector,
searchProductsByFeaturesVector,
searchProductsByTagsVector
} from "../controllers/productController";
const productRouter = express.Router();
// Create a new product
productRouter.post('/products', (req: Request, res: Response) => {
createProduct(req, res);
});
// Get all products
productRouter.get('/products', (req: Request, res: Response) => {
getProducts(req, res);
});
// Get a product by ID
productRouter.get('/products/:id', (req: Request, res: Response) => {
getProductById(req, res);
});
// VECTOR SEARCH ROUTES
// Search products by description vector
productRouter.post('/products/search/description', (req: Request, res: Response) => {
searchProductsByDescriptionVector(req, res);
});
// Search products by tags vector
productRouter.post('/products/search/tags', (req: Request, res: Response) => {
searchProductsByTagsVector(req, res);
});
// Search products by features vector
productRouter.post('/products/search/features', (req: Request, res: Response) => {
searchProductsByFeaturesVector(req, res);
});
export default productRouter;
Implementing Controllers and Services
src/controllers/productController.ts
:
// src/controllers/productController.ts
import { v4 as uuidv4 } from 'uuid';
import { Request, Response } from 'express';
import { productSchema, Product, productIDSchema, featuresSchema, tagsSchema } from '../schemas/productSchema';
import { generateTextVector } from '../services/embeddingService';
import logger from '../utils/logger';
/**
*
* req The request object.
* res The response object to send back the fetched products.
* @returns The list of products.
*/
export const getProducts = async (req: Request, res: Response) => {
// Access productService from app.locals
const { productService } = req.app.locals.services;
try {
const products = await productService.getProducts();
return res.status(200).json({
status: 'success',
data: products,
});
} catch (error: any) {
logger.error('Error getting products:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
/**
*
* req The request object containing product data. ie product id
* res The response object to send back the fetched product.
* @returns The product associated with the given id.
*/
export const getProductById = async (req: Request, res: Response) => {
// Access productService from app.locals
const { productService } = req.app.locals.services;
const { id } = req.params;
// use zod to validate the id
const parseIDResult = productIDSchema.safeParse({ id });
if (!parseIDResult.success) {
return res.status(400).json({
status: 'error',
message: 'Invalid product ID, should be a valid UUID',
});
}
try {
const product = await productService.getProductById(parseIDResult.data.id);
if (!product) {
return res.status(404).json({
status: 'error',
message: 'Product not found',
});
}
return res.status(200).json({
status: 'success',
data: product,
});
} catch (error: any) {
logger.error('Error getting product:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
/**
* Creates a new product with vector embeddings.
* req The request object containing product data.
* res The response object to send back the created product.
*/
export const createProduct = async (req: Request, res: Response) => {
const { productService } = req.app.locals.services;
// zod validation for a new product
const parseResult = productSchema.safeParse(req.body);
if (!parseResult.success) {
const errors = parseResult.error.errors.map(err => ({
path: err.path.join('.'),
message: err.message,
}));
return res.status(400).json({
status: 'error',
errors,
});
}
const product: Product = parseResult.data;
try {
if (!product.id) {
product.id = uuidv4();
}
// Generate description vector
if (product.description) {
product.descriptionVector = await generateTextVector(product.description);
}
// Generate tags vector
if (product.tags && product.tags.length > 0) {
const tagsText = product.tags.join(' ');
product.tagsVector = await generateTextVector(tagsText);
}
// Generate features vector
if (product.features) {
product.featuresVector = await generateTextVector(product.features);
}
const createdProduct = await productService.addProduct(product);
return res.status(201).json({
status: 'success',
data: createdProduct,
});
} catch (error: any) {
logger.error('Error creating product:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
/**
* Searches for products based on a tags vector.
* req The request object containing the query vector.
* res The response object to send back the search results.
*/
export const searchProductsByTagsVector = async (req: Request, res: Response) => {
const { productService } = req.app.locals.services;
const { queryTags, top } = req.body;
// zod validation
const parsedTags = tagsSchema.safeParse({ tags: queryTags });
if (!parsedTags.data) {
return res.status(400).json({
status: 'error',
message: 'Invalid tags. It should be an array of strings ie "queryVector" : ["item","item2"].',
});
}
const tagsText = parsedTags.data.tags.join(' '); // Convert the tags array to a single string
const tagsVector = await generateTextVector(tagsText); // Generate the tags vector
const topResults = top && Number.isInteger(top) && top > 0 ? top : 10; // Get the top results ie 10 results
try {
const results = await productService.searchProductsByTagsVector(tagsVector, topResults);
return res.status(200).json({
status: 'success',
data: results,
});
} catch (error: any) {
logger.error('Error searching products by tags:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
/**
* Searches for products based on a features vector.
* req The request object containing the query vector.
* res The response object to send back the search results.
*/
export const searchProductsByFeaturesVector = async (req: Request, res: Response) => {
const { productService } = req.app.locals.services;
const { queryFeatures, top } = req.body;
// zod validation
const parsedFeature = featuresSchema.safeParse({ tags: queryFeatures });
if (!parsedFeature.success) {
return res.status(400).json({
status: 'error',
message: 'Invalid features. It should be an array of strings ie "queryVector" : ["item","item2"].',
});
}
const tagsText = parsedFeature.data.tags.join(' ');
const featuresVector = await generateTextVector(tagsText);
const topResults = top && Number.isInteger(top) && top > 0 ? top : 10;
try {
const results = await productService.searchProductsByFeaturesVector(featuresVector, topResults);
return res.status(200).json({
status: 'success',
data: results,
});
} catch (error: any) {
logger.error('Error searching products by features:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
/**
* Searches for products based on a description vector.
* req The request object containing the query vector.
* res The response object to send back the search results.
*/
export const searchProductsByDescriptionVector = async (req: Request, res: Response) => {
const { productService } = req.app.locals.services;
const { queryDescription, top } = req.body;
const descriptionVector = await generateTextVector(queryDescription);
const topResults = top && Number.isInteger(top) && top > 0 ? top : 10;
try {
const results = await productService.searchProductsByDescriptionVector(descriptionVector, topResults);
return res.status(200).json({
status: 'success',
data: results,
});
} catch (error: any) {
logger.error('Error searching products by description:', error.message);
return res.status(500).json({
status: 'error',
message: 'Internal Server Error',
});
}
};
src/services/productService.ts
:
// server/src/services/productService.ts
import { Container } from '@azure/cosmos';
import CosmosDB from '../services/cosmosdbService';
import { Product } from '../schemas/productSchema';
class ProductService {
private container: Container;
constructor() {
const cosmosDB = CosmosDB.getInstance();
this.container = cosmosDB.getProductsContainer();
}
/**
* Adds a new product to the Cosmos DB container.
* product The validated product to add.
* @returns The created product resource.
*/
public async addProduct(product: Product) {
const { resource } = await this.container.items.create<Product>(product);
return resource;
}
/**
* Retrieves a product by its ID.
* id The ID of the product.
* @returns The retrieved product or null if not found.
*/
public async getProductById(id: string) {
try {
const { resource } = await this.container.item(id, id).read<Product>();
return resource;
} catch (error: any) {
if (error.code === 404) {
return null;
}
throw error;
}
}
/**
* Retrieves all products from the Cosmos DB container.
* @returns An array of products.
*/
public async getProducts() {
const { resources } = await this.container.items
.query("SELECT c.id, c.name, c.brand, c.sku, c.category, c.price, c.currency, c.stock, c.description, c.features, c.rating, c.reviewsCount, c.tags, c.imageUrl, c.manufacturer, c.model, c.releaseDate, c.warranty, c.dimensions, c.color, c.material, c.origin FROM c")
.fetchAll();
return resources;
}
/**
* Searches for products similar to the provided description vector.
* queryVector The vector representing the search query.
* top The number of top similar products to retrieve.
* @returns An array of products with similarity scores.
*/
public async searchProductsByDescriptionVector(queryVector: number[], top: number = 10) {
const querySpec = {
query: `
SELECT TOP @top
c.id,
c.name,
c.brand,
c.sku,
c.category,
c.price,
c.currency,
c.stock,
c.description,
c.features,
c.rating,
c.reviewsCount,
c.tags,
c.imageUrl,
c.manufacturer,
c.model,
c.releaseDate,
c.warranty,
c.dimensions,
c.color,
c.material,
c.origin,
VectorDistance(c.descriptionVector, @queryVector) AS SimilarityScore
FROM c
ORDER BY VectorDistance(c.descriptionVector, @queryVector)
`,
parameters: [
{ name: "@queryVector", value: queryVector },
{ name: "@top", value: top }
]
};
const { resources } = await this.container.items.query(querySpec).fetchAll();
return resources;
}
/**
* Searches for products similar to the provided tags vector.
* queryVector The vector representing the tags search query.
* top The number of top similar products to retrieve.
* @returns An array of products with similarity scores.
*/
public async searchProductsByTagsVector(queryVector: number[], top: number = 10) {
const querySpec = {
query: `
SELECT TOP @top
c.id,
c.name,
c.brand,
c.sku,
c.category,
c.price,
c.currency,
c.stock,
c.description,
c.features,
c.rating,
c.reviewsCount,
c.tags,
c.imageUrl,
c.manufacturer,
c.model,
c.releaseDate,
c.warranty,
c.dimensions,
c.color,
c.material,
c.origin,
VectorDistance(c.tagsVector, @queryVector) AS SimilarityScore
FROM c
ORDER BY VectorDistance(c.tagsVector, @queryVector)
`,
parameters: [
{ name: "@queryVector", value: queryVector },
{ name: "@top", value: top }
]
};
const { resources } = await this.container.items.query(querySpec).fetchAll();
return resources;
}
/**
* Searches for products similar to the provided features vector.
* queryVector The vector representing the features search query.
* top The number of top similar products to retrieve.
* @returns An array of products with similarity scores.
*/
public async searchProductsByFeaturesVector(queryVector: number[], top: number = 10) {
const querySpec = {
query: `
SELECT TOP @top
c.id,
c.name,
c.brand,
c.sku,
c.category,
c.price,
c.currency,
c.stock,
c.description,
c.features,
c.rating,
c.reviewsCount,
c.tags,
c.imageUrl,
c.manufacturer,
c.model,
c.releaseDate,
c.warranty,
c.dimensions,
c.color,
c.material,
c.origin,
VectorDistance(c.featuresVector, @queryVector) AS SimilarityScore
FROM c
ORDER BY VectorDistance(c.featuresVector, @queryVector)
`,
parameters: [
{ name: "@queryVector", value: queryVector },
{ name: "@top", value: top }
]
};
const { resources } = await this.container.items.query(querySpec).fetchAll();
return resources;
}
}
export default ProductService;
src/services/index.ts
:
// server/src/services/index.ts
import ProductService from './productService';
import CosmosDB from './cosmosdbService';
// Ensure CosmosDB is initialized before creating services
const initializeServices = () => {
const cosmosInstance = CosmosDB.getInstance();
// You can perform any additional service initialization here if needed
return {
productService: new ProductService(),
// Add other services here
};
};
export default initializeServices;
Generating Vector Embeddings
To perform vector similarity searches, we need to generate embeddings for product descriptions, tags, and features. We’ll use Azure OpenAI’s Embedding API for this purpose.
src/services/embeddingService.ts
:
//src/services/embeddingService.ts
import axios from 'axios';
import config from '../config/env';
/**
* Generates a vector embedding for a given text using OpenAI's Embedding API.
* text The input text to embed.
* @returns An array of numbers representing the embedding vector.
*/
export async function generateTextVector(text: string): Promise<number[]> {
const response = await axios.post(config.azureOpenAi.models.embedding.endPoint,
{ input: text },
{
headers: {
'Content-Type': 'application/json',
'api-key': config.azureOpenAi.apiKey,
}
});
const vector = response.data.data[0].embedding;
return vector;
}
This service takes input text and returns its vector embedding, which is then stored in Cosmos DB.
Initializing CosmosDB Service
src/services/cosmosdbService.ts
:
//src/services/cosmosdbService.ts
import {
PartitionKeyDefinitionVersion,
PartitionKeyKind,
CosmosClient,
Container,
CosmosDbDiagnosticLevel,
VectorEmbeddingPolicy,
IndexingPolicy,
VectorIndexType,
VectorEmbeddingDataType,
VectorEmbeddingDistanceFunction
} from '@azure/cosmos';
import logger from '../utils/logger';
import config from '../config/env';
class CosmosDB {
private static instance: CosmosDB;
private client: CosmosClient;
private productsContainer!: Container;
private initialized: boolean = false;
private initializing: Promise<void> | null = null;
// Private constructor to prevent direct instantiation
private constructor() {
this.client = new CosmosClient({
endpoint: config.cosmos.endpoint,
key: config.cosmos.key,
connectionPolicy: {
requestTimeout: 10000,
},
diagnosticLevel: CosmosDbDiagnosticLevel.debug,
});
}
// Public method to get the singleton instance
public static getInstance(): CosmosDB {
if (!CosmosDB.instance) {
CosmosDB.instance = new CosmosDB();
}
return CosmosDB.instance;
}
// Initialize the database and containers
public async initialize(): Promise<void> {
if (this.initialized) {
return;
}
if (this.initializing) {
return this.initializing;
}
this.initializing = this.init();
return this.initializing;
}
// Internal initialization logic
private async init(): Promise<void> {
try {
// Create database if it doesn't exist
const { database } = await this.client.databases.createIfNotExists({
id: config.cosmos.database,
// throughput: 400 // Adjust as needed or make configurable
});
// Initialize productsContainer
this.productsContainer = await this.createContainerIfNotExists(database, 'productsContainer');
this.initialized = true;
logger.info(`Database : ${database.id} initialization successfully`);
logger.info(`Container : ${this.productsContainer.id} initialization successfully`);
} catch (error: any) {
if (error instanceof Error) {
logger.error('Error initializing CosmosDB:', error.message);
logger.error('Stack Trace:', error.stack);
} else {
logger.error('Unexpected error initializing CosmosDB:', JSON.stringify(error, null, 2));
}
throw error;
}
}
// Helper method to create a container if it doesn't exist
private async createContainerIfNotExists(database: any, containerId: string): Promise<Container> {
const vectorEmbeddingPolicy: VectorEmbeddingPolicy = {
vectorEmbeddings: [
{
path: "/imageVector",
dataType: VectorEmbeddingDataType.Float32,
dimensions: 8,
distanceFunction: VectorEmbeddingDistanceFunction.DotProduct,
},
{
path: "/descriptionVector",
dataType: VectorEmbeddingDataType.Float32,
dimensions: 10,
distanceFunction: VectorEmbeddingDistanceFunction.Cosine,
},
{
path: "/tagsVector",
dataType: VectorEmbeddingDataType.Float32,
dimensions: 10, // Adjust dimensions as per your embedding model
distanceFunction: VectorEmbeddingDistanceFunction.Cosine,
},
{
path: "/featuresVector",
dataType: VectorEmbeddingDataType.Float32,
dimensions: 10, // Adjust dimensions as per your embedding model
distanceFunction: VectorEmbeddingDistanceFunction.Cosine,
},
{
path: "/dimensionsVector",
dataType: VectorEmbeddingDataType.Float32,
dimensions: 4, // weight, width, height, depth
distanceFunction: VectorEmbeddingDistanceFunction.Euclidean, // Suitable for numeric vectors
},
],
};
const indexingPolicy: IndexingPolicy = {
vectorIndexes: [
{ path: "/imageVector", type: VectorIndexType.QuantizedFlat },
{ path: "/descriptionVector", type: VectorIndexType.DiskANN },
{ path: "/tagsVector", type: VectorIndexType.DiskANN },
{ path: "/featuresVector", type: VectorIndexType.DiskANN },
{ path: "/dimensionsVector", type: VectorIndexType.DiskANN },
],
includedPaths: [
{
path: "/*",
},
],
excludedPaths: [
{
path: "/_etag/?"
},
{
path: "/_rid/?"
},
{
path: "/_etag/?"
},
{
path: "/_attachments/?"
},
{
path: "/_ts/?"
},
{
path: "/imageVector/*",
},
{
path: "/descriptionVector/*",
},
{
path: "/tagsVector/*",
},
{
path: "/featuresVector/*",
},
{
path: "/dimensionsVector/*",
},
]
};
const { container } = await database.containers.createIfNotExists({
id: containerId,
partitionKey: {
paths: ["/id"],
version: PartitionKeyDefinitionVersion.V2,
kind: PartitionKeyKind.Hash,
},
vectorEmbeddingPolicy: vectorEmbeddingPolicy,
indexingPolicy: indexingPolicy,
});
return container;
}
// Getters for the containers
private getContainer(container: Container): Container {
if (!this.initialized) {
throw new Error('CosmosDB has not been initialized. Call initialize() first.');
}
return container;
}
public getProductsContainer(): Container {
return this.getContainer(this.productsContainer);
}
}
export default CosmosDB;
This service ensures a singleton instance of CosmosDB, initializes the database and containers with the necessary vector policies and indexes.
Defining Product Schema
src/schemas/productSchema.ts
:
//src/schemas/productSchema.ts
import { z } from 'zod';
export const productSchema = z.object({
id: z.string().uuid().optional(),
name: z.string(),
brand: z.string(),
sku: z.string(),
category: z.string(),
price: z.number(),
currency: z.string(),
stock: z.number(),
description: z.string(),
features: z.string(),
rating: z.number(),
reviewsCount: z.number(),
tags: z.array(z.string()),
imageUrl: z.string().url(),
manufacturer: z.string(),
model: z.string(),
releaseDate: z.string(),
warranty: z.string(),
dimensions: z.object({
weight: z.string(),
width: z.string(),
height: z.string(),
depth: z.string(),
}).optional(),
color: z.string(),
material: z.string(),
origin: z.string(),
descriptionVector: z.array(z.number()).optional(),
imageVector: z.array(z.number()).optional(),
tagsVector: z.array(z.number()).optional(),
featuresVector: z.array(z.number()).optional(),
dimensionsVector: z.array(z.number()).optional(),
});
export const productIDSchema = z.object({
id: z.string().uuid(),
});
export const tagsSchema = z.object({
tags: z.array(z.string()),
});
export const featuresSchema = z.object({
tags: z.array(z.string()),
});
export type Product = z.infer<typeof productSchema>;
export type ProductID = z.infer<typeof productIDSchema>;
export type Tags = z.infer<typeof tagsSchema>;
Using Zod for schema validation ensures that incoming data adheres to the expected structure.
Configuring Environment Variables
src/config/env.ts
:
// src/config/env.ts
import dotenv from 'dotenv';
import { z } from 'zod';
// Load environment variables from .env file
dotenv.config();
// Define the schema for environment variables using zod
const EnvSchema = z.object({
PORT: z.coerce.number().default(8000), // Set default to 8000
AZURE_COSMOS_DB_ENDPOINT: z.string({
required_error: "AZURE_COSMOS_DB_ENDPOINT is required",
invalid_type_error: "AZURE_COSMOS_DB_ENDPOINT must be a string",
}),
AZURE_COSMOS_DB_KEY: z.string({
required_error: "AZURE_COSMOS_DB_KEY is required",
invalid_type_error: "AZURE_COSMOS_DB_KEY must be a string",
}),
AZURE_COSMOS_DB: z.string({
required_error: "AZURE_COSMOS_DB is required",
invalid_type_error: "AZURE_COSMOS_DB must be a string",
}),
AZURE_OPENAI_API_KEY: z.string({
required_error: "AZURE_OPENAI_API_KEY is required",
invalid_type_error: "AZURE_OPEN_AI_API_KEY must be a string",
}),
AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT: z.string({
required_error: "AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT is required",
invalid_type_error: "AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT must be a string",
}),
});
// Parse and validate the environment variables
export const env = EnvSchema.parse(process.env);
// Configuration object consolidating all settings
const config = {
port: env.PORT,
cosmos: {
endpoint: env.AZURE_COSMOS_DB_ENDPOINT,
key: env.AZURE_COSMOS_DB_KEY,
database: env.AZURE_COSMOS_DB,
},
azureOpenAi: {
apiKey: env.AZURE_OPENAI_API_KEY,
models: {
embedding: {
endPoint: env.AZURE_OPENAI_TEXT_EMBEDDING_MODEL_ENDPOINT
},
},
},
};
export default config;
This configuration ensures that all necessary environment variables are set and accessible throughout the application.
Setting Up the Express Application
src/app.ts
:
//
import express, { Application } from 'express';
import cors from 'cors';
import helmet from 'helmet';
import requestLogger from './middlewares/log';
import productRouter from './routes/productsRoutes';
import { errorHandler } from './middlewares/errorHandler';
import limiter from './middlewares/rateLimiterHandler';
// Create Express server
const app: Application = express();
// CORS options
const corsOptions = {
origin: 'http://your-allowed-origin.com', // Replace with your allowed origin(s)
methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
allowedHeaders: ['Content-Type'], // Add headers you want to allow ie 'Authorization'
// credentials: true, // Enable if you need to pass cookies or authentication tokens
};
// Middlewares
app.use(cors(corsOptions)); // Enable CORS
app.use(express.json()); // Parse JSON bodies
app.use(express.urlencoded({ extended: true })); // Parse URL-encoded bodies
app.use(helmet()); // Secure your Express apps by setting various HTTP headers
app.use(requestLogger); // HTTP request logger
app.use(limiter); // Rate limiter middleware to limit(Limit each IP to 100 requests per 1 minutes)
app.use('/api', productRouter);
app.use(errorHandler); // Error handler middleware( should be the last middleware)
export default app;
This sets up the Express application with necessary middlewares and routes.
Testing the API
Testing dummy data:
{
"name": "Wireless Bluetooth Headphones",
"brand": "SoundMax",
"sku": "SKU-1001",
"category": "Electronics",
"price": 149.99,
"currency": "USD",
"stock": 250,
"description": "Experience unparalleled sound quality with SoundMax Wireless Bluetooth Headphones. Featuring active noise cancellation, 30-hour battery life, and a comfortable over-ear design.",
"features": "Bluetooth 5.0, Active Noise Cancellation, 30-hour Battery Life, Over-Ear Comfort, Built-in Microphone, Foldable Design",
"rating": 4.5,
"reviewsCount": 1200,
"tags": ["audio", "wireless", "headphones", "electronics", "noise-cancelling"],
"imageUrl": "https://example.com/images/products/SKU-1001.jpg",
"manufacturer": "SoundMax Inc.",
"model": "SM-BT1001",
"releaseDate": "2023-05-15",
"warranty": "2 years",
"dimensions": {
"weight": "250g",
"width": "20cm",
"height": "18cm",
"depth": "7cm"
},
"color": "Black",
"material": "Plastic and Metal",
"origin": "USA"
},
{
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
"category": "Home Automation",
"price": 19.99,
"currency": "USD",
"stock": 500,
"description": "Illuminate your home with BrightHome Smart LED Light Bulbs. Control brightness and color temperature via smartphone or voice commands.",
"features": "Wi-Fi Enabled, Voice Control Compatible, Energy Efficient, Adjustable Brightness, Long Lifespan, Easy Installation",
"rating": 4.3,
"reviewsCount": 850,
"tags": ["lighting", "smart home", "LED", "energy-efficient", "voice-control"],
"imageUrl": "https://example.com/images/products/SKU-1002.jpg",
"manufacturer": "BrightHome Technologies",
"model": "BH-SMART-LB",
"releaseDate": "2023-07-20",
"warranty": "1 year",
"dimensions": {
"weight": "75g",
"width": "6.5cm",
"height": "11cm",
"depth": "6.5cm"
},
"color": "White",
"material": "Glass and Plastic",
"origin": "Germany"
},
{
"name": "4K Ultra HD Action Camera",
"brand": "AdventureCam",
"sku": "SKU-1003",
"category": "Photography",
"price": 249.99,
"currency": "USD",
"stock": 150,
"description": "Capture every adventure in stunning 4K Ultra HD with AdventureCam Action Cameras. Waterproof, shockproof, and built for extreme conditions.",
"features": "4K Video Recording, Waterproof up to 30m, Wi-Fi Connectivity, Image Stabilization, Wide-Angle Lens, Long Battery Life",
"rating": 4.6,
"reviewsCount": 620,
"tags": ["camera", "action", "4K", "waterproof", "photography"],
"imageUrl": "https://example.com/images/products/SKU-1003.jpg",
"manufacturer": "AdventureCam LLC",
"model": "AC-4KUHD",
"releaseDate": "2023-03-10",
"warranty": "2 years",
"dimensions": {
"weight": "180g",
"width": "7cm",
"height": "5cm",
"depth": "4cm"
},
"color": "Red",
"material": "Aluminum and Plastic",
"origin": "China"
},
{
"name": "Ergonomic Office Chair",
"brand": "ComfortSeat",
"sku": "SKU-1004",
"category": "Furniture",
"price": 199.99,
"currency": "USD",
"stock": 80,
"description": "Upgrade your workspace with the ComfortSeat Ergonomic Office Chair. Designed for maximum support and comfort during long working hours.",
"features": "Adjustable Height, Lumbar Support, Breathable Mesh Back, Reclining Function, Swivel Base, Armrest Adjustment",
"rating": 4.7,
"reviewsCount": 430,
"tags": ["furniture", "office", "ergonomic", "chair", "comfort"],
"imageUrl": "https://example.com/images/products/SKU-1004.jpg",
"manufacturer": "ComfortSeat Corp.",
"model": "CS-ERGONOMIC-01",
"releaseDate": "2023-01-25",
"warranty": "3 years",
"dimensions": {
"weight": "12kg",
"width": "70cm",
"height": "120cm",
"depth": "70cm"
},
"color": "Gray",
"material": "Mesh and Metal",
"origin": "Sweden"
},
{
"name": "Stainless Steel Blender",
"brand": "KitchenPro",
"sku": "SKU-1005",
"category": "Kitchen Appliances",
"price": 89.99,
"currency": "USD",
"stock": 300,
"description": "Blend smoothies, soups, and more with the KitchenPro Stainless Steel Blender. Powerful motor and durable blades for all your blending needs.",
"features": "1000W Motor, 1.5L Capacity, Stainless Steel Blades, Multiple Speed Settings, Pulse Function, Easy Clean",
"rating": 4.4,
"reviewsCount": 920,
"tags": ["kitchen", "blender", "appliances", "stainless steel", "smoothies"],
"imageUrl": "https://example.com/images/products/SKU-1005.jpg",
"manufacturer": "KitchenPro Industries",
"model": "KP-BLEND-1000",
"releaseDate": "2023-04-18",
"warranty": "2 years",
"dimensions": {
"weight": "3kg",
"width": "15cm",
"height": "40cm",
"depth": "15cm"
},
"color": "Silver",
"material": "Stainless Steel and Plastic",
"origin": "Italy"
},
{
"name": "Fitness Tracker Watch",
"brand": "HealthMate",
"sku": "SKU-1006",
"category": "Wearables",
"price": 59.99,
"currency": "USD",
"stock": 400,
"description": "Monitor your health and fitness goals with the HealthMate Fitness Tracker Watch. Features heart rate monitoring, step counting, and sleep tracking.",
"features": "Heart Rate Monitor, Step Counter, Sleep Tracking, Waterproof, Smartphone Notifications, Long Battery Life",
"rating": 4.2,
"reviewsCount": 1500,
"tags": ["fitness", "tracker", "watch", "wearables", "health"],
"imageUrl": "https://example.com/images/products/SKU-1006.jpg",
"manufacturer": "HealthMate LLC",
"model": "HM-FIT-2023",
"releaseDate": "2023-02-05",
"warranty": "1 year",
"dimensions": {
"weight": "50g",
"width": "4cm",
"height": "4cm",
"depth": "1cm"
},
"color": "Blue",
"material": "Silicone and Plastic",
"origin": "South Korea"
},
{
"name": "Portable Power Bank 20000mAh",
"brand": "ChargeUp",
"sku": "SKU-1007",
"category": "Mobile Accessories",
"price": 39.99,
"currency": "USD",
"stock": 600,
"description": "Stay powered on the go with the ChargeUp Portable Power Bank. High-capacity 20000mAh battery ensures your devices stay charged throughout the day.",
"features": "20000mAh Capacity, Dual USB Ports, Fast Charging, LED Indicator, Lightweight Design, Durable Build",
"rating": 4.5,
"reviewsCount": 2100,
"tags": ["power bank", "portable", "charging", "mobile", "accessories"],
"imageUrl": "https://example.com/images/products/SKU-1007.jpg",
"manufacturer": "ChargeUp Technologies",
"model": "CU-PB20000",
"releaseDate": "2023-06-12",
"warranty": "1 year",
"dimensions": {
"weight": "500g",
"width": "7cm",
"height": "18cm",
"depth": "7cm"
},
"color": "Black",
"material": "Aluminum and Plastic",
"origin": "Japan"
},
{
"name": "Smart Thermostat",
"brand": "HomeSmart",
"sku": "SKU-1008",
"category": "Home Automation",
"price": 129.99,
"currency": "USD",
"stock": 180,
"description": "Optimize your home's temperature with the HomeSmart Smart Thermostat. Features remote control, scheduling, and energy usage reports.",
"features": "Remote Control via App, Scheduling, Energy Usage Reports, Voice Control Compatible, Easy Installation, Sleek Design",
"rating": 4.6,
"reviewsCount": 760,
"tags": ["thermostat", "smart home", "energy-efficient", "automation", "control"],
"imageUrl": "https://example.com/images/products/SKU-1008.jpg",
"manufacturer": "HomeSmart Inc.",
"model": "HS-THERMO-2023",
"releaseDate": "2023-08-01",
"warranty": "2 years",
"dimensions": {
"weight": "300g",
"width": "10cm",
"height": "15cm",
"depth": "3cm"
},
"color": "White",
"material": "Plastic and Metal",
"origin": "USA"
},
{
"name": "Gaming Mechanical Keyboard",
"brand": "GameMaster",
"sku": "SKU-1009",
"category": "Computer Accessories",
"price": 109.99,
"currency": "USD",
"stock": 220,
"description": "Enhance your gaming experience with the GameMaster Mechanical Keyboard. Features customizable RGB lighting, tactile switches, and durable build.",
"features": "Mechanical Switches, RGB Backlighting, Programmable Keys, N-Key Rollover, USB Passthrough, Detachable Wrist Rest",
"rating": 4.8,
"reviewsCount": 980,
"tags": ["keyboard", "gaming", "mechanical", "RGB", "computer"],
"imageUrl": "https://example.com/images/products/SKU-1009.jpg",
"manufacturer": "GameMaster Technologies",
"model": "GM-MK1009",
"releaseDate": "2023-09-10",
"warranty": "3 years",
"dimensions": {
"weight": "900g",
"width": "45cm",
"height": "5cm",
"depth": "15cm"
},
"color": "Black",
"material": "Aluminum and Plastic",
"origin": "Taiwan"
},
{
"name": "Wireless Charging Pad",
"brand": "ChargeWave",
"sku": "SKU-1010",
"category": "Mobile Accessories",
"price": 29.99,
"currency": "USD",
"stock": 450,
"description": "Conveniently charge your devices with the ChargeWave Wireless Charging Pad. Compatible with all Qi-enabled devices and features fast charging capabilities.",
"features": "Qi-Compatible, Fast Charging, Non-Slip Surface, LED Indicator, Slim Design, Overcharge Protection",
"rating": 4.3,
"reviewsCount": 1340,
"tags": ["charging", "wireless", "mobile", "accessories", "fast-charging"],
"imageUrl": "https://example.com/images/products/SKU-1010.jpg",
"manufacturer": "ChargeWave Ltd.",
"model": "CW-WCP300",
"releaseDate": "2023-07-05",
"warranty": "1 year",
"dimensions": {
"weight": "200g",
"width": "10cm",
"height": "1cm",
"depth": "10cm"
},
"color": "White",
"material": "Plastic",
"origin": "South Korea"
}
With the API fully set up, let’s explore how to interact with it using various endpoints.
Creating a Product
Endpoint: POST http://localhost:8000/api/products
Headers:
Content-Type: application/json
Request Body:
{
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
"category": "Home Automation",
"price": 19.99,
"currency": "USD",
"stock": 500,
"description": "Illuminate your home with BrightHome Smart LED Light Bulbs. Control brightness and color temperature via smartphone or voice commands.",
"features": "Wi-Fi Enabled, Voice Control Compatible, Energy Efficient, Adjustable Brightness, Long Lifespan, Easy Installation",
"rating": 4.3,
"reviewsCount": 850,
"tags": [
"lighting",
"smart home",
"LED",
"energy-efficient",
"voice-control"
],
"imageUrl": "https://example.com/images/products/SKU-1002.jpg",
"manufacturer": "BrightHome Technologies",
"model": "BH-SMART-LB",
"releaseDate": "2023-07-20",
"warranty": "1 year",
"dimensions": {
"weight": "75g",
"width": "6.5cm",
"height": "11cm",
"depth": "6.5cm"
},
"color": "White",
"material": "Glass and Plastic",
"origin": "Germany"
}
Response:
{
"status": "success",
"data": {
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f",
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
"category": "Home Automation",
"price": 19.99,
"currency": "USD",
"stock": 500,
"description": "Illuminate your home with BrightHome Smart LED Light Bulbs. Control brightness and color temperature via smartphone or voice commands.",
"features": "Wi-Fi Enabled, Voice Control Compatible, Energy Efficient, Adjustable Brightness, Long Lifespan, Easy Installation",
"rating": 4.3,
"reviewsCount": 850,
"tags": [
"lighting",
"smart home",
"LED",
"energy-efficient",
"voice-control"
],
"imageUrl": "https://example.com/images/products/SKU-1002.jpg",
"manufacturer": "BrightHome Technologies",
"model": "BH-SMART-LB",
"releaseDate": "2023-07-20",
"warranty": "1 year",
"dimensions": {
"weight": "75g",
"width": "6.5cm",
"height": "11cm",
"depth": "6.5cm"
},
"color": "White",
"material": "Glass and Plastic",
"origin": "Germany",
"descriptionVector": [/* generated vector */],
"tagsVector": [/* generated vector */],
"featuresVector": [/* generated vector */]
}
}
Upon creation, the API generates vector embeddings for the description, tags, and features, storing them alongside the product data.
Retrieving Products
Endpoint: GET http://localhost:8000/api/products
Headers:
Accept: application/json
Response:
{
"status": "success",
"data": [
{
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f",
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
"category": "Home Automation",
"price": 19.99,
"currency": "USD",
"stock": 500,
"description": "Illuminate your home with BrightHome Smart LED Light Bulbs. Control brightness and color temperature via smartphone or voice commands.",
"features": "Wi-Fi Enabled, Voice Control Compatible, Energy Efficient, Adjustable Brightness, Long Lifespan, Easy Installation",
"rating": 4.3,
"reviewsCount": 850,
"tags": [
"lighting",
"smart home",
"LED",
"energy-efficient",
"voice-control"
],
"imageUrl": "https://example.com/images/products/SKU-1002.jpg",
"manufacturer": "BrightHome Technologies",
"model": "BH-SMART-LB",
"releaseDate": "2023-07-20",
"warranty": "1 year",
"dimensions": {
"weight": "75g",
"width": "6.5cm",
"height": "11cm",
"depth": "6.5cm"
},
"color": "White",
"material": "Glass and Plastic",
"origin": "Germany",
"descriptionVector": [/* vector */],
"tagsVector": [/* vector */],
"featuresVector": [/* vector */]
},
// More products...
]
}
Performing Vector Similarity Searches
Vector similarity searches allow users to find products that are semantically similar to their queries. Below are examples of how to perform such searches.
Search by Tags Vector
Endpoint: POST http://localhost:8000/api/products/search/tags
Headers:
Content-Type: application/json
Request Body:
{
"queryVector": ["lighting", "smart home"],
"top": 5
}
Response:
{
"status": "success",
"data": [
{
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f",
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
// Other product details...
"SimilarityScore": 0.95
},
// Top 5 similar products...
]
}
Search by Features Vector
Endpoint: POST http://localhost:8000/api/products/search/features
Headers:
Content-Type: application/json
Request Body:
{
"queryVector": ["Wi-Fi Enabled", "Voice Control"],
"top": 2
}
Response:
{
"status": "success",
"data": [
{
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f",
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
// Other product details...
"SimilarityScore": 0.92
},
// Top 2 similar products...
]
}
Search by Description Vector
Endpoint: POST http://localhost:8000/api/products/search/description
Headers:
Content-Type: application/json
Request Body:
{
"queryVector": ["energy efficient smart lighting"],
"top": 10
}
Response:
{
"status": "success",
"data": [
{
"id": "a2555d74-686a-4c34-af2b-c63fa2b9ed2f",
"name": "Smart LED Light Bulb",
"brand": "BrightHome",
"sku": "SKU-1002",
// Other product details...
"SimilarityScore": 0.98
},
// Top 10 similar products...
]
}
These searches utilize the vector embeddings to retrieve products that closely match the semantic intent of the query, providing more relevant results compared to traditional keyword searches.
Conclusion
Integrating vector similarity search into your e-commerce API using Azure Cosmos DB enhances the search functionality by understanding the semantic relationships between products and user queries. This leads to improved user satisfaction, higher engagement, and increased sales. By leveraging Azure’s robust infrastructure and advanced embedding techniques, you can stay ahead in the competitive e-commerce landscape.
Key Takeaways:
- Vector Embeddings: Transforming textual and categorical data into high-dimensional vectors enables nuanced similarity comparisons.
- Azure Cosmos DB Vector Search: Offers efficient indexing and querying capabilities tailored for vector data.
- API Integration: Seamlessly integrating vector search into your API enhances the overall search experience.
- Scalability: Azure Cosmos DB ensures that your vector search capabilities scale with your growing product catalog.
Embark on implementing vector similarity search in your e-commerce platform to deliver smarter and more intuitive search experiences for your users.
Read More
- What is Azure OpenAI Service?
- Azure OpenAI Service models
- Tutorial: Explore Azure OpenAI Service embeddings and document search
- Index and query vectors in Azure Cosmos DB for NoSQL in JavaScript
- What is a vector database
- Vector database in Azure Cosmos DB for NoSQL
- Vector database in Azure Cosmos DB for MongoDB
- Announcing JavaScript SDK v4 for Azure Cosmos DB
Microsoft Tech Community – Latest Blogs –Read More
Use Microsoft Phi-3.5 to complete text chunking
The introduction of Generative AI has accelerated the transformation of more enterprises into AI. How to make LLM better understand the business of enterprises or different scenarios, data is the key. RAG is the most common method now. Limited by the LLM/SLM token, we need to chunking text data. How to maintain the relevance of text context is a technical problem. Microsoft Phi-3.5 supports 128k tokens ,image analysis capabilities and has strong long context understanding capabilities. This blog attempts to use Microsoft Phi-3.5 Family as the core of text chunking to improve the effectiveness and coherence of chunking content.
Starting from the text content
We need to understand the text structure. From daily news, contracts, or papers, they all contain three parts: text, images, and tables. These three parts allow readers to understand the content. How to extract these contents? From a technical perspective, we can combine Python and different AI technologies. For text and images, pypdf can be used, and for tables, Azure Document Intelligence can be used to obtain more accurate content.
The role of Microsoft Phi-3.5
We have split the document into three parts: text, pictures, and tables. We start to use Microsoft Phi-3.5 to understand these three parts.
- Text – We need to divide the text content into knowledge content, which is more conducive to information retrieval. Microsoft Phi-3.5-mini-instruct as a text content reader, summarize and divide the information points. Prompt is as follows:
You are an expert in content chunking. Please help me chunk user's input text according to the following requirements
1. Truncate the text content into chunks of no more than 300 tokens.
2. Each chunk part should maintain contextual coherence. The truncated content should be retained in its entirety without any additions or modifications.
3. Each chunked part is output JSON format { "chunking": "..." }
4. The final output is a JSON array [{ "chunking" : "..." },{ "chunking" :"..."},{ "chunking" : "..."} ....]
- Images – Images are presented in conjunction with text content. We can use Microsoft Phi-3.5-vision to understand the content of each image in the text. The prompt is as follows:
You are my analysis assistant, help me analyze charts, flowchart, etc. according to the following conditions
1. If it is a chart, please analyze according to the data in the chart and tell me the different details
2. If it is a flowchart, please analyze all the situations in detail according to the flow and describe all process in details, do NOT simplify. Use bullet lists with identation to describe the process
3. The output is json {"chunking":"......"}
4. If it is not a chart or flowchart(more than single node),it does not need to be analyzed, the output is json {"chunking":"NIL"}
- Table – Table is an important content. Through Microsoft Phi-3.5-mini-instruct, you can analyze the data of each table in the content and find the development trend. The prompt is as follows:
You are my markdown table assistant, who can understand all the contents of the table and give analysis.
After we complete the processing of these three parts, we can integrate the data, combine it with the output JSON, and save it into the vector database through Embedding conversion. The vectorized storage of text content is completed.
Solution
We use Prompt flow combined with GitHub Models and Azure AI Search to complete the text chunking. As shown in the figure:
Note
We need to ensure that the output is JSON, but Phi-3.5-mini-instruct sometimes has errors, so check it through check_json
For different documents, different prompts need to be adjusted, otherwise there is no way to deal with a more accurate understanding of the document content
This solution combines Azure AI Search to complete vector storage, and different vector databases can be switched
GitHub Models allows us to have more effective calls during the development and verification phase, but it is recommended to switch to Azure or locally deployed models in the production environment.
We can verify the relevant results through Chat-flow
Download the sample code – Click here
Summary
How to complete text chunking more efficiently is an academic issue, and SLM may be one of the methods to solve it. However, we need to do more work to address the diversity of texts, and there is still a lot of room for improvement in this attempt.
Resources
Learn more about Microsoft Phi-3.5 https://huggingface.co/microsoft/Phi-3.5-mini-instruct
Read Microsoft Phi-3 Cookbook https://aka.ms/phi-3cookbook
GitHub Models https://gh.io/models
Learn more about Microsoft Prompt flow https://microsoft.github.io/promptflow/
Learn more about Microsoft AI Search https://learn.microsoft.com/en-us/azure/search/search-what-is-azure-search
Microsoft Tech Community – Latest Blogs –Read More
欧亚国际开户-17300435119
SQL Server 采用客户端 / 服务器架构,客户端应用程序通过网络连接与 SQL Server 服务器进行通信。客户端可以是各种类型的应用程序,如桌面应用程序、Web 应用程序、移动应用程序等,服务器端负责接收客户端的请求,执行相应的数据库操作,并将结果返回给客户端。服务器端由多个组件组成,包括 SQL Server 引擎、存储引擎、网络协议栈等。
SQL Server 采用客户端 / 服务器架构,客户端应用程序通过网络连接与 SQL Server 服务器进行通信。客户端可以是各种类型的应用程序,如桌面应用程序、Web 应用程序、移动应用程序等,服务器端负责接收客户端的请求,执行相应的数据库操作,并将结果返回给客户端。服务器端由多个组件组成,包括 SQL Server 引擎、存储引擎、网络协议栈等。 Read More
What is 55 Club Invite Code? 127464426126 (Claim Bonus)
127464426126 is your 55 Club Invite Code which you need to apply at the time of creating your new account on 55 Club.
55 Club is a popular online platform where people can make bets on color predictions and try to win rewards. It is very simple, users guess which colors will appear in the next round.And if their guess matches the outcome they will get reward based on their bets.
55 Club gives bonuses and rewards to their new users, giving new users a reason to try it Just remember that betting involves risk, so it’s important to play responsibly and not to put high bets at the start.
55 Club Invite Code 2024
App Name | 55 Club |
55 Club Invite Code | 127464426126 |
Signup Bonus | Rs 200 |
55 Club Invitation Code | 127464426126 |
55 Club Gift Code | 127464426126 |
55 Club Invitation Code
Your 55 Club Invitation Code is 127464426126 . The 55 Club Invitation Code is a special code that gives you rewards and bonuses when you sign up on the 55 Club platform.
It’s an easy way to get extra benefits, especially if you’re new to online color prediction or betting. By entering this invitation code during registration, you can start with some bonus amount, which may help you try more games or make your first bets without spending too much.
How to Use the 55 Club Invite Code
Using the 55 Club invite code during registration is simple, and it gives some extra rewards for you as a new user. Here’s how you can claim the bonus:
- Download and Open the 55 Club App.
- Begin Registration.
- Enter the Invite Code.
- Complete Registration.
127464426126 is your 55 Club Invite Code which you need to apply at the time of creating your new account on 55 Club. 55 Club is a popular online platform where people can make bets on color predictions and try to win rewards. It is very simple, users guess which colors will appear in the next round.And if their guess matches the outcome they will get reward based on their bets.55 Club gives bonuses and rewards to their new users, giving new users a reason to try it Just remember that betting involves risk, so it’s important to play responsibly and not to put high bets at the start.55 Club Invite Code 2024 App Name55 Club55 Club Invite Code127464426126 Signup BonusRs 20055 Club Invitation Code127464426126 55 Club Gift Code127464426126 55 Club Invitation CodeYour 55 Club Invitation Code is 127464426126 . The 55 Club Invitation Code is a special code that gives you rewards and bonuses when you sign up on the 55 Club platform. It’s an easy way to get extra benefits, especially if you’re new to online color prediction or betting. By entering this invitation code during registration, you can start with some bonus amount, which may help you try more games or make your first bets without spending too much.How to Use the 55 Club Invite CodeUsing the 55 Club invite code during registration is simple, and it gives some extra rewards for you as a new user. Here’s how you can claim the bonus:Download and Open the 55 Club App.Begin Registration.Enter the Invite Code.Complete Registration. Read More
How to Mirror iPhone to PC on Windows 11?
Hello everyone,
I got a new iPhone 15 Pro Max and I’m trying to find a reliable way to mirror my iPhone’s screen onto my PC running Windows 11. I’ve seen several methods online, but I’m not sure which ones are the most effective and user-friendly. Ideally, I’m looking for a solution that offers high-quality mirroring without too much lag, and it would be great if it also supports audio transmission.
Has anyone here successfully mirrored their iPhone to a Windows 11 PC? I’m open to both free and paid solutions, as long as they are reliable and straightforward to use. Thank you in advance for your suggestions!
Hello everyone,I got a new iPhone 15 Pro Max and I’m trying to find a reliable way to mirror my iPhone’s screen onto my PC running Windows 11. I’ve seen several methods online, but I’m not sure which ones are the most effective and user-friendly. Ideally, I’m looking for a solution that offers high-quality mirroring without too much lag, and it would be great if it also supports audio transmission. Has anyone here successfully mirrored their iPhone to a Windows 11 PC? I’m open to both free and paid solutions, as long as they are reliable and straightforward to use. Thank you in advance for your suggestions! Read More