The first step in developing generative AI applications is to choose a model. How to choose a model is the key. This includes

When we combine application development with business scenarios, there are many comparisons, such as the generation effects of the same prompt words under different models.
Quick comparison and switching of multiple models
How different models adapt to new application frameworks and solutions to complete projects more effectively.

The release of GitHub Models plays a very important role for developers and different development teams to more effectively select models in the process of developing applications and create applications based on different application frameworks. Let’s take a look at how I use GitHub Models to complete development in different scenarios.

Model comparison

In GitHub Models, through the provided playground, we can complete the comparison of the same prompt for different models.

Let’s take a look at the comparison between Phi-3-mini and Mistral Nemo

Judging from the results, this is an evenly matched result.

Quick comparison and switching of multiple models

Above, we switched models in the playground to compare different models under the same prompt. For development, a more direct approach may be required. With the Azure AI Inference SDK you can quickly switch to different models. You can choose Python, JavaScript, and REST access methods by selecting Code.

If we choose the Phi-3-mini scenario, we can choose to obtain the access method in Code

Of course, you can directly and seamlessly access the programming environment through Codespace.

Adaptation to different application frameworks

Generative AI has different application frameworks combined with models to complete applications, such as GraphRAG. We can use the REST interface provided by GitHub Models to test model solutions other than GPT-4o, such as selecting the latest Meta LLama 3.1 405b Instruct. If the local deployment of this model has been limited by computing power, it will be difficult for individuals and small teams to adopt it. But based on the interface provided by GitHub Models, we can complete the test in the local environment very simply

Configure the environment

Install the GraphRAG Python library

pip install graphrag -U

Create a GraphRAG project

mkdir -p ./ragmd/input

python -m graphrag.index –init –root ./ragmd

Modify settings.yaml

encoding_model: cl100k_base
skip_workflows: []
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_chat # or azure_openai_chat
model: meta-llama-3.1-405b-instruct
model_supports_json: true # recommended if this is available for your model.
max_tokens: 4000
api_base: https://models.inference.ai.azure.com

parallelization:
Stagger: 0.3

async_mode: threaded # or asyncio

embeddings:
async_mode: threaded # or asyncio
llm:
api_key: ${GRAPHRAG_API_KEY}
type: openai_embedding # or azure_openai_embedding
model: jinaai
api_base: http://localhost:5146/v1

Note Please configure GitHub Tokens in .env

Run

python -m graphrag.index –root ./ragmd

Test Results

python -m graphrag.query –root ./ragmd –method global “What’s GraphRAG”

Through GitHub Models, we can quickly use the provided models for model comparison and application development environment testing, which allows model and application testing to be completed more efficiently and quickly in environments with limited computing power.

Learning Resources

Sign Up https://gh.io/models

Introducing GitHub Models: A new generation of AI engineers building on GitHub https://github.blog/news-insights/product-news/introducing-github-models/

Understand Phi-3 https://aka.ms/phi-3cookbook

Learn about GraphRAG https://microsoft.github.io/

​Microsoft Tech Community – Latest Blogs –Read More