Previously, I shared with you how to use Phi-3-mini on AIPC’s NPU and iPhone. Some people want to know more about the experience of using macOS and how to use Apple Silicon to accelerate SLM models. This blog will share with you relevant knowledge, including how to use Apple MLX Framework to accelerate Phi-3-mini operation, fine-tune, and combine Llama.cpp for quantitative operation.

What’s MLX Framework

MLX is an array framework for machine learning research on Apple silicon, brought to you by Apple machine learning research.

MLX is designed by machine learning researchers for machine learning researchers. The framework is intended to be user-friendly, but still efficient to train and deploy models. The design of the framework itself is also conceptually simple. We intend to make it easy for researchers to extend and improve MLX with the goal of quickly exploring new ideas.

LLMs can be accelerated in Apple Silicon devices through MLX, and models can be run locally very conveniently.

Installation

Installing MLX is easy, you will need Python 3.11.x+, then install it in the terminal

pip install mlx-lm

Run MLX’s instructions

1. Running Phi-3-mini in Terminal with MLX

python -m mlx_lm.generate –model microsoft/Phi-3-mini-4k-instruct –max-token 2048 –prompt “<|user|>nCan you introduce yourself<|end|>n<|assistant|>”

2. Quantizing Phi-3-mini with MLX in Terminal

python -m mlx_lm.convert –hf-path microsoft/Phi-3-mini-4k-instruct

3. Running Phi-3-mini with MLX in Jupyter Notebook

Note: Please read Inference Phi-3 with Apple MLX Framework to Learn more

Fine-tuning with MLX Framework

We generally need GPU acceleration to complete model training or fine-tuning, but in Apple devices you can use Apple silicon’s MPS(Metal Performance Shaders) to replace the GPU to complete model training and fine-tuning.

What’s Metal Performance Shaders

The Metal Performance Shaders framework contains a collection of highly optimized compute and graphics shaders that are designed to integrate easily and efficiently into your Metal app. These data-parallel primitives are specially tuned to take advantage of the unique hardware characteristics of each GPU family to ensure optimal performance.

Sample – Using LoRA to fine-tuning Phi-3-mini with MLX

1. Data preparation

By default, MLX Framework requires the jsonl format of train, test, and eval, and is combined with Lora to complete fine-tuning jobs.

Note:

jsonl data format ：

{“text”: “<|user|>nWhen were iron maidens commonly used? <|end|>n<|assistant|> nIron maidens were never commonly used <|end|>”}
{“text”: “<|user|>nWhat did humans evolve from? <|end|>n<|assistant|> nHumans and apes evolved from a common ancestor <|end|>”}
{“text”: “<|user|>nIs 91 a prime number? <|end|>n<|assistant|> nNo, 91 is not a prime number <|end|>”}
….

Our example uses TruthfulQA’s data , but the amount of data is relatively insufficient, so the fine-tuning results are not necessarily the best. It is recommended that learners use better data based on their own scenarios to complete.

The data format is combined with the Phi-3 template

Please download data from this link , please inculde all .jsonl in data folder

2. Fine-tuning in your terminal

Please run this command in terminal

python -m mlx_lm.lora –model microsoft/Phi-3-mini-4k-instruct –train –data ./data –iters 1000

Note: This is LoRA fine-tuning, MLX framework not published QLoRA

3. Run Fine-tuning adapter to test

You can run fine-tuning adapter in terminal,like this

python -m mlx_lm.generate –model microsoft/Phi-3-mini-4k-instruct –adapter-path ./adapters –max-token 2048 –prompt “Why do chameleons change colors? “ –eos-token “<|end|>”

and run original model to compare result

python -m mlx_lm.generate –model microsoft/Phi-3-mini-4k-instruct –max-token 2048 –prompt “Why do chameleons change colors? “ –eos-token “<|end|>”

You can try to compare the results of Fine-tuning with the original model

4. Merge adapters to generate new models

python -m mlx_lm.fuse –model microsoft/Phi-3-mini-4k-instruct

5. Running quantified fine-tuning models using ollama

Before use, please configure your llama.cpp environment

git clone https://github.com/ggerganov/llama.cpp.git

cd llama.cpp

pip install -r requirements.txt

python convert.py ‘Your meger model path’ –outfile phi-3-mini-ft.gguf –outtype f16

Note:

Now supports quantization conversion of fp32, fp16 and INT 8

The merged model is missing tokenizer.model, please download it from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct

set Ollma Model file（If not install ollama ,please read [Ollama QuickStart](../02.QuickStart/Ollama_QuickStart.md）

FROM ./phi-3-mini-ft.gguf
PARAMETER stop “<|end|>”

run command in terminal

ollama create phi3ft -f Modelfile

ollama run phi3ft “Why do chameleons change colors?”

Note: Please read Fine-tuning Phi-3 with Apple MLX Framework to Learn more

Resources

Read Phi-3 CookBook https://aka.ms/phi-3cookbook
MLX framework Repo https://github.com/ml-explore
Learn more about MLX Framework https://ml-explore.github.io/mlx/
Hugging face Phi-3 Family https://huggingface.co/collections/microsoft/phi-3-6626e15e9585a200d2d761e3

​Microsoft Tech Community – Latest Blogs –Read More 

Now available on-demand, for free, is the recording of our Azure Webinar session “Empowering cloud efficiency through FinOps”.

Microsoft Cloud Solution Architect, Arthur Clares, explains short term & long term objectives with FinOps

FinOps Foundation Founder & Executive Director, J.R. Storment, and Microsoft speakers Sonia Cuff, Arthur Clares and Thomas Lewis, explain FinOps concepts and relevant Microsoft solutions and resources. You’ll learn about:

The FinOps operational framework and cultural practice, which maximises the business value of cloud;
Microsoft FinOps Interactive Guides, for exploring Microsoft solutions that enable FinOps Capabilities;
Quick wins for rapid cloud cost reduction;
How to prepare for Environmental, Social & Governance (ESG) regulations;
Microsoft resources to help you with your FinOps journey.

Demonstration of Microsoft’s FinOps Interactive Guides

To access the webinar recording, register here: Empowering cloud efficiency through FinOps Webinar

With thanks to J.R. Storment at the team at The FinOps Foundation.

​Microsoft Tech Community – Latest Blogs –Read More 

Project based on .NET Blazor Web App in Server render mode.

Blazor currently has no built-in function for utilizing local browser storage. However this feature can come in useful for storing user information or partially completed forms etc. Luckily implementing a class for facilitating this function is straightforward enough. Here I’ll run over the steps required to get local storage up and running in your Blazor web app.

Note: This method uses JSRuntime so make sure if you want to call it on page/component load you will need to run it on “OnAfterRender” rather than “OnInitialize” otherwise you will get an error.

Here is a screenshot of my file structure so you can see the additional files I’ve made and where I’ve placed them:

Making the javascript functions:

Within the wwwroot of your app, make a new javascript file (I’ve called mine LocalStorage.js) and enter the following code:

window.LocalStorageActions = {
setItem: function (key, value) {
localStorage.setItem(key, value);
},
getItem: function (key) {
return localStorage.getItem(key);
},
removeItem: function (key) {
localStorage.removeItem(key);
},
clearData: function () {
localStorage.clear();
}
}

These are the four functions we will need for setting, getting, removing and clearing local storage in the browser.

Make a C# class to call the functions:

To keep things tidy, I’ve put the C# class for calling the local storage functions and it’s interface into a folder called helpers, but you can put this wherever it’s most appropriate in your solution.

You can create a new .cs file and add the following code:

using Microsoft.AspNetCore.Components;
using Microsoft.JSInterop;

namespace BlazorLocalStorage.Helpers
{
public class LocalStorageHelper : ILocalStorageHelper
{
private readonly IJSRuntime _javaScript;

public LocalStorageHelper(IJSRuntime javaScript)
{
_javaScript = javaScript;
}

public async Task SetItem(string key, string value)
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.setItem”, key, value);
}

public async Task<string> GetItem(string key)
{
return await _javaScript.InvokeAsync<string>(“LocalStorageActions.getItem”, key);
}

public async Task RemoveItem(string key)
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.removeItem”, key);
}

public async Task ClearData()
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.clearData”);
}
}
}

To quickly produce the interface for this class you can right click the class name (LocalStorageHelper in this case) and select “Quick Actions and Refactoring” and then select “Extract Interface”

This will open a popup for extracting an interface from the class that you can select OK and a new file called ILocalStorageHelper.cs will appear and the LocalStorageHelper class will now inherit from the interface that’s been created.

Registering the new files ready for use:

Firstly we want to reference the new javascript file for use in the application. To do this go to the App.razor file and add the following line in the <body> section of the file.

<script src=”LocalStorage.js”></script>

Next, we need to register the LocalStorageHelper so it can be injected wherever we want to use it. To do this open the Program.cs file and add the following line before the builder.Build() method it called.

builder.Services.AddScoped<ILocalStorageHelper, LocalStorageHelper>();

Let’s try it out:

I’ve made a little example of the new class being used by modifying the Home.razor file to use the various functions we’ve created. Before adding the main code I have created a little css class to centre the content of the page to keep things neat. So you can add the following code snippet to app.css.

.center-screen {

display: flex;

justify-content: center;

align-items: center;

text-align: center;

min-height: 70vh;

}

Now we can go into the Home.razor file and replace the content with the following.

@PAGE “/”
@using BlazorLocalStorage.Helpers
@rendermode InteractiveServer

<PageTitle>Home</PageTitle>

<div class=”center-screen”>
<div>

<div class=”row”>
<label>Text-1</label>
<InputText class=”m-2″ @bind-Value=”Text1″ />
</div>

<div class=”row”>

<button class=”btn btn-success m-2″ @onclick=”Save”>Save</button>

<button class=”btn btn-info m-2″ @onclick=”Load”>Load</button>

<button class=”btn btn-warning m-2″ @onclick=”Remove”>Remove</button>

<button class=”btn btn-danger m-2″ @onclick=”Clear”>Clear</button>

</div>

</div>
</div>

@code {
[Inject] ILocalStorageHelper LocalStorage { get; set; }

public string Text1 { get; set; }

public async Task Save()
{
await LocalStorage.SetItem(“textData”, Text1);
}

public async Task Load()
{
Text1 = await LocalStorage.GetItem(“textData”);
}

public async Task Remove()
{
await LocalStorage.RemoveItem(“textData”);
}

public async Task Clear()
{
await LocalStorage.ClearData();
}

}

Apologies for the garish use of coloured buttons but it’s just meant to be an obvious demo. If you open the browsers developer tools and navigate to “Application” the select “Local Storage” you will be able to see the data stored, updated or removed as you use the demo.

If you refresh the page or close and open the browser and select load it will pull the stored data back in to the textbox.

That’s all there is to it!

If you want to see the project, I’ve stuck a copy up on github that you can have a look at:

https://github.com/DrGav/BlazorLocalStorage 

Have yourselves a wonderful day,

Gavin.

​Project based on .NET Blazor Web App in Server render mode. Blazor currently has no built-in function for utilizing local browser storage. However this feature can come in useful for storing user information or partially completed forms etc. Luckily implementing a class for facilitating this function is straightforward enough. Here I’ll run over the steps required to get local storage up and running in your Blazor web app. Note: This method uses JSRuntime so make sure if you want to call it on page/component load you will need to run it on “OnAfterRender” rather than “OnInitialize” otherwise you will get an error. Here is a screenshot of my file structure so you can see the additional files I’ve made and where I’ve placed them:   Making the javascript functions:Within the wwwroot of your app, make a new javascript file (I’ve called mine LocalStorage.js) and enter the following code:  window.LocalStorageActions = {
setItem: function (key, value) {
localStorage.setItem(key, value);
},
getItem: function (key) {
return localStorage.getItem(key);
},
removeItem: function (key) {
localStorage.removeItem(key);
},
clearData: function () {
localStorage.clear();
}
}  These are the four functions we will need for setting, getting, removing and clearing local storage in the browser. Make a C# class to call the functions:To keep things tidy, I’ve put the C# class for calling the local storage functions and it’s interface into a folder called helpers, but you can put this wherever it’s most appropriate in your solution.You can create a new .cs file and add the following code:  using Microsoft.AspNetCore.Components;
using Microsoft.JSInterop;

namespace BlazorLocalStorage.Helpers
{
public class LocalStorageHelper : ILocalStorageHelper
{
private readonly IJSRuntime _javaScript;

public LocalStorageHelper(IJSRuntime javaScript)
{
_javaScript = javaScript;
}

public async Task SetItem(string key, string value)
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.setItem”, key, value);
}

public async Task<string> GetItem(string key)
{
return await _javaScript.InvokeAsync<string>(“LocalStorageActions.getItem”, key);
}

public async Task RemoveItem(string key)
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.removeItem”, key);
}

public async Task ClearData()
{
await _javaScript.InvokeAsync<string>(“LocalStorageActions.clearData”);
}
}
}  To quickly produce the interface for this class you can right click the class name (LocalStorageHelper in this case) and select “Quick Actions and Refactoring” and then select “Extract Interface”  This will open a popup for extracting an interface from the class that you can select OK and a new file called ILocalStorageHelper.cs will appear and the LocalStorageHelper class will now inherit from the interface that’s been created. Registering the new files ready for use:Firstly we want to reference the new javascript file for use in the application. To do this go to the App.razor file and add the following line in the <body> section of the file.  <script src=”LocalStorage.js”></script>  Next, we need to register the LocalStorageHelper so it can be injected wherever we want to use it. To do this open the Program.cs file and add the following line before the builder.Build() method it called.  builder.Services.AddScoped<ILocalStorageHelper, LocalStorageHelper>();  Let’s try it out:I’ve made a little example of the new class being used by modifying the Home.razor file to use the various functions we’ve created. Before adding the main code I have created a little css class to centre the content of the page to keep things neat. So you can add the following code snippet to app.css.  .center-screen {

display: flex;

justify-content: center;

align-items: center;

text-align: center;

min-height: 70vh;

}  Now we can go into the Home.razor file and replace the content with the following.  @PAGE “/”
@using BlazorLocalStorage.Helpers
@rendermode InteractiveServer

<PageTitle>Home</PageTitle>

<div class=”center-screen”>
<div>

<div class=”row”>
<label>Text-1</label>
<InputText class=”m-2″ @bind-Value=”Text1″ />
</div>

<div class=”row”>

<button class=”btn btn-success m-2″ @onclick=”Save”>Save</button>

<button class=”btn btn-info m-2″ @onclick=”Load”>Load</button>

<button class=”btn btn-warning m-2″ @onclick=”Remove”>Remove</button>

<button class=”btn btn-danger m-2″ @onclick=”Clear”>Clear</button>

</div>

</div>
</div>

@code {
[Inject] ILocalStorageHelper LocalStorage { get; set; }

public string Text1 { get; set; }

public async Task Save()
{
await LocalStorage.SetItem(“textData”, Text1);
}

public async Task Load()
{
Text1 = await LocalStorage.GetItem(“textData”);
}

public async Task Remove()
{
await LocalStorage.RemoveItem(“textData”);
}

public async Task Clear()
{
await LocalStorage.ClearData();
}

}  Apologies for the garish use of coloured buttons but it’s just meant to be an obvious demo. If you open the browsers developer tools and navigate to “Application” the select “Local Storage” you will be able to see the data stored, updated or removed as you use the demo.    If you refresh the page or close and open the browser and select load it will pull the stored data back in to the textbox. That’s all there is to it! If you want to see the project, I’ve stuck a copy up on github that you can have a look at:https://github.com/DrGav/BlazorLocalStorage  Have yourselves a wonderful day,Gavin.  Read More

​