Category: News
How will i add another value (e.g. TLOG_MMF_COR_0003) in the below
Hi,
How will i add another value (e.g. TLOG_MMF_COR_0003) in the below ‘Type:=xlCaptionDoesNotBeginWith’ where existing
value1 ‘ATLOG_MMF_0003’ is there already:
=======================================================
ActiveSheet.PivotTables(“PivotTable9”).PivotFields(“USSMG_File-list”). _
PivotFilters.Add2 Type:=xlCaptionDoesNotBeginWith, Value1:=”ATLOG_MMF_0003“
Hi, How will i add another value (e.g. TLOG_MMF_COR_0003) in the below ‘Type:=xlCaptionDoesNotBeginWith’ where existingvalue1 ‘ATLOG_MMF_0003’ is there already:=======================================================ActiveSheet.PivotTables(“PivotTable9”).PivotFields(“USSMG_File-list”). _PivotFilters.Add2 Type:=xlCaptionDoesNotBeginWith, Value1:=”ATLOG_MMF_0003″ Read More
Removing an attachment strips non-Microsoft X-* headers from a message
I’ve run into an odd behavior that doesn’t seem to be documented. When I delete an attachment from an email message via Remove-MgUserMessageAttachment, Graph appears to strip all non-Microsoft X-* Internet message headers from the message.
For example, an existing X-Spam header will disappear, but X-MS-Exchange* headers will remain.
Is this behavior documented anywhere either as a bug or a feature? Is it just me?
I’ve run into an odd behavior that doesn’t seem to be documented. When I delete an attachment from an email message via Remove-MgUserMessageAttachment, Graph appears to strip all non-Microsoft X-* Internet message headers from the message. For example, an existing X-Spam header will disappear, but X-MS-Exchange* headers will remain. Is this behavior documented anywhere either as a bug or a feature? Is it just me? Read More
This site is read only at the farm administrator’s request.
Share Point will not allow me to upload my reports
Share Point will not allow me to upload my reports Read More
How to check collision between several robots?
I have created four robots (rigidBodyTrees) with the Robotics Toolbox.
In reality, this is one robot with four arms.
I chose to create four different robots, because I want to ignore self collision (the self being the same arm).
I want to generate configurations for each arm (each robot) and then check whether there is collision between them.
How do I do this? The documentation says:
checkCollision(robot,config,worldObjects)
This suggests that I can only enter the configuration for one robot and the worldObjects cannot be placed depending on the configuration that is generated but are ‘fixed’.
I really hope someone can help me!I have created four robots (rigidBodyTrees) with the Robotics Toolbox.
In reality, this is one robot with four arms.
I chose to create four different robots, because I want to ignore self collision (the self being the same arm).
I want to generate configurations for each arm (each robot) and then check whether there is collision between them.
How do I do this? The documentation says:
checkCollision(robot,config,worldObjects)
This suggests that I can only enter the configuration for one robot and the worldObjects cannot be placed depending on the configuration that is generated but are ‘fixed’.
I really hope someone can help me! I have created four robots (rigidBodyTrees) with the Robotics Toolbox.
In reality, this is one robot with four arms.
I chose to create four different robots, because I want to ignore self collision (the self being the same arm).
I want to generate configurations for each arm (each robot) and then check whether there is collision between them.
How do I do this? The documentation says:
checkCollision(robot,config,worldObjects)
This suggests that I can only enter the configuration for one robot and the worldObjects cannot be placed depending on the configuration that is generated but are ‘fixed’.
I really hope someone can help me! robot, collision checking, rigid body tree MATLAB Answers — New Questions
If statement in SharePoint
Good Evening
I am trying to enter what I feel is a rather simple “IF” statement but I can’t figure it out.
I have a Field titled “Item Type” and one titled “FLW – Business Days Remaining” and want to return a “2” if the Item Type equal “Prospect” and “FLW – Business Days Remaining” is grater than 2.
This is what I have
=IF(AND([Item Type]=”Prospect”,[FLW – Business Days Remaining] >2,”2″)))
Any help would be greatly appreciated
Good EveningI am trying to enter what I feel is a rather simple “IF” statement but I can’t figure it out. I have a Field titled “Item Type” and one titled “FLW – Business Days Remaining” and want to return a “2” if the Item Type equal “Prospect” and “FLW – Business Days Remaining” is grater than 2. This is what I have=IF(AND([Item Type]=”Prospect”,[FLW – Business Days Remaining] >2,”2″))) Any help would be greatly appreciated Read More
Copilot Studio – Questions about Knowledge
Hello!
We’re testing copilot studio and we’re having questions about the knowledge we’re uploading.
We’re using it for employees to ask internal questions.
Is there anyone who is using it and performing well?
Do you have any recommendations on what the file should look like?
Pdf, Word, Excel, with or without topics
I would appreciate it if you could support us
Thank you for your attention
Hello!We’re testing copilot studio and we’re having questions about the knowledge we’re uploading.We’re using it for employees to ask internal questions.Is there anyone who is using it and performing well?Do you have any recommendations on what the file should look like?Pdf, Word, Excel, with or without topicsI would appreciate it if you could support usThank you for your attention Read More
Bell Notifications doesn’t work. Again.
and above link forwards on
But I definitely shall have them.
and above link forwards on
But I definitely shall have them. Read More
Leveraging phi-3 for an Enhanced Semantic Cache in RAG Applications
The field of Generative AI (GenAI) is rapidly evolving, with Large Language Models (LLMs) playing a central role. Building responsive and efficient applications using these models is crucial. Retrieval-Augmented Generation (RAG) applications, which combine retrieval and generation techniques, have emerged as a powerful solution for generating high-quality responses. However, a key challenge arises in handling repeat queries efficiently while maintaining contextually accurate and diverse responses. This blog post explores a solution that addresses this challenge. We propose a multi-layered approach that utilizes a semantic cache layer and phi-3, a Small Language Model (SLM) from Microsoft, to rewrite responses. This approach enhances both performance and user experience.
Demystifying RAG: Retrieval Meets Generation
Retrieval-Augmented Generation (RAG) is a cutting-edge framework that extends the capabilities of natural language generation models by incorporating information retrieval.
Here’s how it works:
User Query: This is the initial input from the user.
App service: Central component that orchestrates the entire RAG workflow, managing user queries, interacting with the cache and search service, and delivering final responses.
Vectorize Query: Leverage OpenAI Embedding models to vectorize the user query into numerical representations. These vectors, similar to fingerprints, allow for efficient comparison and retrieval of relevant information from the vector store and semantic cache.
Semantic Cache Store: This component acts as a storage unit for responses to previously encountered queries, and to check if the current user query aligns with any queries stored in the semantic cache. If a response is found in the cache (cache-hit), the response is fetched and sent to the user.
Vector Store: If no matching query is found in the cache (cache-miss), leverage Azure AI Search service to scour the vast corpus of text to identify relevant documents or snippets based on the user’s query.
Azure OpenAI LLM (GPT 3.5/4/4o): The retrieved documents from AI Search are fed to these LLMs to craft a response, in-context to the user’s query.
Logs: These are used to monitor and analyze system performance.
What is Semantic Cache?
Semantic caching plays a pivotal role in enhancing the efficiency and responsiveness of Retrieval-Augmented Generation (RAG) applications. This section delves into its significance and functionality within the broader architecture:
Understanding Semantic Cache
Storage and Retrieval: The semantic cache acts as a specialized storage unit that stores responses to previously encountered queries. It indexes these responses based on the semantic content of the queries, allowing for efficient retrieval when similar queries are encountered in the future.
Query Matching: When a user query is received, it undergoes vectorization using embedding models to create a numerical representation. This representation is compared against stored queries in the semantic cache. If a match is found (cache-hit), the corresponding response is fetched without the need for additional computation.
Benefits of Semantic Cache:
Speed: Responses retrieved from the semantic cache are delivered almost instantaneously, significantly reducing latency compared to generating responses from scratch.
Resource Efficiency: By reusing pre-computed responses, semantic caching optimizes resource utilization, allowing computational resources to be allocated more effectively.
Consistency: Cached responses ensure consistency in answers to frequently asked questions or similar queries, maintaining a coherent user experience.
Scalability: As the volume of queries increases, semantic caching scales efficiently by storing and retrieving responses based on semantic similarities rather than raw text matching.
Implementing Semantic Cache in RAG
Integration with RAG Workflow: The semantic cache is seamlessly integrated into the RAG workflow, typically managed by the application service. Upon receiving a user query, the application service first checks the semantic cache for a matching response.
Update and Refresh: Regular updates and maintenance of the semantic cache are essential to ensure that responses remain relevant and up to date. This may involve periodic pruning of outdated entries and adding new responses based on recent user interactions.
Performance Monitoring: Monitoring tools track the performance of the semantic cache, providing insights into cache-hit rates, response retrieval times, and overall system efficiency. These metrics guide optimization efforts and ensure continuous improvement.
Challenges in RAG with Semantic Caching
While RAG models are undeniably powerful, they encounter some hurdles:
Repetitive Queries: When users pose similar or identical queries repeatedly, it can lead to redundant processing, resulting in slower response times.
Response Consistency: Ensuring responses maintain contextual accuracy and relevance, especially for similar queries, is crucial.
Computational Burden: Generating responses from scratch for every query can be computationally expensive, impacting resource utilization.
Improving the Semantic Cache with phi-3
To address these challenges, we propose a multi-layered approach built on top of RAG architecture with semantic caching that leverages phi-3, a Small Language Model (SLM) from Microsoft, to dynamically rewrite cached responses retrieved from the semantic cache for similar repeat queries. This ensures responses remain contextually relevant and varied, even when served from the cache.
Major change in the architeture above is addition of phi-3, When a matching query is found in the cache, the retrieved cached response is routed through phi-3. This SLM analyzes the cached response and the current user query, dynamically rewriting the cached response to better suit the nuances of the new query.
By integrating phi-3 into the semantic cache layer, we can achieve the following:
Dynamic Rewriting: When a query matching a cached response is received, phi-3 steps in. It analyses the cached response and the user’s current query, identifying nuances and differences. Subsequently, phi-3 rewrites the cached response to seamlessly incorporate the specific context of the new query while preserving the core meaning. This ensures that even cached responses feel fresh, relevant, and up to date.
Reduced Computational Load: By leveraging phi-3 for rewriting cached responses, we significantly reduce the burden on the larger, computationally expensive LLMs (like GPT-3). This frees up resources for the LLM to handle complex or novel queries that require its full generative power.
Improved Response Diversity: Even for repetitive queries, phi-3 injects variation into the responses through rewriting. This prevents users from encountering identical responses repeatedly, enhancing the overall user experience.
Implementation Considerations
Integrating phi-3 into your RAG application requires careful planning and execution:
Semantic Cache Management: Efficient management of the semantic cache is crucial to ensure quick access to relevant cached responses. Regular updates and pruning of the cache can help maintain its effectiveness.
Fine-Tuning phi-3: Fine-tuning phi-3 to handle specific rewriting tasks can further enhance its performance and ensure it aligns well with the context of your application.
Monitoring and Analytics: Continuous monitoring and analytics can help identify patterns in user queries and optimize the caching strategy. Logs play a crucial role in this aspect, providing insights into the system’s performance and areas for improvement.
Conclusion
The integration of phi-3 into the semantic cache layer of a RAG application represents a significant advancement in handling repeat queries efficiently while maintaining contextually accurate and diverse responses. By leveraging the dynamic rewriting capabilities of phi-3, we can enhance both the performance and user experience of RAG applications.
This multi-layered approach not only addresses the challenges of repetitive queries and computational burden but also ensures that responses remain fresh and relevant, even when served from the cache. As Generative AI continues to evolve, such innovations will play a crucial role in building responsive and efficient applications that can meet the diverse needs of users.
Incorporating these strategies into your RAG application can help you stay ahead in the rapidly evolving field of Generative AI, delivering high-quality and contextually accurate responses that enhance user satisfaction and engagement.
Microsoft Tech Community – Latest Blogs –Read More
Why are the gradients not backpropagating into the encoder in this custom loop?
I am building a convolutional autoencoder using a custom training loop. When I attempt to reconstruct the images, the network’s output degenerates to guessing the same incorrect value for all inputs. However, training the autoencoder in a single stack with the trainnet function works fine, indicating that the gradient updates are unable to bridge the bottleneck layer in the custom training loop. Unfortunately, I need to use the custom training loop for a different task and am prohibited from using TensorFlow or PyTorch.
What is the syntax to ensure that the encoder is able to update based on the decoder’s reconstruction performance?
%% Functional ‘trainnet’ loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
aeLayers = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
autoencoder = dlnetwork(aeLayers);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
options = trainingOptions("adam", …
InitialLearnRate=learnRate,…
MaxEpochs=30, …
Plots="training-progress", …
TargetDataFormats="SSCB", …
InputDataFormats="SSCB", …
MiniBatchSize=miniBatchSize, …
OutputNetwork="last-iteration", …
Shuffle="every-epoch");
autoencoder = trainnet(dlarray(xTrain, ‘SSCB’),dlarray(xTrain, ‘SSCB’), …
autoencoder, ‘mse’, options);
%% Testing
YTest = predict(autoencoder, dlarray(xTest, ‘SSCB’));
indices = randi(size(xTest, 4), [1, size(xTest, 4)]); % Shuffle YTest & xTest
xTest = xTest(:,:,:,indices); YTest = YTest(:,:,:,indices);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = extractdata(YTest(:,:,:,1:numImages));
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Nonfunctional Custom Training Loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Encoder
layersE = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer];
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
layersD = [
imageInputLayer(projectionSize)
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
netE = dlnetwork(layersE);
netD = dlnetwork(layersD);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
% Create training minibatchqueue
dsTrain = arrayDatastore(xTrain,IterationDimension=4);
numOutputs = 1;
mbq = minibatchqueue(dsTrain,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFormat="SSCB", …
MiniBatchFcn=@preprocessMiniBatch,…
PartialMiniBatch="return");
%Initialize the parameters for the Adam solver.
trailingAvgE = [];
trailingAvgSqE = [];
trailingAvgD = [];
trailingAvgSqD = [];
%Calculate the total number of iterations for the training progress monitor
numIterationsPerEpoch = ceil(size(xTrain, 4) / miniBatchSize);
numIterations = numEpochs * numIterationsPerEpoch;
epoch = 0;
iteration = 0;
%Initialize the training progress monitor.
monitor = trainingProgressMonitor( …
Metrics="TrainingLoss", …
Info=["Epoch", "LearningRate"], …
XLabel="Iteration");
%% Training
while epoch < numEpochs && ~monitor.Stop
epoch = epoch + 1;
% Shuffle data.
shuffle(mbq);
% Loop over mini-batches.
while hasdata(mbq) && ~monitor.Stop
% Assess validation criterion
iteration = iteration + 1;
% Read mini-batch of data.
X = next(mbq);
% Evaluate loss and gradients.
[loss,gradientsE,gradientsD] = dlfeval(@modelLoss,netE,netD,X);
% Update learnable parameters.
[netE,trailingAvgE,trailingAvgSqE] = adamupdate(netE, …
gradientsE,trailingAvgE,trailingAvgSqE,iteration,learnRate);
[netD, trailingAvgD, trailingAvgSqD] = adamupdate(netD, …
gradientsD,trailingAvgD,trailingAvgSqD,iteration,learnRate);
updateInfo(monitor, …
LearningRate=learnRate, …
Epoch=string(epoch) + " of " + string(numEpochs));
recordMetrics(monitor,iteration, …
TrainingLoss=loss);
monitor.Progress = 100*iteration/numIterations;
end
end
%% Testing
dsTest = arrayDatastore(xTest,IterationDimension=4);
numOutputs = 1;
ntest = size(xTest, 4);
indices = randi(ntest,[1,ntest]);
xTest = xTest(:,:,:,indices);% Shuffle test data
mbqTest = minibatchqueue(dsTest,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFcn=@preprocessMiniBatch, …
MiniBatchFormat="SSCB");
YTest = modelPredictions(netE,netD,mbqTest);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = YTest(:,:,:,1:numImages);
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Functions
function [loss,gradientsE,gradientsD] = modelLoss(netE,netD,X)
% Forward through encoder.
Z = forward(netE,X);
% Forward through decoder.
Xrecon = forward(netD,Z);
% Calculate loss and gradients.
loss = regularizedLoss(Xrecon,X);
[gradientsE,gradientsD] = dlgradient(loss,netE.Learnables,netD.Learnables);
end
function loss = regularizedLoss(Xrecon,X)
% Image Reconstruction loss.
reconstructionLoss = l2loss(Xrecon, X, ‘NormalizationFactor’,’all-elements’);
% Combined loss.
loss = reconstructionLoss;
end
function Xrecon = modelPredictions(netE,netD,mbq)
Xrecon = [];
shuffle(mbq)
% Loop over mini-batches.
while hasdata(mbq)
X = next(mbq);
% Pass through encoder
Z = predict(netE,X);
% Pass through decoder to get reconstructed images
XGenerated = predict(netD,Z);
% Extract and concatenate predictions.
Xrecon = cat(4,Xrecon,extractdata(XGenerated));
end
end
function X = preprocessMiniBatch(Xcell)
% Concatenate.
X = cat(4,Xcell{:});
endI am building a convolutional autoencoder using a custom training loop. When I attempt to reconstruct the images, the network’s output degenerates to guessing the same incorrect value for all inputs. However, training the autoencoder in a single stack with the trainnet function works fine, indicating that the gradient updates are unable to bridge the bottleneck layer in the custom training loop. Unfortunately, I need to use the custom training loop for a different task and am prohibited from using TensorFlow or PyTorch.
What is the syntax to ensure that the encoder is able to update based on the decoder’s reconstruction performance?
%% Functional ‘trainnet’ loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
aeLayers = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
autoencoder = dlnetwork(aeLayers);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
options = trainingOptions("adam", …
InitialLearnRate=learnRate,…
MaxEpochs=30, …
Plots="training-progress", …
TargetDataFormats="SSCB", …
InputDataFormats="SSCB", …
MiniBatchSize=miniBatchSize, …
OutputNetwork="last-iteration", …
Shuffle="every-epoch");
autoencoder = trainnet(dlarray(xTrain, ‘SSCB’),dlarray(xTrain, ‘SSCB’), …
autoencoder, ‘mse’, options);
%% Testing
YTest = predict(autoencoder, dlarray(xTest, ‘SSCB’));
indices = randi(size(xTest, 4), [1, size(xTest, 4)]); % Shuffle YTest & xTest
xTest = xTest(:,:,:,indices); YTest = YTest(:,:,:,indices);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = extractdata(YTest(:,:,:,1:numImages));
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Nonfunctional Custom Training Loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Encoder
layersE = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer];
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
layersD = [
imageInputLayer(projectionSize)
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
netE = dlnetwork(layersE);
netD = dlnetwork(layersD);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
% Create training minibatchqueue
dsTrain = arrayDatastore(xTrain,IterationDimension=4);
numOutputs = 1;
mbq = minibatchqueue(dsTrain,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFormat="SSCB", …
MiniBatchFcn=@preprocessMiniBatch,…
PartialMiniBatch="return");
%Initialize the parameters for the Adam solver.
trailingAvgE = [];
trailingAvgSqE = [];
trailingAvgD = [];
trailingAvgSqD = [];
%Calculate the total number of iterations for the training progress monitor
numIterationsPerEpoch = ceil(size(xTrain, 4) / miniBatchSize);
numIterations = numEpochs * numIterationsPerEpoch;
epoch = 0;
iteration = 0;
%Initialize the training progress monitor.
monitor = trainingProgressMonitor( …
Metrics="TrainingLoss", …
Info=["Epoch", "LearningRate"], …
XLabel="Iteration");
%% Training
while epoch < numEpochs && ~monitor.Stop
epoch = epoch + 1;
% Shuffle data.
shuffle(mbq);
% Loop over mini-batches.
while hasdata(mbq) && ~monitor.Stop
% Assess validation criterion
iteration = iteration + 1;
% Read mini-batch of data.
X = next(mbq);
% Evaluate loss and gradients.
[loss,gradientsE,gradientsD] = dlfeval(@modelLoss,netE,netD,X);
% Update learnable parameters.
[netE,trailingAvgE,trailingAvgSqE] = adamupdate(netE, …
gradientsE,trailingAvgE,trailingAvgSqE,iteration,learnRate);
[netD, trailingAvgD, trailingAvgSqD] = adamupdate(netD, …
gradientsD,trailingAvgD,trailingAvgSqD,iteration,learnRate);
updateInfo(monitor, …
LearningRate=learnRate, …
Epoch=string(epoch) + " of " + string(numEpochs));
recordMetrics(monitor,iteration, …
TrainingLoss=loss);
monitor.Progress = 100*iteration/numIterations;
end
end
%% Testing
dsTest = arrayDatastore(xTest,IterationDimension=4);
numOutputs = 1;
ntest = size(xTest, 4);
indices = randi(ntest,[1,ntest]);
xTest = xTest(:,:,:,indices);% Shuffle test data
mbqTest = minibatchqueue(dsTest,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFcn=@preprocessMiniBatch, …
MiniBatchFormat="SSCB");
YTest = modelPredictions(netE,netD,mbqTest);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = YTest(:,:,:,1:numImages);
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Functions
function [loss,gradientsE,gradientsD] = modelLoss(netE,netD,X)
% Forward through encoder.
Z = forward(netE,X);
% Forward through decoder.
Xrecon = forward(netD,Z);
% Calculate loss and gradients.
loss = regularizedLoss(Xrecon,X);
[gradientsE,gradientsD] = dlgradient(loss,netE.Learnables,netD.Learnables);
end
function loss = regularizedLoss(Xrecon,X)
% Image Reconstruction loss.
reconstructionLoss = l2loss(Xrecon, X, ‘NormalizationFactor’,’all-elements’);
% Combined loss.
loss = reconstructionLoss;
end
function Xrecon = modelPredictions(netE,netD,mbq)
Xrecon = [];
shuffle(mbq)
% Loop over mini-batches.
while hasdata(mbq)
X = next(mbq);
% Pass through encoder
Z = predict(netE,X);
% Pass through decoder to get reconstructed images
XGenerated = predict(netD,Z);
% Extract and concatenate predictions.
Xrecon = cat(4,Xrecon,extractdata(XGenerated));
end
end
function X = preprocessMiniBatch(Xcell)
% Concatenate.
X = cat(4,Xcell{:});
end I am building a convolutional autoencoder using a custom training loop. When I attempt to reconstruct the images, the network’s output degenerates to guessing the same incorrect value for all inputs. However, training the autoencoder in a single stack with the trainnet function works fine, indicating that the gradient updates are unable to bridge the bottleneck layer in the custom training loop. Unfortunately, I need to use the custom training loop for a different task and am prohibited from using TensorFlow or PyTorch.
What is the syntax to ensure that the encoder is able to update based on the decoder’s reconstruction performance?
%% Functional ‘trainnet’ loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
aeLayers = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
autoencoder = dlnetwork(aeLayers);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
options = trainingOptions("adam", …
InitialLearnRate=learnRate,…
MaxEpochs=30, …
Plots="training-progress", …
TargetDataFormats="SSCB", …
InputDataFormats="SSCB", …
MiniBatchSize=miniBatchSize, …
OutputNetwork="last-iteration", …
Shuffle="every-epoch");
autoencoder = trainnet(dlarray(xTrain, ‘SSCB’),dlarray(xTrain, ‘SSCB’), …
autoencoder, ‘mse’, options);
%% Testing
YTest = predict(autoencoder, dlarray(xTest, ‘SSCB’));
indices = randi(size(xTest, 4), [1, size(xTest, 4)]); % Shuffle YTest & xTest
xTest = xTest(:,:,:,indices); YTest = YTest(:,:,:,indices);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = extractdata(YTest(:,:,:,1:numImages));
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Nonfunctional Custom Training Loop
clear
close all
clc
% Get handwritten digit data
xTrain = digitTrain4DArrayData;
xTest = digitTest4DArrayData;
% Check that all pixel values are min-max scaled
assert(max(xTrain(:)) == 1); assert(min(xTrain(:)) == 0);
assert(max(xTest(:)) == 1); assert(min(xTest(:)) == 0);
imageSize = [28 28 1];
%% Layer definitions
% Encoder
layersE = [
imageInputLayer(imageSize)
convolution2dLayer(3,32,Padding="same",Stride=2)
reluLayer
convolution2dLayer(3,64,Padding="same",Stride=2)
reluLayer];
% Latent projection
projectionSize = [7 7 64];
numInputChannels = imageSize(3);
% Decoder
layersD = [
imageInputLayer(projectionSize)
transposedConv2dLayer(3,64,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,32,Cropping="same",Stride=2)
reluLayer
transposedConv2dLayer(3,numInputChannels,Cropping="same")
sigmoidLayer(Name=’Output’)
];
netE = dlnetwork(layersE);
netD = dlnetwork(layersD);
%% Training Parameters
numEpochs = 150;
miniBatchSize = 25;
learnRate = 1e-3;
% Create training minibatchqueue
dsTrain = arrayDatastore(xTrain,IterationDimension=4);
numOutputs = 1;
mbq = minibatchqueue(dsTrain,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFormat="SSCB", …
MiniBatchFcn=@preprocessMiniBatch,…
PartialMiniBatch="return");
%Initialize the parameters for the Adam solver.
trailingAvgE = [];
trailingAvgSqE = [];
trailingAvgD = [];
trailingAvgSqD = [];
%Calculate the total number of iterations for the training progress monitor
numIterationsPerEpoch = ceil(size(xTrain, 4) / miniBatchSize);
numIterations = numEpochs * numIterationsPerEpoch;
epoch = 0;
iteration = 0;
%Initialize the training progress monitor.
monitor = trainingProgressMonitor( …
Metrics="TrainingLoss", …
Info=["Epoch", "LearningRate"], …
XLabel="Iteration");
%% Training
while epoch < numEpochs && ~monitor.Stop
epoch = epoch + 1;
% Shuffle data.
shuffle(mbq);
% Loop over mini-batches.
while hasdata(mbq) && ~monitor.Stop
% Assess validation criterion
iteration = iteration + 1;
% Read mini-batch of data.
X = next(mbq);
% Evaluate loss and gradients.
[loss,gradientsE,gradientsD] = dlfeval(@modelLoss,netE,netD,X);
% Update learnable parameters.
[netE,trailingAvgE,trailingAvgSqE] = adamupdate(netE, …
gradientsE,trailingAvgE,trailingAvgSqE,iteration,learnRate);
[netD, trailingAvgD, trailingAvgSqD] = adamupdate(netD, …
gradientsD,trailingAvgD,trailingAvgSqD,iteration,learnRate);
updateInfo(monitor, …
LearningRate=learnRate, …
Epoch=string(epoch) + " of " + string(numEpochs));
recordMetrics(monitor,iteration, …
TrainingLoss=loss);
monitor.Progress = 100*iteration/numIterations;
end
end
%% Testing
dsTest = arrayDatastore(xTest,IterationDimension=4);
numOutputs = 1;
ntest = size(xTest, 4);
indices = randi(ntest,[1,ntest]);
xTest = xTest(:,:,:,indices);% Shuffle test data
mbqTest = minibatchqueue(dsTest,numOutputs, …
MiniBatchSize = miniBatchSize, …
MiniBatchFcn=@preprocessMiniBatch, …
MiniBatchFormat="SSCB");
YTest = modelPredictions(netE,netD,mbqTest);
% Display test images
numImages = 64;
figure
subplot(1,2,1)
preds = YTest(:,:,:,1:numImages);
I = imtile(preds);
imshow(I)
title("Reconstructed Images")
subplot(1,2,2)
orgs = xTest(:,:,:,1:numImages);
I = imtile(orgs);
imshow(I)
title("Original Images")
%% Functions
function [loss,gradientsE,gradientsD] = modelLoss(netE,netD,X)
% Forward through encoder.
Z = forward(netE,X);
% Forward through decoder.
Xrecon = forward(netD,Z);
% Calculate loss and gradients.
loss = regularizedLoss(Xrecon,X);
[gradientsE,gradientsD] = dlgradient(loss,netE.Learnables,netD.Learnables);
end
function loss = regularizedLoss(Xrecon,X)
% Image Reconstruction loss.
reconstructionLoss = l2loss(Xrecon, X, ‘NormalizationFactor’,’all-elements’);
% Combined loss.
loss = reconstructionLoss;
end
function Xrecon = modelPredictions(netE,netD,mbq)
Xrecon = [];
shuffle(mbq)
% Loop over mini-batches.
while hasdata(mbq)
X = next(mbq);
% Pass through encoder
Z = predict(netE,X);
% Pass through decoder to get reconstructed images
XGenerated = predict(netD,Z);
% Extract and concatenate predictions.
Xrecon = cat(4,Xrecon,extractdata(XGenerated));
end
end
function X = preprocessMiniBatch(Xcell)
% Concatenate.
X = cat(4,Xcell{:});
end deep learning, autoencoder, regularization, initialization, custom loops MATLAB Answers — New Questions
Integrating Two Unrelated Values
Hi!
I am trying to integrate two different series that correlate to the same image. I was able to obtain X, Y, and Z values for an image in which the x-values correlate with length, the y-values correlate with width, and the z-values correlate with intensity. However, because these measurements are taken across a rectangular ROI, there are 528 x-values (length) and 504 y-values (width), as the object resembles 1/2 of an ellipse.
I would like the integrate these values so that I can plot (length x width x intensity) for my given shape. I have tried to integrate these values by plotting them on the same scatterplot, however I am not having a lot of sucess. I also am not having any luck finding code that will allow me to integrate these values.
Does anyone know an effective way to integrate two "unrelated" values?Hi!
I am trying to integrate two different series that correlate to the same image. I was able to obtain X, Y, and Z values for an image in which the x-values correlate with length, the y-values correlate with width, and the z-values correlate with intensity. However, because these measurements are taken across a rectangular ROI, there are 528 x-values (length) and 504 y-values (width), as the object resembles 1/2 of an ellipse.
I would like the integrate these values so that I can plot (length x width x intensity) for my given shape. I have tried to integrate these values by plotting them on the same scatterplot, however I am not having a lot of sucess. I also am not having any luck finding code that will allow me to integrate these values.
Does anyone know an effective way to integrate two "unrelated" values? Hi!
I am trying to integrate two different series that correlate to the same image. I was able to obtain X, Y, and Z values for an image in which the x-values correlate with length, the y-values correlate with width, and the z-values correlate with intensity. However, because these measurements are taken across a rectangular ROI, there are 528 x-values (length) and 504 y-values (width), as the object resembles 1/2 of an ellipse.
I would like the integrate these values so that I can plot (length x width x intensity) for my given shape. I have tried to integrate these values by plotting them on the same scatterplot, however I am not having a lot of sucess. I also am not having any luck finding code that will allow me to integrate these values.
Does anyone know an effective way to integrate two "unrelated" values? image analysis, integration, 3d plots, function, matrix manipulation MATLAB Answers — New Questions
Same Name but Not Recognized Variable
Hello everyone, I am having issues when calling upon variables. I have a bunch of variables in this code and when I try to call them or hover over them it doesn’t highlight the others as the same.
I have included some pictures of my issue as I’m not quite sure how to describe it. The first one below shows me attempting to call a function, and the variables in calling that function are shown. However, when I call the function, I get this error: (in red of course)
The problem I’ve noticed is when I highlight a variable within calling that function, in the photo below I’ve highlighted "CurrThrottleData", it isn’t highlighted everywhere else. A few places, but not in the function definition below.
Same issue is shown below with "fileName"
My next thought was maybe the function name is having issues or it’s because I have the code separated into sections, but the function name lights up just fine in a different section. It previously worked when the function was called from a separate matlab file but it is really important that the function definitions and the code runs in one parent file for it all.
Any advice and help is appreciate, I have tried the following:
-Control f and replace everything with the same name but different capitalization to make sure it’d change it everywhere, this had no effect.
-Changing the name entirely to something different and manuall copy and pasting the name
-Copying the code in segments and running it in a new file
-Closing and opening matlab incase it needed a brain break… i guess hahaHello everyone, I am having issues when calling upon variables. I have a bunch of variables in this code and when I try to call them or hover over them it doesn’t highlight the others as the same.
I have included some pictures of my issue as I’m not quite sure how to describe it. The first one below shows me attempting to call a function, and the variables in calling that function are shown. However, when I call the function, I get this error: (in red of course)
The problem I’ve noticed is when I highlight a variable within calling that function, in the photo below I’ve highlighted "CurrThrottleData", it isn’t highlighted everywhere else. A few places, but not in the function definition below.
Same issue is shown below with "fileName"
My next thought was maybe the function name is having issues or it’s because I have the code separated into sections, but the function name lights up just fine in a different section. It previously worked when the function was called from a separate matlab file but it is really important that the function definitions and the code runs in one parent file for it all.
Any advice and help is appreciate, I have tried the following:
-Control f and replace everything with the same name but different capitalization to make sure it’d change it everywhere, this had no effect.
-Changing the name entirely to something different and manuall copy and pasting the name
-Copying the code in segments and running it in a new file
-Closing and opening matlab incase it needed a brain break… i guess haha Hello everyone, I am having issues when calling upon variables. I have a bunch of variables in this code and when I try to call them or hover over them it doesn’t highlight the others as the same.
I have included some pictures of my issue as I’m not quite sure how to describe it. The first one below shows me attempting to call a function, and the variables in calling that function are shown. However, when I call the function, I get this error: (in red of course)
The problem I’ve noticed is when I highlight a variable within calling that function, in the photo below I’ve highlighted "CurrThrottleData", it isn’t highlighted everywhere else. A few places, but not in the function definition below.
Same issue is shown below with "fileName"
My next thought was maybe the function name is having issues or it’s because I have the code separated into sections, but the function name lights up just fine in a different section. It previously worked when the function was called from a separate matlab file but it is really important that the function definitions and the code runs in one parent file for it all.
Any advice and help is appreciate, I have tried the following:
-Control f and replace everything with the same name but different capitalization to make sure it’d change it everywhere, this had no effect.
-Changing the name entirely to something different and manuall copy and pasting the name
-Copying the code in segments and running it in a new file
-Closing and opening matlab incase it needed a brain break… i guess haha codeprocessing, matlab, variable-errors, data, function, functions, matlab function, embedded matlab function MATLAB Answers — New Questions
Entra ID Connect Download Issue
Good afternoon,
I’ve been trying all day to download the latest version of Entra ID Connect and it seems the download link is failing due to a server error. I’ve tried on different machines on different networks with the same result. Is anyone else seeing this or know the best way to open a ticket to let them know? Or is there another Microsoft mirror I could use?
Sends you to the redirect URL of:
https://go.microsoft.com/fwlink/?LinkId=615771
Which ultimately just sends you to the last URL I’ll post below.
This one, however:
https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/reference-connect-version-history#looking-for-the-latest-versions
Sends you directly to the actual download link of:
https://www.microsoft.com/en-us/download/details.aspx?id=47594
Which results in a server error.
Any help is greatly appreciated. Thank you so much.
Good afternoon, I’ve been trying all day to download the latest version of Entra ID Connect and it seems the download link is failing due to a server error. I’ve tried on different machines on different networks with the same result. Is anyone else seeing this or know the best way to open a ticket to let them know? Or is there another Microsoft mirror I could use? This link:https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/how-to-connect-install-roadmap#install-microsoft-entra-connectSends you to the redirect URL of:https://go.microsoft.com/fwlink/?LinkId=615771Which ultimately just sends you to the last URL I’ll post below.This one, however:https://learn.microsoft.com/en-us/entra/identity/hybrid/connect/reference-connect-version-history#looking-for-the-latest-versionsSends you directly to the actual download link of:https://www.microsoft.com/en-us/download/details.aspx?id=47594Which results in a server error. Any help is greatly appreciated. Thank you so much. Read More
Subforms in Tabs: Can They Be Dragged-and-Dropped In, or Cut-and-Pasted?
Hello, I have a question or two about using tab controls and subforms. I am using Access 2021, and am a novice at database design.
I am designing a movie collection database, and I currently have three subforms. I’d like to have them in tabs, rather than all on one form.
My question: Is it possible to drag and drop the subforms into the tabs, or cut-and-paste them? Or do I have to build the subforms over again, from scratch?
Which method of placing subforms in the tabs do you prefer? Thank you!
Hello, I have a question or two about using tab controls and subforms. I am using Access 2021, and am a novice at database design. I am designing a movie collection database, and I currently have three subforms. I’d like to have them in tabs, rather than all on one form. My question: Is it possible to drag and drop the subforms into the tabs, or cut-and-paste them? Or do I have to build the subforms over again, from scratch? Which method of placing subforms in the tabs do you prefer? Thank you! Read More
Stacked Bar Chart as Office Timeline
I am using Microsoft 365, but I do not have access to Power BI.
I am trying to create a stacked bar chart as a replacement for our firm’s Office Timeline, which currently exists on our outdated, slowly dying intranet site. What I’m looking for is something similar to this:
Projects will be on the Y axis, with a perpetual calendar on the X axis – the color changes represent the end of each project phase, and the black boxes represent milestone dates within the phase.
I have brought in the data, added a perpetual calendar and formatted the dates as numbers. I established a ‘base date’, but I’m not sure that I need it. The resulting stacked bar chart is not even close to what I’m trying to do, and I think the reason is that I need to incorporate a formula to match the ending phase dates to the perpetual calendar (see the data example below).
Do any of you know of a tutorial I could study or a general method to do what I’m trying to accomplish? I have looked at dozens of articles and YouTube videos and I can’t find anything that addresses this.
Thanks,
Traci
I am using Microsoft 365, but I do not have access to Power BI.I am trying to create a stacked bar chart as a replacement for our firm’s Office Timeline, which currently exists on our outdated, slowly dying intranet site. What I’m looking for is something similar to this:Projects will be on the Y axis, with a perpetual calendar on the X axis – the color changes represent the end of each project phase, and the black boxes represent milestone dates within the phase. I have brought in the data, added a perpetual calendar and formatted the dates as numbers. I established a ‘base date’, but I’m not sure that I need it. The resulting stacked bar chart is not even close to what I’m trying to do, and I think the reason is that I need to incorporate a formula to match the ending phase dates to the perpetual calendar (see the data example below). Do any of you know of a tutorial I could study or a general method to do what I’m trying to accomplish? I have looked at dozens of articles and YouTube videos and I can’t find anything that addresses this. Thanks,Traci Read More
Search in Outlook 16.8 does not work on Mac Book Air Ventura 13.2.1
Hi there,
bought the a new mac and the search in Outlook does not work, is not existing. I work in Outlook 16.86 on Mac Book Air Ventura 13.2.1
I de-installed MS and re-installed = followed all steps from MS. Emptied the trash and shut my mac. Re- started it and started again with re-installing. I bought the MS package and thought, I am good. But I am not.
On my old Mac 15.8, all is working in Outlook.
Can you pls advise…. I tried to download the older version of Outlook, but failed here, too.
Thanks, Sabine
Hi there, bought the a new mac and the search in Outlook does not work, is not existing. I work in Outlook 16.86 on Mac Book Air Ventura 13.2.1I de-installed MS and re-installed = followed all steps from MS. Emptied the trash and shut my mac. Re- started it and started again with re-installing. I bought the MS package and thought, I am good. But I am not.On my old Mac 15.8, all is working in Outlook. Can you pls advise…. I tried to download the older version of Outlook, but failed here, too. Thanks, Sabine Read More
Want breaks in line graph
I’m trying to follow this to see gaps in my line graph, but the gaps option is grayed out. What to do?
Display empty cells, null (#N/A) values, and hidden worksheet data in a chart – Microsoft Support
Note: This is fake data, just testing out the line graph formatting.
I’m trying to follow this to see gaps in my line graph, but the gaps option is grayed out. What to do? Display empty cells, null (#N/A) values, and hidden worksheet data in a chart – Microsoft Support Note: This is fake data, just testing out the line graph formatting. Read More
Export all Tenant Team Chats
I have read several postings here and on other sites but its still not clear if it is possible or how to do get a list of all team chats.
Our CEO is asking and not providing much information to why or what the purpose is, what they are looking for. Not sure if they are looking for the group chat titles, I assume 1to1 chats are not required as there could be hundreds/thousands of these.
Some of the reading suggests there may be a way to see;
all the chats a specific person belongs to.eDiscovery might be able to be used but seems to imply this is more about reconstruction of a chat rather than just a simple list. But this would require a Premium license and I am not sure it actually gives a list.
It also not clear how external “Chats” from a meeting would play in to this, it seems a bit more complicated than what our CEO understands.
Can anyone confirm if we can pull a list of Team chat?
I have read several postings here and on other sites but its still not clear if it is possible or how to do get a list of all team chats.Our CEO is asking and not providing much information to why or what the purpose is, what they are looking for. Not sure if they are looking for the group chat titles, I assume 1to1 chats are not required as there could be hundreds/thousands of these. Some of the reading suggests there may be a way to see;all the chats a specific person belongs to.eDiscovery might be able to be used but seems to imply this is more about reconstruction of a chat rather than just a simple list. But this would require a Premium license and I am not sure it actually gives a list.It also not clear how external “Chats” from a meeting would play in to this, it seems a bit more complicated than what our CEO understands. Can anyone confirm if we can pull a list of Team chat? Read More
I encountered error though the input data to INS block is exactly 3-column
I’m using the uavPackageDelivery example in UAV toolbox and making small modification in this module.(in "fly full mission mode")
i want to integrate an INS sensor into it,and i’m passing necessary parameters in this sensor.
but once the simulation starts,error message is shown below.
I’m passing exactly 3-column array into it,but still received error above.
Am i right to use INS here?if wrong,where else should i use the INS block?
thxI’m using the uavPackageDelivery example in UAV toolbox and making small modification in this module.(in "fly full mission mode")
i want to integrate an INS sensor into it,and i’m passing necessary parameters in this sensor.
but once the simulation starts,error message is shown below.
I’m passing exactly 3-column array into it,but still received error above.
Am i right to use INS here?if wrong,where else should i use the INS block?
thx I’m using the uavPackageDelivery example in UAV toolbox and making small modification in this module.(in "fly full mission mode")
i want to integrate an INS sensor into it,and i’m passing necessary parameters in this sensor.
but once the simulation starts,error message is shown below.
I’m passing exactly 3-column array into it,but still received error above.
Am i right to use INS here?if wrong,where else should i use the INS block?
thx simulink, simulation, sensor, uav toolbox, ins MATLAB Answers — New Questions
Iterate over struct with length>1, with multiple fields
I have a struct with size>1, and multiple 16 fields.
Each fieldname corresponds to some quantity or property, and the struct stores these properties for 47 different items.
I’m trying to iterate the whole dataset. Preferrably, I’d like to iterate by fieldname and retrieve an array for each filename because within each field name the variable type is uniform.
To illustrate:
K>> teststruct = struct(‘name’, [‘Alice’, ‘Bob’, ‘Eve’], ‘age’, {24, 45, 35})
teststruct =
1×3 struct array with fields:
name
age
This shows up nicely as a table in the worspace browser.
However, if I iterate by fieldname, this goes wrong (spurious empty lines removed for readability):
K>> fnames = fieldnames(teststruct);
K>> teststruct.(fnames{1})
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
What I wanted was an array of all the names, instead I get three answers, of compounded names.
If I do the same with the ‘age’ field, at least each answer contains only one age, but they’re still not in any kind of structure I could use in a code which does not know the size or field names of a struct it needs to deal with. In fact, if I assign the above to a variable, this happens:
K>> names = teststruct.(fnames{1})
names =
‘AliceBobEve’
Doing the same with the other field only gives me the first age. One correct piece of data at least, but still not at all what I wanted…
I tried applying the code I found here, which promises to print the contents of an entire struct, but the same happens: I only get the first value of everything.
I know that I could loop over the struct indices first, and access them like this:
value = testsruct(i).(fieldnames(j))
…but then I’d be getting them separately, one by one, instead of getting back the cell array (or any other kind of array) that was used to define the struct in the first place, which is way easier to deal with.
Is that possible somehow?I have a struct with size>1, and multiple 16 fields.
Each fieldname corresponds to some quantity or property, and the struct stores these properties for 47 different items.
I’m trying to iterate the whole dataset. Preferrably, I’d like to iterate by fieldname and retrieve an array for each filename because within each field name the variable type is uniform.
To illustrate:
K>> teststruct = struct(‘name’, [‘Alice’, ‘Bob’, ‘Eve’], ‘age’, {24, 45, 35})
teststruct =
1×3 struct array with fields:
name
age
This shows up nicely as a table in the worspace browser.
However, if I iterate by fieldname, this goes wrong (spurious empty lines removed for readability):
K>> fnames = fieldnames(teststruct);
K>> teststruct.(fnames{1})
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
What I wanted was an array of all the names, instead I get three answers, of compounded names.
If I do the same with the ‘age’ field, at least each answer contains only one age, but they’re still not in any kind of structure I could use in a code which does not know the size or field names of a struct it needs to deal with. In fact, if I assign the above to a variable, this happens:
K>> names = teststruct.(fnames{1})
names =
‘AliceBobEve’
Doing the same with the other field only gives me the first age. One correct piece of data at least, but still not at all what I wanted…
I tried applying the code I found here, which promises to print the contents of an entire struct, but the same happens: I only get the first value of everything.
I know that I could loop over the struct indices first, and access them like this:
value = testsruct(i).(fieldnames(j))
…but then I’d be getting them separately, one by one, instead of getting back the cell array (or any other kind of array) that was used to define the struct in the first place, which is way easier to deal with.
Is that possible somehow? I have a struct with size>1, and multiple 16 fields.
Each fieldname corresponds to some quantity or property, and the struct stores these properties for 47 different items.
I’m trying to iterate the whole dataset. Preferrably, I’d like to iterate by fieldname and retrieve an array for each filename because within each field name the variable type is uniform.
To illustrate:
K>> teststruct = struct(‘name’, [‘Alice’, ‘Bob’, ‘Eve’], ‘age’, {24, 45, 35})
teststruct =
1×3 struct array with fields:
name
age
This shows up nicely as a table in the worspace browser.
However, if I iterate by fieldname, this goes wrong (spurious empty lines removed for readability):
K>> fnames = fieldnames(teststruct);
K>> teststruct.(fnames{1})
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
ans =
‘AliceBobEve’
What I wanted was an array of all the names, instead I get three answers, of compounded names.
If I do the same with the ‘age’ field, at least each answer contains only one age, but they’re still not in any kind of structure I could use in a code which does not know the size or field names of a struct it needs to deal with. In fact, if I assign the above to a variable, this happens:
K>> names = teststruct.(fnames{1})
names =
‘AliceBobEve’
Doing the same with the other field only gives me the first age. One correct piece of data at least, but still not at all what I wanted…
I tried applying the code I found here, which promises to print the contents of an entire struct, but the same happens: I only get the first value of everything.
I know that I could loop over the struct indices first, and access them like this:
value = testsruct(i).(fieldnames(j))
…but then I’d be getting them separately, one by one, instead of getting back the cell array (or any other kind of array) that was used to define the struct in the first place, which is way easier to deal with.
Is that possible somehow? struct, indexing MATLAB Answers — New Questions
simplify and reorganize symbolic expression
Hello everyone,
I am trying to simplify a symbolic expression (which is the result of the script below) but i am having trouble as it is pretty big and there are many parameters. My objective is to reorganize the expression "B" as powers of variables "c1" and "c2" with all the other parameters included in the coefficents of the polynomial expression (they are gonna be substitued by values in other calculations). I have tried with "simplify" and triyng to nest "collect" into one another.
Does anybody have any tips on how to make the process more efficient? Are there any functions, which I am not aware of right now, that can simplify polynomial expression with regards to the coefficients that multiply the different powers? how about with multivariable expressioon as is my case?
also, is there a way to make math expressions with powers, parentesis, fractions, etc. more readable in the command window? now i have the standard formatting setting of matlab which makes long expressions pretty hard to read
Thank you very much
clear; clc;
syms m J a1 a2
syms k1 k2 k_a12 k_a21
syms c1 c2 c_a12 c_a21
syms lambda
% matrices definitions
M = [m 0;
0 J];
K = [k1 + k2 + (k_a12+k_a21) k1*a1 – k2*a2 + (k_a21*a1 – k_a12*a2);
k1*a1 – k2*a2 + (k_a12*a1-k_a21*a2) k1*a1^2 + k2*a2^2 – (k_a12+k_a21)*a1*a2];
C = [c1 + c2 + (c_a12+c_a21) c1*a1 – c2*a2 + (c_a21*a1 – c_a12*a2);
c1*a1 – c2*a2 + (c_a12*a1-c_a21*a2) c1*a1^2 + c2*a2^2 – (c_a12+c_a21)*a1*a2];
det_M = det(M);
det_K = simplify(collect(collect(collect(collect(det(K), 2*a1), a2), a1^2), a2^2));
det_C = simplify(collect(collect(collect(collect(det(C), 2*a1), a2), a1^2), a2^2));
Mat = (lambda^2)*M + lambda*C + K;
determinant = collect(det(Mat), lambda);
coeff = coeffs(determinant, lambda);
b0 = simplify(collect(collect(collect(collect(coeff(1), k_a12), k_a21), k2), k1))
b1 = simplify(collect(collect(collect(collect(coeff(2), 2*a1), a2), a1^2), a2^2))
b2 = simplify(collect(collect(collect(collect(collect(collect(coeff(3), m), J), c1), c2), c_a12), c_a21))
b3 = simplify(collect(collect(collect(collect(collect(collect(coeff(4), a1^2), a2^2), a1), a2), m), J))
B = b1*b2*b3 – b1^2 – b0*b3^2;
B = expand(B);
B = simplify(collect(B, c1))
this is B that i get:
(a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)*(m*(a1^2*k1 + a2^2*k2 – a1*a2*k_a12 – a1*a2*k_a21) + J*(k1 + k2 + k_a12 + k_a21) + c1*c2*(a1 + a2)^2 – c_a12*c_a21*(a1 + a2)^2) – (a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))^2*(k1*k2 – k_a12*k_a21) – (a1 + a2)^4*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)^2Hello everyone,
I am trying to simplify a symbolic expression (which is the result of the script below) but i am having trouble as it is pretty big and there are many parameters. My objective is to reorganize the expression "B" as powers of variables "c1" and "c2" with all the other parameters included in the coefficents of the polynomial expression (they are gonna be substitued by values in other calculations). I have tried with "simplify" and triyng to nest "collect" into one another.
Does anybody have any tips on how to make the process more efficient? Are there any functions, which I am not aware of right now, that can simplify polynomial expression with regards to the coefficients that multiply the different powers? how about with multivariable expressioon as is my case?
also, is there a way to make math expressions with powers, parentesis, fractions, etc. more readable in the command window? now i have the standard formatting setting of matlab which makes long expressions pretty hard to read
Thank you very much
clear; clc;
syms m J a1 a2
syms k1 k2 k_a12 k_a21
syms c1 c2 c_a12 c_a21
syms lambda
% matrices definitions
M = [m 0;
0 J];
K = [k1 + k2 + (k_a12+k_a21) k1*a1 – k2*a2 + (k_a21*a1 – k_a12*a2);
k1*a1 – k2*a2 + (k_a12*a1-k_a21*a2) k1*a1^2 + k2*a2^2 – (k_a12+k_a21)*a1*a2];
C = [c1 + c2 + (c_a12+c_a21) c1*a1 – c2*a2 + (c_a21*a1 – c_a12*a2);
c1*a1 – c2*a2 + (c_a12*a1-c_a21*a2) c1*a1^2 + c2*a2^2 – (c_a12+c_a21)*a1*a2];
det_M = det(M);
det_K = simplify(collect(collect(collect(collect(det(K), 2*a1), a2), a1^2), a2^2));
det_C = simplify(collect(collect(collect(collect(det(C), 2*a1), a2), a1^2), a2^2));
Mat = (lambda^2)*M + lambda*C + K;
determinant = collect(det(Mat), lambda);
coeff = coeffs(determinant, lambda);
b0 = simplify(collect(collect(collect(collect(coeff(1), k_a12), k_a21), k2), k1))
b1 = simplify(collect(collect(collect(collect(coeff(2), 2*a1), a2), a1^2), a2^2))
b2 = simplify(collect(collect(collect(collect(collect(collect(coeff(3), m), J), c1), c2), c_a12), c_a21))
b3 = simplify(collect(collect(collect(collect(collect(collect(coeff(4), a1^2), a2^2), a1), a2), m), J))
B = b1*b2*b3 – b1^2 – b0*b3^2;
B = expand(B);
B = simplify(collect(B, c1))
this is B that i get:
(a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)*(m*(a1^2*k1 + a2^2*k2 – a1*a2*k_a12 – a1*a2*k_a21) + J*(k1 + k2 + k_a12 + k_a21) + c1*c2*(a1 + a2)^2 – c_a12*c_a21*(a1 + a2)^2) – (a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))^2*(k1*k2 – k_a12*k_a21) – (a1 + a2)^4*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)^2 Hello everyone,
I am trying to simplify a symbolic expression (which is the result of the script below) but i am having trouble as it is pretty big and there are many parameters. My objective is to reorganize the expression "B" as powers of variables "c1" and "c2" with all the other parameters included in the coefficents of the polynomial expression (they are gonna be substitued by values in other calculations). I have tried with "simplify" and triyng to nest "collect" into one another.
Does anybody have any tips on how to make the process more efficient? Are there any functions, which I am not aware of right now, that can simplify polynomial expression with regards to the coefficients that multiply the different powers? how about with multivariable expressioon as is my case?
also, is there a way to make math expressions with powers, parentesis, fractions, etc. more readable in the command window? now i have the standard formatting setting of matlab which makes long expressions pretty hard to read
Thank you very much
clear; clc;
syms m J a1 a2
syms k1 k2 k_a12 k_a21
syms c1 c2 c_a12 c_a21
syms lambda
% matrices definitions
M = [m 0;
0 J];
K = [k1 + k2 + (k_a12+k_a21) k1*a1 – k2*a2 + (k_a21*a1 – k_a12*a2);
k1*a1 – k2*a2 + (k_a12*a1-k_a21*a2) k1*a1^2 + k2*a2^2 – (k_a12+k_a21)*a1*a2];
C = [c1 + c2 + (c_a12+c_a21) c1*a1 – c2*a2 + (c_a21*a1 – c_a12*a2);
c1*a1 – c2*a2 + (c_a12*a1-c_a21*a2) c1*a1^2 + c2*a2^2 – (c_a12+c_a21)*a1*a2];
det_M = det(M);
det_K = simplify(collect(collect(collect(collect(det(K), 2*a1), a2), a1^2), a2^2));
det_C = simplify(collect(collect(collect(collect(det(C), 2*a1), a2), a1^2), a2^2));
Mat = (lambda^2)*M + lambda*C + K;
determinant = collect(det(Mat), lambda);
coeff = coeffs(determinant, lambda);
b0 = simplify(collect(collect(collect(collect(coeff(1), k_a12), k_a21), k2), k1))
b1 = simplify(collect(collect(collect(collect(coeff(2), 2*a1), a2), a1^2), a2^2))
b2 = simplify(collect(collect(collect(collect(collect(collect(coeff(3), m), J), c1), c2), c_a12), c_a21))
b3 = simplify(collect(collect(collect(collect(collect(collect(coeff(4), a1^2), a2^2), a1), a2), m), J))
B = b1*b2*b3 – b1^2 – b0*b3^2;
B = expand(B);
B = simplify(collect(B, c1))
this is B that i get:
(a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)*(m*(a1^2*k1 + a2^2*k2 – a1*a2*k_a12 – a1*a2*k_a21) + J*(k1 + k2 + k_a12 + k_a21) + c1*c2*(a1 + a2)^2 – c_a12*c_a21*(a1 + a2)^2) – (a1 + a2)^2*(m*(c1*a1^2 + (- c_a12 – c_a21)*a1*a2 + c2*a2^2) + J*(c1 + c2 + c_a12 + c_a21))^2*(k1*k2 – k_a12*k_a21) – (a1 + a2)^4*(c1*k2 + c2*k1 – c_a12*k_a21 – c_a21*k_a12)^2 symbolic math, simplify, coefficients, multivariable expression, polynomial, collect, factor MATLAB Answers — New Questions