Month: June 2024
Passing base64 content as knowledge source to custom copilot
Hello
We have a requirement where we need to fetch documents from DMS and pass them as knowledge source to custom copilot. The HTTP request returns base64 content of the document. What needs to be done to let copilot consume this as knowledge source and provide output.
Hello We have a requirement where we need to fetch documents from DMS and pass them as knowledge source to custom copilot. The HTTP request returns base64 content of the document. What needs to be done to let copilot consume this as knowledge source and provide output. Read More
Multi-thread parsing and loading thousands of csv files
I have a folder with 2500 csv files, each 15MB each. I currently have a script that reads each csv into a cell array container as follows at the bottom.
Unfortunately this serial process takes a very long time to open each csv one by one.
Ideally I would like to multi-thread or open multiple csv files in parallel and save them into either their own set of cell arrays per ‘thread’ and later combine and sort them, or into one big cell array as it is currently.
%% IMPORT FILES
directory = ‘\headnodeuserdataGeorgeANSTOANSTO Day 2DataD14’;
datafiles = dir(append(directory,’*.csv’));
N=length(datafiles);
a = 0;
data = cell(1,N);
f = waitbar(a,’Importing Data…’);
for i = 1:N
data{i} = read_csv(strcat(datafiles(i).folder, ”, datafiles(i).name));
waitbar(i/N,f);
end
waitbar(1,f);
close(f);I have a folder with 2500 csv files, each 15MB each. I currently have a script that reads each csv into a cell array container as follows at the bottom.
Unfortunately this serial process takes a very long time to open each csv one by one.
Ideally I would like to multi-thread or open multiple csv files in parallel and save them into either their own set of cell arrays per ‘thread’ and later combine and sort them, or into one big cell array as it is currently.
%% IMPORT FILES
directory = ‘\headnodeuserdataGeorgeANSTOANSTO Day 2DataD14’;
datafiles = dir(append(directory,’*.csv’));
N=length(datafiles);
a = 0;
data = cell(1,N);
f = waitbar(a,’Importing Data…’);
for i = 1:N
data{i} = read_csv(strcat(datafiles(i).folder, ”, datafiles(i).name));
waitbar(i/N,f);
end
waitbar(1,f);
close(f); I have a folder with 2500 csv files, each 15MB each. I currently have a script that reads each csv into a cell array container as follows at the bottom.
Unfortunately this serial process takes a very long time to open each csv one by one.
Ideally I would like to multi-thread or open multiple csv files in parallel and save them into either their own set of cell arrays per ‘thread’ and later combine and sort them, or into one big cell array as it is currently.
%% IMPORT FILES
directory = ‘\headnodeuserdataGeorgeANSTOANSTO Day 2DataD14’;
datafiles = dir(append(directory,’*.csv’));
N=length(datafiles);
a = 0;
data = cell(1,N);
f = waitbar(a,’Importing Data…’);
for i = 1:N
data{i} = read_csv(strcat(datafiles(i).folder, ”, datafiles(i).name));
waitbar(i/N,f);
end
waitbar(1,f);
close(f); multi-thread, csv, file, file processing, speed, file handling, file import, data import, multiple, parsing, parallel, parallel computing MATLAB Answers — New Questions
Missing pulse with variable phase shift and configuring action qualifier
Hello,
I am using simulink to program c2000 TMS320F28379D microcontroller to test a dc-ac converter. I have a variable phase-shift input to the epwm. My TBPRD is 330 and the phase shift counter value can vary from -165 to 165 for up count mode. I am having issue with missing pulses near zero crossing. I posted this question on TI forum and got a suggestion to implement a trigger to check if TBCTR is less than CMPA and the TBPHS value is greater than CMPA. Below is the link for the link for the question I asked and the response. I see that there is a way to specify software synchronization through input port. However, I am not sure how to configure TBCTR and input the suggested logic in simulink.
The suggested logic is toselect EPWMxSYNC as the T1 event source and then set the action taken on T1-up to be setting the EPWM2A high. In this way, the action from a T1-up event would supplement what the missing CMPA(up) = 1000 action would have done.
Keep in mind that when these conditions are not true, the T1-up/down events need to be disabled to prevent it from affecting your desired duty cycle. Let me know if you have further questions here or have made any other progress.
How can I implement this logic in the epwm to handle these missing pulses?
Question I posted on TI forum
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1372906/tms320f28379d-tms320f28379d-missing-pulses-with-variable-phase-shift
TI forum response
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1312604/tms320f28379d-tms320f28379d-phase-shift-pwm-missing-pulse
I have also attached the simulink file. Would it be possible to check if my epwm configurations are correct? I have a missing pulse near the zero crossing of ac line voltage since a sharp phase shift transition happens there.
Thank you in advance.Hello,
I am using simulink to program c2000 TMS320F28379D microcontroller to test a dc-ac converter. I have a variable phase-shift input to the epwm. My TBPRD is 330 and the phase shift counter value can vary from -165 to 165 for up count mode. I am having issue with missing pulses near zero crossing. I posted this question on TI forum and got a suggestion to implement a trigger to check if TBCTR is less than CMPA and the TBPHS value is greater than CMPA. Below is the link for the link for the question I asked and the response. I see that there is a way to specify software synchronization through input port. However, I am not sure how to configure TBCTR and input the suggested logic in simulink.
The suggested logic is toselect EPWMxSYNC as the T1 event source and then set the action taken on T1-up to be setting the EPWM2A high. In this way, the action from a T1-up event would supplement what the missing CMPA(up) = 1000 action would have done.
Keep in mind that when these conditions are not true, the T1-up/down events need to be disabled to prevent it from affecting your desired duty cycle. Let me know if you have further questions here or have made any other progress.
How can I implement this logic in the epwm to handle these missing pulses?
Question I posted on TI forum
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1372906/tms320f28379d-tms320f28379d-missing-pulses-with-variable-phase-shift
TI forum response
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1312604/tms320f28379d-tms320f28379d-phase-shift-pwm-missing-pulse
I have also attached the simulink file. Would it be possible to check if my epwm configurations are correct? I have a missing pulse near the zero crossing of ac line voltage since a sharp phase shift transition happens there.
Thank you in advance. Hello,
I am using simulink to program c2000 TMS320F28379D microcontroller to test a dc-ac converter. I have a variable phase-shift input to the epwm. My TBPRD is 330 and the phase shift counter value can vary from -165 to 165 for up count mode. I am having issue with missing pulses near zero crossing. I posted this question on TI forum and got a suggestion to implement a trigger to check if TBCTR is less than CMPA and the TBPHS value is greater than CMPA. Below is the link for the link for the question I asked and the response. I see that there is a way to specify software synchronization through input port. However, I am not sure how to configure TBCTR and input the suggested logic in simulink.
The suggested logic is toselect EPWMxSYNC as the T1 event source and then set the action taken on T1-up to be setting the EPWM2A high. In this way, the action from a T1-up event would supplement what the missing CMPA(up) = 1000 action would have done.
Keep in mind that when these conditions are not true, the T1-up/down events need to be disabled to prevent it from affecting your desired duty cycle. Let me know if you have further questions here or have made any other progress.
How can I implement this logic in the epwm to handle these missing pulses?
Question I posted on TI forum
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1372906/tms320f28379d-tms320f28379d-missing-pulses-with-variable-phase-shift
TI forum response
https://e2e.ti.com/support/microcontrollers/c2000-microcontrollers-group/c2000/f/c2000-microcontrollers-forum/1312604/tms320f28379d-tms320f28379d-phase-shift-pwm-missing-pulse
I have also attached the simulink file. Would it be possible to check if my epwm configurations are correct? I have a missing pulse near the zero crossing of ac line voltage since a sharp phase shift transition happens there.
Thank you in advance. missing pulse, trigger MATLAB Answers — New Questions
ROS2, MATLAB, GAZEBO.
I’m using ros2/Matlab and need the tranformations in a chart. How could I do that.I’m using ros2/Matlab and need the tranformations in a chart. How could I do that. I’m using ros2/Matlab and need the tranformations in a chart. How could I do that. ros2 MATLAB Answers — New Questions
Confidence interval calculation. Cannot compute confidence interval for imaginary values.
clear; clc
load Steady_State_Data.mat % this contains the wavelength of light and absorbance of substrate and sample
%% Preamble
% Fundamental constants
h = 4.0135667696*10^-15; % units: eV/ Hz
c = 3*10^8; % SI units
% Clean up of data to select range of values
Absorbance = log10(T_substrate./T_sample);
E = (h*c./(Lambda*10^-9));
e = E >= 0 & E <= 2.0; % returns boolean value of the indices that satisfies the logical condition defined above
A = find(e); % gives indices that are non-zero
N = length(A); % no. of elements that are non-zero
n = length(E) + 1;
% Data for fitting
E_p = E(n-N:167);
Abs = Absorbance(n-N:167) – min(Absorbance);
% Abs = Absorbance(n-N:167);
% Fitting function
function F = EM_SS(p, e_p)
for i = 1:numel(e_p)
E_p = e_p(i);
F(i) = p(1)*(2*pi*sqrt(p(4))/E_p)*(1/p(6))*…
(integral(@(E)sech(((E_p – E)./p(6)))*(1 + 10*p(5)*(E – p(3)) + …
126*(p(5))^2*(E – p(3))^2)/(1 – exp(-2*pi*sqrt(p(4)/(E – p(3))))), p(3), Inf, ‘ArrayValued’, 1)) + …
p(2)*(2*pi*p(4)^3/2)*1/p(6)*(…
(1/1^3)*sech((E_p – p(3) + p(4)/1^2)./p(6)) + …
(1/2^3)*sech((E_p – p(3) + p(4)/2^2)./p(6)) + …
(1/3^3)*sech((E_p – p(3) + p(4)/3^2)./p(6)) + …
(1/4^3)*sech((E_p – p(3) + p(4)/4^2)./p(6)) + …
(1/5^3)*sech((E_p – p(3) + p(4)/5^2)./p(6)) + …
(1/6^3)*sech((E_p – p(3) + p(4)/6^2)./p(6)) + …
(1/7^3)*sech((E_p – p(3) + p(4)/7^2)./p(6)));
end
F = F(:);
end
%% Curve fitting options
% Initial parameter guess and bounds
lb = [0,0,0,0,0,0]; ub = [Inf, Inf, 1.65, 0.03, Inf, Inf];
p0 = [0.13, 0.11, 1.4, 0.025, 0.12, 0.035]; % refer to the next line for their order
% p0 = [A1, A2, Eg, Eb, R, g]
% lsqcurvefit and choose between different algorithm that lsqcurvefit employs (3C1, comment those lines that are not choosen and uncomment the line that is choosen, if not, matlab will take the last line of "optim_lsq" by default)
optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘levenberg-marquardt’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘trust-region-reflective’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘interior-point’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% Solver for lsqcurvefit
[p, residual, exitflag, output, jacobian] = lsqcurvefit(@EM_SS, p0, E_p, Abs, lb, ub);
%% Standard error calculation
ci = nlparci(p, residual, ‘Jacobian’, jacobian);
lowerbar = p – ci(:,1);
uppperbar = ci(:,2) – p;
if p == real(p)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if residual == real(residual)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if J == real(J)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
%% Plot command
plot(E_p, Abs, ‘o’)
hold on
plot(E_p, EM_SS(p, E_p))
xlabel(‘Probe energy (eV)’)
ylabel(‘Absorbance.O.D’)
legend(‘Experimental Data’, ‘Fitted Curve’)
hold off
%% Parameter values (refer to command window)
p1 = p(1,1);
p2 = p(1,2);
p3 = p(1,3);
p4 = p(1,4);
p5 = p(1,5);
p6 = p(1,6);
X1 = [‘ A1 = ‘, num2str(p1)];
X2 = [‘ A2 = ‘, num2str(p2)];
X3 = [‘ Eg = ‘, num2str(p3)];
X4 = [‘ Eb = ‘, num2str(p4)];
X5 = [‘ R = ‘, num2str(p5)];
X6 = [‘ g = ‘, num2str(p6)];
disp(X1);
disp(X2);
disp(X3);
disp(X4);
disp(X5);
disp(X6);
Hi all, I am doing a fitting to my experimental data and I am now trying to calculate the confidence interval. However, it seems like there is something wrong with my code and the computer can’t really calculate it. May I ask where I am wrong? I tried following the instructions on this link but I still can’t get any results.clear; clc
load Steady_State_Data.mat % this contains the wavelength of light and absorbance of substrate and sample
%% Preamble
% Fundamental constants
h = 4.0135667696*10^-15; % units: eV/ Hz
c = 3*10^8; % SI units
% Clean up of data to select range of values
Absorbance = log10(T_substrate./T_sample);
E = (h*c./(Lambda*10^-9));
e = E >= 0 & E <= 2.0; % returns boolean value of the indices that satisfies the logical condition defined above
A = find(e); % gives indices that are non-zero
N = length(A); % no. of elements that are non-zero
n = length(E) + 1;
% Data for fitting
E_p = E(n-N:167);
Abs = Absorbance(n-N:167) – min(Absorbance);
% Abs = Absorbance(n-N:167);
% Fitting function
function F = EM_SS(p, e_p)
for i = 1:numel(e_p)
E_p = e_p(i);
F(i) = p(1)*(2*pi*sqrt(p(4))/E_p)*(1/p(6))*…
(integral(@(E)sech(((E_p – E)./p(6)))*(1 + 10*p(5)*(E – p(3)) + …
126*(p(5))^2*(E – p(3))^2)/(1 – exp(-2*pi*sqrt(p(4)/(E – p(3))))), p(3), Inf, ‘ArrayValued’, 1)) + …
p(2)*(2*pi*p(4)^3/2)*1/p(6)*(…
(1/1^3)*sech((E_p – p(3) + p(4)/1^2)./p(6)) + …
(1/2^3)*sech((E_p – p(3) + p(4)/2^2)./p(6)) + …
(1/3^3)*sech((E_p – p(3) + p(4)/3^2)./p(6)) + …
(1/4^3)*sech((E_p – p(3) + p(4)/4^2)./p(6)) + …
(1/5^3)*sech((E_p – p(3) + p(4)/5^2)./p(6)) + …
(1/6^3)*sech((E_p – p(3) + p(4)/6^2)./p(6)) + …
(1/7^3)*sech((E_p – p(3) + p(4)/7^2)./p(6)));
end
F = F(:);
end
%% Curve fitting options
% Initial parameter guess and bounds
lb = [0,0,0,0,0,0]; ub = [Inf, Inf, 1.65, 0.03, Inf, Inf];
p0 = [0.13, 0.11, 1.4, 0.025, 0.12, 0.035]; % refer to the next line for their order
% p0 = [A1, A2, Eg, Eb, R, g]
% lsqcurvefit and choose between different algorithm that lsqcurvefit employs (3C1, comment those lines that are not choosen and uncomment the line that is choosen, if not, matlab will take the last line of "optim_lsq" by default)
optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘levenberg-marquardt’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘trust-region-reflective’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘interior-point’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% Solver for lsqcurvefit
[p, residual, exitflag, output, jacobian] = lsqcurvefit(@EM_SS, p0, E_p, Abs, lb, ub);
%% Standard error calculation
ci = nlparci(p, residual, ‘Jacobian’, jacobian);
lowerbar = p – ci(:,1);
uppperbar = ci(:,2) – p;
if p == real(p)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if residual == real(residual)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if J == real(J)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
%% Plot command
plot(E_p, Abs, ‘o’)
hold on
plot(E_p, EM_SS(p, E_p))
xlabel(‘Probe energy (eV)’)
ylabel(‘Absorbance.O.D’)
legend(‘Experimental Data’, ‘Fitted Curve’)
hold off
%% Parameter values (refer to command window)
p1 = p(1,1);
p2 = p(1,2);
p3 = p(1,3);
p4 = p(1,4);
p5 = p(1,5);
p6 = p(1,6);
X1 = [‘ A1 = ‘, num2str(p1)];
X2 = [‘ A2 = ‘, num2str(p2)];
X3 = [‘ Eg = ‘, num2str(p3)];
X4 = [‘ Eb = ‘, num2str(p4)];
X5 = [‘ R = ‘, num2str(p5)];
X6 = [‘ g = ‘, num2str(p6)];
disp(X1);
disp(X2);
disp(X3);
disp(X4);
disp(X5);
disp(X6);
Hi all, I am doing a fitting to my experimental data and I am now trying to calculate the confidence interval. However, it seems like there is something wrong with my code and the computer can’t really calculate it. May I ask where I am wrong? I tried following the instructions on this link but I still can’t get any results. clear; clc
load Steady_State_Data.mat % this contains the wavelength of light and absorbance of substrate and sample
%% Preamble
% Fundamental constants
h = 4.0135667696*10^-15; % units: eV/ Hz
c = 3*10^8; % SI units
% Clean up of data to select range of values
Absorbance = log10(T_substrate./T_sample);
E = (h*c./(Lambda*10^-9));
e = E >= 0 & E <= 2.0; % returns boolean value of the indices that satisfies the logical condition defined above
A = find(e); % gives indices that are non-zero
N = length(A); % no. of elements that are non-zero
n = length(E) + 1;
% Data for fitting
E_p = E(n-N:167);
Abs = Absorbance(n-N:167) – min(Absorbance);
% Abs = Absorbance(n-N:167);
% Fitting function
function F = EM_SS(p, e_p)
for i = 1:numel(e_p)
E_p = e_p(i);
F(i) = p(1)*(2*pi*sqrt(p(4))/E_p)*(1/p(6))*…
(integral(@(E)sech(((E_p – E)./p(6)))*(1 + 10*p(5)*(E – p(3)) + …
126*(p(5))^2*(E – p(3))^2)/(1 – exp(-2*pi*sqrt(p(4)/(E – p(3))))), p(3), Inf, ‘ArrayValued’, 1)) + …
p(2)*(2*pi*p(4)^3/2)*1/p(6)*(…
(1/1^3)*sech((E_p – p(3) + p(4)/1^2)./p(6)) + …
(1/2^3)*sech((E_p – p(3) + p(4)/2^2)./p(6)) + …
(1/3^3)*sech((E_p – p(3) + p(4)/3^2)./p(6)) + …
(1/4^3)*sech((E_p – p(3) + p(4)/4^2)./p(6)) + …
(1/5^3)*sech((E_p – p(3) + p(4)/5^2)./p(6)) + …
(1/6^3)*sech((E_p – p(3) + p(4)/6^2)./p(6)) + …
(1/7^3)*sech((E_p – p(3) + p(4)/7^2)./p(6)));
end
F = F(:);
end
%% Curve fitting options
% Initial parameter guess and bounds
lb = [0,0,0,0,0,0]; ub = [Inf, Inf, 1.65, 0.03, Inf, Inf];
p0 = [0.13, 0.11, 1.4, 0.025, 0.12, 0.035]; % refer to the next line for their order
% p0 = [A1, A2, Eg, Eb, R, g]
% lsqcurvefit and choose between different algorithm that lsqcurvefit employs (3C1, comment those lines that are not choosen and uncomment the line that is choosen, if not, matlab will take the last line of "optim_lsq" by default)
optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘levenberg-marquardt’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘trust-region-reflective’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% optim_lsq = optimoptions(‘lsqcurvefit’, ‘Algorithm’, ‘interior-point’, ‘MaxFunctionEvaluations’,1000, ‘MaxIterations’, 1000, ‘FunctionTolerance’,10^-9, ‘StepTolerance’, 10^-9);
% Solver for lsqcurvefit
[p, residual, exitflag, output, jacobian] = lsqcurvefit(@EM_SS, p0, E_p, Abs, lb, ub);
%% Standard error calculation
ci = nlparci(p, residual, ‘Jacobian’, jacobian);
lowerbar = p – ci(:,1);
uppperbar = ci(:,2) – p;
if p == real(p)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if residual == real(residual)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
if J == real(J)
disp(‘Real’)
else
disp(‘Complex with non-zero imaginary’)
end
%% Plot command
plot(E_p, Abs, ‘o’)
hold on
plot(E_p, EM_SS(p, E_p))
xlabel(‘Probe energy (eV)’)
ylabel(‘Absorbance.O.D’)
legend(‘Experimental Data’, ‘Fitted Curve’)
hold off
%% Parameter values (refer to command window)
p1 = p(1,1);
p2 = p(1,2);
p3 = p(1,3);
p4 = p(1,4);
p5 = p(1,5);
p6 = p(1,6);
X1 = [‘ A1 = ‘, num2str(p1)];
X2 = [‘ A2 = ‘, num2str(p2)];
X3 = [‘ Eg = ‘, num2str(p3)];
X4 = [‘ Eb = ‘, num2str(p4)];
X5 = [‘ R = ‘, num2str(p5)];
X6 = [‘ g = ‘, num2str(p6)];
disp(X1);
disp(X2);
disp(X3);
disp(X4);
disp(X5);
disp(X6);
Hi all, I am doing a fitting to my experimental data and I am now trying to calculate the confidence interval. However, it seems like there is something wrong with my code and the computer can’t really calculate it. May I ask where I am wrong? I tried following the instructions on this link but I still can’t get any results. confidence interval, error bars MATLAB Answers — New Questions
Chronically slow PC running Windows 10 Home
Hi all.
My Windows 10 Home PC has been experiencing slow performance for a long time. Despite having a decent specification, including an Intel i7 processor and 16 GB of RAM, my PC takes a long time to start up and load applications. It’s not just the startup process that’s slow, but also the general responsiveness of the system. For example, it takes a few seconds to open a simple text file, and even basic tasks like browsing the web or checking email can be slow and unresponsive. I’m looking for any advice or suggestions that might help me identify and fix the underlying cause of this problem.
Hi all.My Windows 10 Home PC has been experiencing slow performance for a long time. Despite having a decent specification, including an Intel i7 processor and 16 GB of RAM, my PC takes a long time to start up and load applications. It’s not just the startup process that’s slow, but also the general responsiveness of the system. For example, it takes a few seconds to open a simple text file, and even basic tasks like browsing the web or checking email can be slow and unresponsive. I’m looking for any advice or suggestions that might help me identify and fix the underlying cause of this problem. Read More
Unable to Log into Epic Account via System Proxy on Domain-Joined Hosts in an Intranet Environment
Hello everyone,
I am using Windows 10 Enterprise LTSC, version 19044.4412. My domain controller and the computers within the domain are all in an intranet environment. Initially, after installing the system and joining the domain, configuring the system proxy allows me to log into the Epic Launcher and my Epic account without any issues.
However, after a certain period, be it a week or a month, attempting to log into my Epic account using the system proxy causes the login process to stall on the offline login page. The proxy server shows traffic passing through normally, but the login fails.
At this point, if I remove the computer from the domain controller and join the WORKGROUP workgroup, then configure the system proxy again, the login page for Epic appears normally. Subsequently, rejoining the domain and configuring the system proxy once more also allows me to log in to Epic without any issues.
Any assistance or insights into resolving this issue would be greatly appreciated. Thank you.
Hello everyone,I am using Windows 10 Enterprise LTSC, version 19044.4412. My domain controller and the computers within the domain are all in an intranet environment. Initially, after installing the system and joining the domain, configuring the system proxy allows me to log into the Epic Launcher and my Epic account without any issues.However, after a certain period, be it a week or a month, attempting to log into my Epic account using the system proxy causes the login process to stall on the offline login page. The proxy server shows traffic passing through normally, but the login fails.At this point, if I remove the computer from the domain controller and join the WORKGROUP workgroup, then configure the system proxy again, the login page for Epic appears normally. Subsequently, rejoining the domain and configuring the system proxy once more also allows me to log in to Epic without any issues.Any assistance or insights into resolving this issue would be greatly appreciated. Thank you. Read More
Query Problem
Good morning,
I wonder if someone could help, I have two queries based on date ranges, I would like to see them side by side but cant work out how to.
select customer, sum(nett)Nett,sum(items)Items,Sum(area)M2
from goorders
where ordered BETWEEN GETDATE()-7 AND GETDATE()
group by customer
order by Nett DESC
select customer, sum(nett)Nett,sum(items)Items,Sum(area)M2
from goorders
where ordered BETWEEN GETDATE()-15 AND GETDATE()-8
group by customer
order by Nett DESC
I would like to see as :-
Customer Nett (This Week) Nett (Last Week) Items (This Week)Items (Last Week)M2 ()This Week)M2 (Lats Week)
Can anyone better than me help please,, i would be vert grateful.
JD
Good morning, I wonder if someone could help, I have two queries based on date ranges, I would like to see them side by side but cant work out how to. select customer, sum(nett)Nett,sum(items)Items,Sum(area)M2from goorderswhere ordered BETWEEN GETDATE()-7 AND GETDATE()group by customerorder by Nett DESC select customer, sum(nett)Nett,sum(items)Items,Sum(area)M2from goorderswhere ordered BETWEEN GETDATE()-15 AND GETDATE()-8group by customerorder by Nett DESC I would like to see as :- Customer Nett (This Week) Nett (Last Week) Items (This Week)Items (Last Week)M2 ()This Week)M2 (Lats Week) Can anyone better than me help please,, i would be vert grateful. JD Read More
using bookings email address as full mailbox
I have created a bookings page. The bookings page then gets an email address.
In the config of the bookings page it is possible to assign a license to this booking page mail address so that it becomes a full mailbox. After doing this I can login via web to the mailbox. I can handle all email in the mailbox of the booking page and I have direct normal access to the calendar.
If I would use a different mailbox then the mailbox of the booking page, then I would need to add this mailbox as staff. The calendar of that mailbox will then be the leading “staff” and I can add it as “team member” like below.
In line with that I am now added the mailbox of the booking page itself as staff. But there is an unexpected result. The staff is set to guest and can not be changed. It looks like this:
This seems to give all kind of limitation – of which the most visible is that the “events on the office calendar affect availability” is greyed out. This seems not how it should be. How can I make that I can set this bookings page mailbox to “team member”. Does that require another type of license or do we need to make changed is the office / exchange admin settings?
I have created a bookings page. The bookings page then gets an email address. In the config of the bookings page it is possible to assign a license to this booking page mail address so that it becomes a full mailbox. After doing this I can login via web to the mailbox. I can handle all email in the mailbox of the booking page and I have direct normal access to the calendar. If I would use a different mailbox then the mailbox of the booking page, then I would need to add this mailbox as staff. The calendar of that mailbox will then be the leading “staff” and I can add it as “team member” like below. In line with that I am now added the mailbox of the booking page itself as staff. But there is an unexpected result. The staff is set to guest and can not be changed. It looks like this: This seems to give all kind of limitation – of which the most visible is that the “events on the office calendar affect availability” is greyed out. This seems not how it should be. How can I make that I can set this bookings page mailbox to “team member”. Does that require another type of license or do we need to make changed is the office / exchange admin settings? Read More
Is it real mandatory to pass AZ-104 before I can take AZ-305?
I see bit confusing data around the learn portal and the forum, so asking this again. Is it real mandatory to pass AZ-104 before I can take AZ-305? Or is that just a recommendation/ guidance only?
Currently I hold AZ-900 and want to get AZ-305 without doing AZ-104. I do work with Azure in my work place, so I’m hands on. Only reason I skip the AZ-104 is that I work on Azure and also I want to speed up the certification path towards AZ-305 if that makes sense. Please clarify.
I see bit confusing data around the learn portal and the forum, so asking this again. Is it real mandatory to pass AZ-104 before I can take AZ-305? Or is that just a recommendation/ guidance only? Currently I hold AZ-900 and want to get AZ-305 without doing AZ-104. I do work with Azure in my work place, so I’m hands on. Only reason I skip the AZ-104 is that I work on Azure and also I want to speed up the certification path towards AZ-305 if that makes sense. Please clarify. Read More
Is there something wrong with M365 spam filtering?
Hi All,
I work for a small MSP and have received multiple reports of legitimate mail being flagged as spam by 365 and sent to the recipient’s Outlook Junkbox, seemingly at random.
It’s happening to too many clients in such a short space of time to be anything other than a service issue but there’s no mention of it anywhere.
Is anyone else noticing anything similar?
Hi All, I work for a small MSP and have received multiple reports of legitimate mail being flagged as spam by 365 and sent to the recipient’s Outlook Junkbox, seemingly at random. It’s happening to too many clients in such a short space of time to be anything other than a service issue but there’s no mention of it anywhere. Is anyone else noticing anything similar? Read More
Exclude sensitive labelled document from Microsoft 365 copilot lookup
Hi,
In my organization i have AIP labels which is used only for classification not for protection. while implementing copilot, is it possible to exclude copilot access to sensitive labelled documents while looking up based on user prompt. Thanks.
Hi, In my organization i have AIP labels which is used only for classification not for protection. while implementing copilot, is it possible to exclude copilot access to sensitive labelled documents while looking up based on user prompt. Thanks. Read More
Where are all the IT-heroes? The challenges of adoption.
AI-assisted: This article was first written in it’s entirety by me, then I asked ChatGPT to Factcheck it for me, and the resulting changes with reference to sourves where then reevaltuated and modified before posting. If you like to see the text in it’s various stages, please tell me.
This is a short version of a Norwegian article I’m writing where I focused on the lack of IT competense in SMB. The core issues might scale to enterprise, so please join the discussion with reflections from your experience.
Historical Context
During the 90s and early 2000s, many companies built their own IT infrastructure, often with in-house IT departments managing servers and business-critical applications. However, as the MSP model gained traction, companies began outsourcing their IT needs to reduce costs and improve efficiency. By the early 2000s, MSPs became a popular solution for businesses of all sizes, offering services such as remote monitoring, management of servers and networks, and later expanding into areas like managed security and cloud services (Fast IT Support Calif/Hawaii) (The AME Group).
Present Issues
Today, many companies, especially SMBs, have significantly downsized their internal IT departments, if they exist at all. The role of CTO has often become more of a titular position, with many strategic IT decisions falling to CFOs or even general administrative staff. This shift has led to a lack of strategic IT leadership, making companies more reactive than proactive in their technology adoption (Fast IT Support Calif/Hawaii).
Even larger companies with dedicated IT groups often struggle with limited budgets, leading to a focus on immediate issues (“firefighting”) rather than long-term IT strategy and infrastructure improvements.
Pressing Issues and Resulting Problems
One of the main issues arising from this lack of in-house IT competence is that tool and software decisions are frequently made based on sales pitches rather than strategic alignment with the company’s needs. This often results in a fragmented IT architecture, where information is scattered across multiple systems, and legacy systems remain in use because their data hasn’t been migrated properly (The AME Group) (Mindsight).
SMBs also tend to look to enterprise-level best practices, which aren’t always feasible due to resource constraints. This mismatch can further complicate their IT environments, leading to inefficiencies and increased administrative burden.
AI and Copilot
Introducing advanced tools like AI and Copilot into this chaotic environment can indeed be challenging. While these technologies hold significant promise, they are not a panacea for deep-rooted structural and organizational issues. A foundational restructuring of IT systems and processes might be necessary to fully leverage these innovations (Mindsight).
The discussion
Did I hit close to home or miss the field? What are your reflections on the challenges we meet when introducing a new tool? Is this limited to small companies in a small country called Norway?
Thanks in advance for sharing your thoughts and reflections!
AI-assisted: This article was first written in it’s entirety by me, then I asked ChatGPT to Factcheck it for me, and the resulting changes with reference to sourves where then reevaltuated and modified before posting. If you like to see the text in it’s various stages, please tell me.This is a short version of a Norwegian article I’m writing where I focused on the lack of IT competense in SMB. The core issues might scale to enterprise, so please join the discussion with reflections from your experience. Historical ContextDuring the 90s and early 2000s, many companies built their own IT infrastructure, often with in-house IT departments managing servers and business-critical applications. However, as the MSP model gained traction, companies began outsourcing their IT needs to reduce costs and improve efficiency. By the early 2000s, MSPs became a popular solution for businesses of all sizes, offering services such as remote monitoring, management of servers and networks, and later expanding into areas like managed security and cloud services (Fast IT Support Calif/Hawaii) (The AME Group). Present IssuesToday, many companies, especially SMBs, have significantly downsized their internal IT departments, if they exist at all. The role of CTO has often become more of a titular position, with many strategic IT decisions falling to CFOs or even general administrative staff. This shift has led to a lack of strategic IT leadership, making companies more reactive than proactive in their technology adoption (Fast IT Support Calif/Hawaii).Even larger companies with dedicated IT groups often struggle with limited budgets, leading to a focus on immediate issues (“firefighting”) rather than long-term IT strategy and infrastructure improvements. Pressing Issues and Resulting ProblemsOne of the main issues arising from this lack of in-house IT competence is that tool and software decisions are frequently made based on sales pitches rather than strategic alignment with the company’s needs. This often results in a fragmented IT architecture, where information is scattered across multiple systems, and legacy systems remain in use because their data hasn’t been migrated properly (The AME Group) (Mindsight). SMBs also tend to look to enterprise-level best practices, which aren’t always feasible due to resource constraints. This mismatch can further complicate their IT environments, leading to inefficiencies and increased administrative burden. AI and CopilotIntroducing advanced tools like AI and Copilot into this chaotic environment can indeed be challenging. While these technologies hold significant promise, they are not a panacea for deep-rooted structural and organizational issues. A foundational restructuring of IT systems and processes might be necessary to fully leverage these innovations (Mindsight). The discussionDid I hit close to home or miss the field? What are your reflections on the challenges we meet when introducing a new tool? Is this limited to small companies in a small country called Norway?Thanks in advance for sharing your thoughts and reflections! Read More
MS Access passthrough query performs very slow when executing a stored procedure
I have passthrough query in my Access frontend that I use to import data from a sql server db. Before executing the passthrough query I pass the connectionstring and the sql to the querydef object and then I pass the output to a DAO recordset:
qdf.Connect = maconnectionstring
qdf.SQL = “EXEC MyStoredProc ” & param1 & “, ” param2
Set rstDAO = qdf.OpenRecordset
When I execute the stored procedure in SSMS using the same parameters it takes 2 seconds. In MS Access I need to set the ODBCTimeout parameter to 4 minutes to avoid a timeout error.
Any ideas how to improve the performance of the passthrough query?
I have passthrough query in my Access frontend that I use to import data from a sql server db. Before executing the passthrough query I pass the connectionstring and the sql to the querydef object and then I pass the output to a DAO recordset:qdf.Connect = maconnectionstring
qdf.SQL = “EXEC MyStoredProc ” & param1 & “, ” param2
Set rstDAO = qdf.OpenRecordset When I execute the stored procedure in SSMS using the same parameters it takes 2 seconds. In MS Access I need to set the ODBCTimeout parameter to 4 minutes to avoid a timeout error.Any ideas how to improve the performance of the passthrough query? Read More
Weird sound problem Win 11 computer
I’ve been experiencing a weird sound problem on my Windows 11 computer. Whenever I’m watching videos, listening to music, or even just browsing the internet, I start to hear strange sounds coming from my speakers. These sounds are not part of the content I’m playing, but rather seem to be generated by the computer itself. I’ve tried restarting my computer, updating my drivers, and even checking for malware, but the problem persists. I’m not sure what’s causing this issue, I need your help.
I’ve been experiencing a weird sound problem on my Windows 11 computer. Whenever I’m watching videos, listening to music, or even just browsing the internet, I start to hear strange sounds coming from my speakers. These sounds are not part of the content I’m playing, but rather seem to be generated by the computer itself. I’ve tried restarting my computer, updating my drivers, and even checking for malware, but the problem persists. I’m not sure what’s causing this issue, I need your help. Read More
Create and deploy a web app with Python and Azure – A video tutorial!
Have you ever wanted to create your own website or web service, but felt overwhelmed about how to get started with web development? Or have you already built a web app with Python, but wondered how you can deploy it online? If so, you might be interested in watching this video by and Pamela Fox and Renee Noble, two Python Cloud Advocates at Microsoft with a passion for web development and education!
They’ll show you how to create a web app with Python Flask (or it’s async-sibling Quart) and show you how you can deploy you can take your web app live by deploying it on Azure. This is the first episode in this live stream series (which you can also catch on demand!) by Pamela and friends covering different web dev frameworks and practices! (And you can catch it in Spanish too!)
In this video, you will learn how the web works and why you (likely) need a web server to serve up your web app. You will also discover how a backend web framework, like Flask, helps you develop your web server quickly and easily and how it lets you use templates to build dynamic sites – whether it’s your first site or a complex full-scale service.
Want to improve your web server throughput? Find out how you can port your app to Quart, the asynchronous version of Flask. And of course, you’ll see how you can deploy your web apps to Azure App Service in minutes and learn what other deployment options you might be interested in seeking out depending on your project.
The video is perfect for beginners who have some familiarity with Python, but no prior experience with web development or deployment. It is also suitable for intermediate or advanced developers who want to refresh their skills or learn new tricks. Take a look at the video to learn some concepts and follow along with our coding and deployment demos! You can grab the demo repo here and check out the slides too.
Microsoft Tech Community – Latest Blogs –Read More
User equipment in Small cell Network!
How to define a user equipment in a picocell or anykind of cell?How to define a user equipment in a picocell or anykind of cell? How to define a user equipment in a picocell or anykind of cell? small cells, user equipment MATLAB Answers — New Questions
plot some nodes from an .txt file
Hi! I would like to hope if there is an easy way to plot only the outermost nodes (the red nodes in the figure).
I am leaving the filename.txt file representing the nodes.Hi! I would like to hope if there is an easy way to plot only the outermost nodes (the red nodes in the figure).
I am leaving the filename.txt file representing the nodes. Hi! I would like to hope if there is an easy way to plot only the outermost nodes (the red nodes in the figure).
I am leaving the filename.txt file representing the nodes. nodes, plot, plotting, txt, text file, file MATLAB Answers — New Questions
Why/How is Circshift so fast?
I have the following function in Matlab:
function [RawDataKK]=DataCompressKKTest(Data,RXangle,s)
tSize=size(Data,1);
xSize=size(Data,2);
TXSize=size(Data,3);
RXSize=numel(RXangle);
RawDataKK=zeros(tSize,TXSize,RXSize);
DataTemp=zeros(tSize,xSize);
slope = s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
for nR=1:RXSize
for nT=1:TXSize
for nx=1:xSize
DataTemp(:,nx)=circshift(Data(:,nx,nT),nShift(nR,nx));
end
RawDataKK(:,nT,nR)=sum(DataTemp,2);
end
end
end
I tried implementing it as a C++ function using the Eigen toolbox and the Matlab Data API for C++. The basic logic of the C++ implementation uses a reindexing approach to implement circshift.
“` lang-cpp
#include <iostream>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <Eigen/Dense>
#include <MatlabDataArray.hpp>
#include <cmath>
#include <math.h>
#include <omp.h>
class MexFunction : public matlab::mex::Function {
public:
// Factory to create MATLAB data arrays
matlab::data::ArrayFactory factory;
void operator()(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
checkArguments(outputs, inputs);
// Initialize input parameters
int tSize = inputs[0][0];
int xSize = inputs[0][1];
int TXSize = inputs[0][2];
int RXSize = inputs[0][3];
int RFlen = inputs[0][4];
auto ptr = getDataPtr<std::complex<double>>(inputs[1]);
Eigen::Map< const Eigen::MatrixXcd > Data( ptr, RFlen, xSize );
double s = inputs[2][0];
auto ptr2 = getDataPtr<double>(inputs[3]);
Eigen::Map< const Eigen::VectorXd > RXangle( ptr2, RXSize );
auto ptr3 = getDataPtr<int32_t>(inputs[4]);
Eigen::Map< const Eigen::MatrixXi > indices( ptr3, tSize, xSize*RXSize );
outputs[0] = factory.createArray<std::complex<double>>({static_cast<size_t>(tSize),static_cast<size_t>(TXSize*RXSize)});
auto ptrRecon = getOutDataPtr<std::complex<double>>(outputs[0]);
Eigen::Map<Eigen::MatrixXcd> RFDataKK(ptrRecon,tSize,TXSize*RXSize);
// Get num threads
int numThreads = omp_get_max_threads();
int nProc = omp_get_num_procs();
omp_set_num_threads(nProc*2);
#pragma omp parallel for
for (int nR = 0; nR < RXSize; nR++) {
Eigen::MatrixXcd shiftedData = Eigen::MatrixXcd::Zero(tSize*TXSize,xSize);
for (int nT = 0; nT < TXSize; nT++){
extractData(shiftedData(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
Data(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
indices(Eigen::all,Eigen::seq(nR*xSize, (nR+1)*xSize-1)), xSize, tSize);
}
RFDataKK(Eigen::all,Eigen::seq(nR*TXSize,(nR+1)*TXSize-1) ) = shiftedData.rowwise().sum().reshaped(tSize,TXSize);
}
}
void extractData(Eigen::Ref<Eigen::MatrixXcd> shiftedData, const Eigen::Ref<const Eigen::MatrixXcd>& Data,
const Eigen::Ref<const Eigen::MatrixXi>& indices, const long int& xSize, const long int& tSize) {
for (int i = 0; i < tSize*xSize; i++) {
int nt = i % tSize;
int nx = i / tSize;
int row = indices(nt,nx) % tSize;
int col = indices(nt,nx) / tSize;
shiftedData(row,col) = Data(nt,nx);
}
}
void checkArguments(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
matlab::data::ArrayFactory factory;
// TODO: Implement remaining checks
}
template <typename T>
const T* getDataPtr(matlab::data::Array arr) {
const matlab::data::TypedArray<T> arr_t = arr;
matlab::data::TypedIterator<const T> it(arr_t.begin());
return it.operator->();
}
template <typename T>
T* getOutDataPtr(matlab::data::Array& arr) {
auto range = matlab::data::getWritableElements<T>(arr);
return range.begin().operator->();
}
};
“`
Where I precompute the values of indices using the following Matlab function:
function linIndices=DataCompressKKIndices(RXangle,s,tSize,xSize,TXSize,RXSize)
shiftedData = zeros(tSize, xSize, TXSize);
RFDatKK = zeros(tSize,TXSize,RXSize);
RFDataKK = zeros(tSize,xSize*RXSize);
slope=s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
indices = mod((0:tSize-1).’ + reshape(nShift.’,[],1).’,tSize);
linIndices = indices + repmat((0:xSize-1)*tSize,1,RXSize);
end
When I perform a speed test in Matlab after compiling with the following parameters (with mingw GCC version 8.3), I find that I have not gained a meaningful speedup for the array sizes I am working with. OpenMP parallelization should at least give me an order of magnitude speed up (I have 24 threads on my machine). There are two questions that yields:
1. Why is a reindexing approach slower than doing circshift? A reindexing approach in Matlab (not shown here) is almost 2x slower than using nested for loops and circshift.
2. What exactly could circshift be doing under the hood that is so efficient? Is there some funky pointer arithmetic that could accomplish the functionality of circshift?
Compilation and testing code:
%% Compilation
mingwFlags = {‘CXXFLAGS="$CXXFLAGS -march=native -std=c++14 -fno-math-errno -ffast-math -fopenmp -DNDEBUG -w -Wno-error"’,…
‘LDFLAGS="$LDFLAGS -fopenmp"’,’CXXOPTIMFLAGS="-O3"’};
% ipath is the path to the eigen library.
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKK.cpp’); toc
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKKIndices.cpp’); toc
%% Initialize some random data
…
% tSize = 2688, xSize = 192, TXSize = 15, RXSize = 16 for trial cases
%% Timing Test
nTrials = 100;
T = zeros(nTrials,4);
for i = 1:nTrials
tic; indices = DataCompressKKIndices(p.RXangle,s,double(p.szRFframe+1),double(p.numEl),p.na,p.nRX);
RawDataKK3 = CompressKKIndices(param,cRF(1:param(5),p.ConnMap),s,p.RXangle,int32(indices));
RawDataKK3 = reshape(RawDataKK3,[p.szRFframe+1,p.na,p.nRX]); T(i,1) = toc;
tic; RawDataKK=DataCompressKKTest(Data,p.RXangle,s); T(i,2) = toc;
end
chkT = mean(T,1)
Timing Results:
chkT =
C++ version: 0.4867 0.8299I have the following function in Matlab:
function [RawDataKK]=DataCompressKKTest(Data,RXangle,s)
tSize=size(Data,1);
xSize=size(Data,2);
TXSize=size(Data,3);
RXSize=numel(RXangle);
RawDataKK=zeros(tSize,TXSize,RXSize);
DataTemp=zeros(tSize,xSize);
slope = s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
for nR=1:RXSize
for nT=1:TXSize
for nx=1:xSize
DataTemp(:,nx)=circshift(Data(:,nx,nT),nShift(nR,nx));
end
RawDataKK(:,nT,nR)=sum(DataTemp,2);
end
end
end
I tried implementing it as a C++ function using the Eigen toolbox and the Matlab Data API for C++. The basic logic of the C++ implementation uses a reindexing approach to implement circshift.
“` lang-cpp
#include <iostream>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <Eigen/Dense>
#include <MatlabDataArray.hpp>
#include <cmath>
#include <math.h>
#include <omp.h>
class MexFunction : public matlab::mex::Function {
public:
// Factory to create MATLAB data arrays
matlab::data::ArrayFactory factory;
void operator()(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
checkArguments(outputs, inputs);
// Initialize input parameters
int tSize = inputs[0][0];
int xSize = inputs[0][1];
int TXSize = inputs[0][2];
int RXSize = inputs[0][3];
int RFlen = inputs[0][4];
auto ptr = getDataPtr<std::complex<double>>(inputs[1]);
Eigen::Map< const Eigen::MatrixXcd > Data( ptr, RFlen, xSize );
double s = inputs[2][0];
auto ptr2 = getDataPtr<double>(inputs[3]);
Eigen::Map< const Eigen::VectorXd > RXangle( ptr2, RXSize );
auto ptr3 = getDataPtr<int32_t>(inputs[4]);
Eigen::Map< const Eigen::MatrixXi > indices( ptr3, tSize, xSize*RXSize );
outputs[0] = factory.createArray<std::complex<double>>({static_cast<size_t>(tSize),static_cast<size_t>(TXSize*RXSize)});
auto ptrRecon = getOutDataPtr<std::complex<double>>(outputs[0]);
Eigen::Map<Eigen::MatrixXcd> RFDataKK(ptrRecon,tSize,TXSize*RXSize);
// Get num threads
int numThreads = omp_get_max_threads();
int nProc = omp_get_num_procs();
omp_set_num_threads(nProc*2);
#pragma omp parallel for
for (int nR = 0; nR < RXSize; nR++) {
Eigen::MatrixXcd shiftedData = Eigen::MatrixXcd::Zero(tSize*TXSize,xSize);
for (int nT = 0; nT < TXSize; nT++){
extractData(shiftedData(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
Data(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
indices(Eigen::all,Eigen::seq(nR*xSize, (nR+1)*xSize-1)), xSize, tSize);
}
RFDataKK(Eigen::all,Eigen::seq(nR*TXSize,(nR+1)*TXSize-1) ) = shiftedData.rowwise().sum().reshaped(tSize,TXSize);
}
}
void extractData(Eigen::Ref<Eigen::MatrixXcd> shiftedData, const Eigen::Ref<const Eigen::MatrixXcd>& Data,
const Eigen::Ref<const Eigen::MatrixXi>& indices, const long int& xSize, const long int& tSize) {
for (int i = 0; i < tSize*xSize; i++) {
int nt = i % tSize;
int nx = i / tSize;
int row = indices(nt,nx) % tSize;
int col = indices(nt,nx) / tSize;
shiftedData(row,col) = Data(nt,nx);
}
}
void checkArguments(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
matlab::data::ArrayFactory factory;
// TODO: Implement remaining checks
}
template <typename T>
const T* getDataPtr(matlab::data::Array arr) {
const matlab::data::TypedArray<T> arr_t = arr;
matlab::data::TypedIterator<const T> it(arr_t.begin());
return it.operator->();
}
template <typename T>
T* getOutDataPtr(matlab::data::Array& arr) {
auto range = matlab::data::getWritableElements<T>(arr);
return range.begin().operator->();
}
};
“`
Where I precompute the values of indices using the following Matlab function:
function linIndices=DataCompressKKIndices(RXangle,s,tSize,xSize,TXSize,RXSize)
shiftedData = zeros(tSize, xSize, TXSize);
RFDatKK = zeros(tSize,TXSize,RXSize);
RFDataKK = zeros(tSize,xSize*RXSize);
slope=s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
indices = mod((0:tSize-1).’ + reshape(nShift.’,[],1).’,tSize);
linIndices = indices + repmat((0:xSize-1)*tSize,1,RXSize);
end
When I perform a speed test in Matlab after compiling with the following parameters (with mingw GCC version 8.3), I find that I have not gained a meaningful speedup for the array sizes I am working with. OpenMP parallelization should at least give me an order of magnitude speed up (I have 24 threads on my machine). There are two questions that yields:
1. Why is a reindexing approach slower than doing circshift? A reindexing approach in Matlab (not shown here) is almost 2x slower than using nested for loops and circshift.
2. What exactly could circshift be doing under the hood that is so efficient? Is there some funky pointer arithmetic that could accomplish the functionality of circshift?
Compilation and testing code:
%% Compilation
mingwFlags = {‘CXXFLAGS="$CXXFLAGS -march=native -std=c++14 -fno-math-errno -ffast-math -fopenmp -DNDEBUG -w -Wno-error"’,…
‘LDFLAGS="$LDFLAGS -fopenmp"’,’CXXOPTIMFLAGS="-O3"’};
% ipath is the path to the eigen library.
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKK.cpp’); toc
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKKIndices.cpp’); toc
%% Initialize some random data
…
% tSize = 2688, xSize = 192, TXSize = 15, RXSize = 16 for trial cases
%% Timing Test
nTrials = 100;
T = zeros(nTrials,4);
for i = 1:nTrials
tic; indices = DataCompressKKIndices(p.RXangle,s,double(p.szRFframe+1),double(p.numEl),p.na,p.nRX);
RawDataKK3 = CompressKKIndices(param,cRF(1:param(5),p.ConnMap),s,p.RXangle,int32(indices));
RawDataKK3 = reshape(RawDataKK3,[p.szRFframe+1,p.na,p.nRX]); T(i,1) = toc;
tic; RawDataKK=DataCompressKKTest(Data,p.RXangle,s); T(i,2) = toc;
end
chkT = mean(T,1)
Timing Results:
chkT =
C++ version: 0.4867 0.8299 I have the following function in Matlab:
function [RawDataKK]=DataCompressKKTest(Data,RXangle,s)
tSize=size(Data,1);
xSize=size(Data,2);
TXSize=size(Data,3);
RXSize=numel(RXangle);
RawDataKK=zeros(tSize,TXSize,RXSize);
DataTemp=zeros(tSize,xSize);
slope = s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
for nR=1:RXSize
for nT=1:TXSize
for nx=1:xSize
DataTemp(:,nx)=circshift(Data(:,nx,nT),nShift(nR,nx));
end
RawDataKK(:,nT,nR)=sum(DataTemp,2);
end
end
end
I tried implementing it as a C++ function using the Eigen toolbox and the Matlab Data API for C++. The basic logic of the C++ implementation uses a reindexing approach to implement circshift.
“` lang-cpp
#include <iostream>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <Eigen/Dense>
#include <MatlabDataArray.hpp>
#include <cmath>
#include <math.h>
#include <omp.h>
class MexFunction : public matlab::mex::Function {
public:
// Factory to create MATLAB data arrays
matlab::data::ArrayFactory factory;
void operator()(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
checkArguments(outputs, inputs);
// Initialize input parameters
int tSize = inputs[0][0];
int xSize = inputs[0][1];
int TXSize = inputs[0][2];
int RXSize = inputs[0][3];
int RFlen = inputs[0][4];
auto ptr = getDataPtr<std::complex<double>>(inputs[1]);
Eigen::Map< const Eigen::MatrixXcd > Data( ptr, RFlen, xSize );
double s = inputs[2][0];
auto ptr2 = getDataPtr<double>(inputs[3]);
Eigen::Map< const Eigen::VectorXd > RXangle( ptr2, RXSize );
auto ptr3 = getDataPtr<int32_t>(inputs[4]);
Eigen::Map< const Eigen::MatrixXi > indices( ptr3, tSize, xSize*RXSize );
outputs[0] = factory.createArray<std::complex<double>>({static_cast<size_t>(tSize),static_cast<size_t>(TXSize*RXSize)});
auto ptrRecon = getOutDataPtr<std::complex<double>>(outputs[0]);
Eigen::Map<Eigen::MatrixXcd> RFDataKK(ptrRecon,tSize,TXSize*RXSize);
// Get num threads
int numThreads = omp_get_max_threads();
int nProc = omp_get_num_procs();
omp_set_num_threads(nProc*2);
#pragma omp parallel for
for (int nR = 0; nR < RXSize; nR++) {
Eigen::MatrixXcd shiftedData = Eigen::MatrixXcd::Zero(tSize*TXSize,xSize);
for (int nT = 0; nT < TXSize; nT++){
extractData(shiftedData(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
Data(Eigen::seq(nT*tSize,(nT+1)*tSize-1),Eigen::all),
indices(Eigen::all,Eigen::seq(nR*xSize, (nR+1)*xSize-1)), xSize, tSize);
}
RFDataKK(Eigen::all,Eigen::seq(nR*TXSize,(nR+1)*TXSize-1) ) = shiftedData.rowwise().sum().reshaped(tSize,TXSize);
}
}
void extractData(Eigen::Ref<Eigen::MatrixXcd> shiftedData, const Eigen::Ref<const Eigen::MatrixXcd>& Data,
const Eigen::Ref<const Eigen::MatrixXi>& indices, const long int& xSize, const long int& tSize) {
for (int i = 0; i < tSize*xSize; i++) {
int nt = i % tSize;
int nx = i / tSize;
int row = indices(nt,nx) % tSize;
int col = indices(nt,nx) / tSize;
shiftedData(row,col) = Data(nt,nx);
}
}
void checkArguments(matlab::mex::ArgumentList outputs, matlab::mex::ArgumentList inputs) {
std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
matlab::data::ArrayFactory factory;
// TODO: Implement remaining checks
}
template <typename T>
const T* getDataPtr(matlab::data::Array arr) {
const matlab::data::TypedArray<T> arr_t = arr;
matlab::data::TypedIterator<const T> it(arr_t.begin());
return it.operator->();
}
template <typename T>
T* getOutDataPtr(matlab::data::Array& arr) {
auto range = matlab::data::getWritableElements<T>(arr);
return range.begin().operator->();
}
};
“`
Where I precompute the values of indices using the following Matlab function:
function linIndices=DataCompressKKIndices(RXangle,s,tSize,xSize,TXSize,RXSize)
shiftedData = zeros(tSize, xSize, TXSize);
RFDatKK = zeros(tSize,TXSize,RXSize);
RFDataKK = zeros(tSize,xSize*RXSize);
slope=s*sin(RXangle(:))/2;
nShift = round(slope.*((0:xSize-1) – (xSize-1)*(1-sign(RXangle(:)))/2));
indices = mod((0:tSize-1).’ + reshape(nShift.’,[],1).’,tSize);
linIndices = indices + repmat((0:xSize-1)*tSize,1,RXSize);
end
When I perform a speed test in Matlab after compiling with the following parameters (with mingw GCC version 8.3), I find that I have not gained a meaningful speedup for the array sizes I am working with. OpenMP parallelization should at least give me an order of magnitude speed up (I have 24 threads on my machine). There are two questions that yields:
1. Why is a reindexing approach slower than doing circshift? A reindexing approach in Matlab (not shown here) is almost 2x slower than using nested for loops and circshift.
2. What exactly could circshift be doing under the hood that is so efficient? Is there some funky pointer arithmetic that could accomplish the functionality of circshift?
Compilation and testing code:
%% Compilation
mingwFlags = {‘CXXFLAGS="$CXXFLAGS -march=native -std=c++14 -fno-math-errno -ffast-math -fopenmp -DNDEBUG -w -Wno-error"’,…
‘LDFLAGS="$LDFLAGS -fopenmp"’,’CXXOPTIMFLAGS="-O3"’};
% ipath is the path to the eigen library.
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKK.cpp’); toc
tic; mex(ipath,mingwFlags{1},mingwFlags{2},mingwFlags{3},’CompressKKIndices.cpp’); toc
%% Initialize some random data
…
% tSize = 2688, xSize = 192, TXSize = 15, RXSize = 16 for trial cases
%% Timing Test
nTrials = 100;
T = zeros(nTrials,4);
for i = 1:nTrials
tic; indices = DataCompressKKIndices(p.RXangle,s,double(p.szRFframe+1),double(p.numEl),p.na,p.nRX);
RawDataKK3 = CompressKKIndices(param,cRF(1:param(5),p.ConnMap),s,p.RXangle,int32(indices));
RawDataKK3 = reshape(RawDataKK3,[p.szRFframe+1,p.na,p.nRX]); T(i,1) = toc;
tic; RawDataKK=DataCompressKKTest(Data,p.RXangle,s); T(i,2) = toc;
end
chkT = mean(T,1)
Timing Results:
chkT =
C++ version: 0.4867 0.8299 performance, mex, c++, optimization MATLAB Answers — New Questions
Why do I receive an ‘Invalid input argument of type ‘double’. Input must be a structure or a Java or COM object’ error while loading data in a simbiology fit program?
I deviced a kinetic model using simbiology, simulated it and everything worked fine. Then I loaded a data set from excel containing three columns: ID, Time and concentration. I set ID as group, Time as independent and concentration as depended variables. But when I run the program it says
Invalid input argument of type ‘double’. Input must be a structure or a Java or COM object.
I’ve also tried loading the data directly from the workspace and it gave the same error.
I converted the data to a structure and that is not listed as a valid input type while importing a dataset. I checked online tutorials and everyone seems to be able to load ‘double’ types.
The reaction is just A+B–>null
And dB/dt = -k*[B] and A is constantI deviced a kinetic model using simbiology, simulated it and everything worked fine. Then I loaded a data set from excel containing three columns: ID, Time and concentration. I set ID as group, Time as independent and concentration as depended variables. But when I run the program it says
Invalid input argument of type ‘double’. Input must be a structure or a Java or COM object.
I’ve also tried loading the data directly from the workspace and it gave the same error.
I converted the data to a structure and that is not listed as a valid input type while importing a dataset. I checked online tutorials and everyone seems to be able to load ‘double’ types.
The reaction is just A+B–>null
And dB/dt = -k*[B] and A is constant I deviced a kinetic model using simbiology, simulated it and everything worked fine. Then I loaded a data set from excel containing three columns: ID, Time and concentration. I set ID as group, Time as independent and concentration as depended variables. But when I run the program it says
Invalid input argument of type ‘double’. Input must be a structure or a Java or COM object.
I’ve also tried loading the data directly from the workspace and it gave the same error.
I converted the data to a structure and that is not listed as a valid input type while importing a dataset. I checked online tutorials and everyone seems to be able to load ‘double’ types.
The reaction is just A+B–>null
And dB/dt = -k*[B] and A is constant simbiology, double, structures, data import MATLAB Answers — New Questions