Plot exceeding time limit due to large dataset
I have a dataset in which I have to categorize the rise of UV levels from a satellite.
As this satellite orbits the Earth there are periods where it cannot process the UV levels.
The raw data that I am getting is close to a square wave.
I have to categorize this data into rising where the satellite comes out of the influence of the Earth,
OnDuty where the satellite is able to record data properly and falling where it is going behind the Earth.
The dataset contains a timestamp of frequency 1Hz where one value is recorded every second.
I want to categorize these three classes into for atleast now, red, green and blue.
The dataset contains the said timestamp and the corresponding UV values.
I am currently using a threshold to categorize the data and is working for a smaller dataset, however with a dataset of 80000 entry points my code is not able to run efficiently.
Here is the code. Where states stores the ith values nature. The problem lies within the for loop that runs for the plotting.
function categorize_square_wave()
y_foo = readtable("80000entries.csv", Range=’uv_values’);
y_values = table2array(y_foo);
k = 1; % Sensitivity parameter
differences = diff(y_values);
% comments have been put to remove staticstical computation
mu = mean(differences);
sigma = std(differences);
% Thresholds
T_high = mu + k * sigma;
T_low = mu – k * sigma;
% coded directly to reduce computation
% T_high = 500;
% T_low = -100;
% Categorizing based on threshold values
% Initialize states with zeros (default to dutiful)
states = zeros(1, length(differences));
% Assign states using vectorization
states(differences > T_high) = 1; % Rising
states(differences < T_low) = -1; % Falling
% Collect dutiful values
% dutiful_values = y_values(states == 0);
figure;
hold on;
for i = 1:length(differences)
if states(i) == 1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘g’, ‘LineWidth’, 2); % Green for Rising
elseif states(i) == -1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘r’, ‘LineWidth’, 2); % Red for Falling
else
plot([i, i+1], [y_values(i), y_values(i+1)], ‘b’, ‘LineWidth’, 2); % Blue for Dutiful
% dutiful_values = [dutiful_values, y_values(i)]; % Collect Dutiful values
end
end
% Plot original values in dashed lines
plot(y_values, ‘k–‘, ‘LineWidth’, 1); % Black dashed line for original values
yline(T_high, ‘r–‘, ‘T_{high}’, ‘LabelVerticalAlignment’, ‘bottom’, ‘LabelHorizontalAlignment’, ‘right’); % Dashed red line for T_high
xlabel(‘Sample Number’);
ylabel(‘y(t)’);
title(‘Classification of Changes in Square Wave’);
hold off;
end
categorize_square_wave();
% we have a good estimation of what values are
% is there a way to use flags on each state so that we can just plot
% without checking each value?
% implemetn flagsI have a dataset in which I have to categorize the rise of UV levels from a satellite.
As this satellite orbits the Earth there are periods where it cannot process the UV levels.
The raw data that I am getting is close to a square wave.
I have to categorize this data into rising where the satellite comes out of the influence of the Earth,
OnDuty where the satellite is able to record data properly and falling where it is going behind the Earth.
The dataset contains a timestamp of frequency 1Hz where one value is recorded every second.
I want to categorize these three classes into for atleast now, red, green and blue.
The dataset contains the said timestamp and the corresponding UV values.
I am currently using a threshold to categorize the data and is working for a smaller dataset, however with a dataset of 80000 entry points my code is not able to run efficiently.
Here is the code. Where states stores the ith values nature. The problem lies within the for loop that runs for the plotting.
function categorize_square_wave()
y_foo = readtable("80000entries.csv", Range=’uv_values’);
y_values = table2array(y_foo);
k = 1; % Sensitivity parameter
differences = diff(y_values);
% comments have been put to remove staticstical computation
mu = mean(differences);
sigma = std(differences);
% Thresholds
T_high = mu + k * sigma;
T_low = mu – k * sigma;
% coded directly to reduce computation
% T_high = 500;
% T_low = -100;
% Categorizing based on threshold values
% Initialize states with zeros (default to dutiful)
states = zeros(1, length(differences));
% Assign states using vectorization
states(differences > T_high) = 1; % Rising
states(differences < T_low) = -1; % Falling
% Collect dutiful values
% dutiful_values = y_values(states == 0);
figure;
hold on;
for i = 1:length(differences)
if states(i) == 1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘g’, ‘LineWidth’, 2); % Green for Rising
elseif states(i) == -1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘r’, ‘LineWidth’, 2); % Red for Falling
else
plot([i, i+1], [y_values(i), y_values(i+1)], ‘b’, ‘LineWidth’, 2); % Blue for Dutiful
% dutiful_values = [dutiful_values, y_values(i)]; % Collect Dutiful values
end
end
% Plot original values in dashed lines
plot(y_values, ‘k–‘, ‘LineWidth’, 1); % Black dashed line for original values
yline(T_high, ‘r–‘, ‘T_{high}’, ‘LabelVerticalAlignment’, ‘bottom’, ‘LabelHorizontalAlignment’, ‘right’); % Dashed red line for T_high
xlabel(‘Sample Number’);
ylabel(‘y(t)’);
title(‘Classification of Changes in Square Wave’);
hold off;
end
categorize_square_wave();
% we have a good estimation of what values are
% is there a way to use flags on each state so that we can just plot
% without checking each value?
% implemetn flags I have a dataset in which I have to categorize the rise of UV levels from a satellite.
As this satellite orbits the Earth there are periods where it cannot process the UV levels.
The raw data that I am getting is close to a square wave.
I have to categorize this data into rising where the satellite comes out of the influence of the Earth,
OnDuty where the satellite is able to record data properly and falling where it is going behind the Earth.
The dataset contains a timestamp of frequency 1Hz where one value is recorded every second.
I want to categorize these three classes into for atleast now, red, green and blue.
The dataset contains the said timestamp and the corresponding UV values.
I am currently using a threshold to categorize the data and is working for a smaller dataset, however with a dataset of 80000 entry points my code is not able to run efficiently.
Here is the code. Where states stores the ith values nature. The problem lies within the for loop that runs for the plotting.
function categorize_square_wave()
y_foo = readtable("80000entries.csv", Range=’uv_values’);
y_values = table2array(y_foo);
k = 1; % Sensitivity parameter
differences = diff(y_values);
% comments have been put to remove staticstical computation
mu = mean(differences);
sigma = std(differences);
% Thresholds
T_high = mu + k * sigma;
T_low = mu – k * sigma;
% coded directly to reduce computation
% T_high = 500;
% T_low = -100;
% Categorizing based on threshold values
% Initialize states with zeros (default to dutiful)
states = zeros(1, length(differences));
% Assign states using vectorization
states(differences > T_high) = 1; % Rising
states(differences < T_low) = -1; % Falling
% Collect dutiful values
% dutiful_values = y_values(states == 0);
figure;
hold on;
for i = 1:length(differences)
if states(i) == 1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘g’, ‘LineWidth’, 2); % Green for Rising
elseif states(i) == -1
plot([i, i+1], [y_values(i), y_values(i+1)], ‘r’, ‘LineWidth’, 2); % Red for Falling
else
plot([i, i+1], [y_values(i), y_values(i+1)], ‘b’, ‘LineWidth’, 2); % Blue for Dutiful
% dutiful_values = [dutiful_values, y_values(i)]; % Collect Dutiful values
end
end
% Plot original values in dashed lines
plot(y_values, ‘k–‘, ‘LineWidth’, 1); % Black dashed line for original values
yline(T_high, ‘r–‘, ‘T_{high}’, ‘LabelVerticalAlignment’, ‘bottom’, ‘LabelHorizontalAlignment’, ‘right’); % Dashed red line for T_high
xlabel(‘Sample Number’);
ylabel(‘y(t)’);
title(‘Classification of Changes in Square Wave’);
hold off;
end
categorize_square_wave();
% we have a good estimation of what values are
% is there a way to use flags on each state so that we can just plot
% without checking each value?
% implemetn flags data cleaning, plotting, optimization MATLAB Answers — New Questions