speeding up the data analysis
I am working on a project where I have to analyse high frequency wind data (50 Hz) coming from a wind turbine data measurement system. this amount of data must be converted from .dat binary files to .mat files which I can use in matlab. The data must then be filtered and then averaged over 10 minutes to be compared to the data from another measurement system. Doing all this requires the analysis of thousands of data and it’s very time consuming (right now it is about 105 seconds just for the data of 2 days). How can I speed the process up?
for ii = 3:length(filedir)
filename = filedir(ii).name;
newdata = ReadFamosDataIntoTimeTable(filename);
% filter
IsConsidered = newdata.avbladeangleGRe<40 … % normal operation
& newdata.RAWS9>0.5 … % good availability
& ~isnan(newdata.iv10mswindspeed2GRe) & newdata.av100msabswinddirectionGRe>180 & newdata.av100msabswinddirectionGRe<250;
TurbineData(ii-2).date = filename;
TurbineData(ii-2).iv10mswindspeed2GRe = mean(newdata.iv10mswindspeed2GRe(IsConsidered));
TurbineData(ii-2).ivactivepowerGRe = mean(newdata.ivactivepowerGRe(IsConsidered));
TurbineData(ii-2).CalculatedAirdensity_GRe = mean(newdata.CalculatedAirdensity_GRe(IsConsidered));
TurbineData(ii-2).HWShub1 = mean(newdata.HWShub1(IsConsidered));
TurbineData(ii-2).av100msabswinddirectionGRe = mean(newdata.av100msabswinddirectionGRe(IsConsidered));
end
The function ReadFamosDataIntoTimeTable is to convert the .dat binary data into .mat dataI am working on a project where I have to analyse high frequency wind data (50 Hz) coming from a wind turbine data measurement system. this amount of data must be converted from .dat binary files to .mat files which I can use in matlab. The data must then be filtered and then averaged over 10 minutes to be compared to the data from another measurement system. Doing all this requires the analysis of thousands of data and it’s very time consuming (right now it is about 105 seconds just for the data of 2 days). How can I speed the process up?
for ii = 3:length(filedir)
filename = filedir(ii).name;
newdata = ReadFamosDataIntoTimeTable(filename);
% filter
IsConsidered = newdata.avbladeangleGRe<40 … % normal operation
& newdata.RAWS9>0.5 … % good availability
& ~isnan(newdata.iv10mswindspeed2GRe) & newdata.av100msabswinddirectionGRe>180 & newdata.av100msabswinddirectionGRe<250;
TurbineData(ii-2).date = filename;
TurbineData(ii-2).iv10mswindspeed2GRe = mean(newdata.iv10mswindspeed2GRe(IsConsidered));
TurbineData(ii-2).ivactivepowerGRe = mean(newdata.ivactivepowerGRe(IsConsidered));
TurbineData(ii-2).CalculatedAirdensity_GRe = mean(newdata.CalculatedAirdensity_GRe(IsConsidered));
TurbineData(ii-2).HWShub1 = mean(newdata.HWShub1(IsConsidered));
TurbineData(ii-2).av100msabswinddirectionGRe = mean(newdata.av100msabswinddirectionGRe(IsConsidered));
end
The function ReadFamosDataIntoTimeTable is to convert the .dat binary data into .mat data I am working on a project where I have to analyse high frequency wind data (50 Hz) coming from a wind turbine data measurement system. this amount of data must be converted from .dat binary files to .mat files which I can use in matlab. The data must then be filtered and then averaged over 10 minutes to be compared to the data from another measurement system. Doing all this requires the analysis of thousands of data and it’s very time consuming (right now it is about 105 seconds just for the data of 2 days). How can I speed the process up?
for ii = 3:length(filedir)
filename = filedir(ii).name;
newdata = ReadFamosDataIntoTimeTable(filename);
% filter
IsConsidered = newdata.avbladeangleGRe<40 … % normal operation
& newdata.RAWS9>0.5 … % good availability
& ~isnan(newdata.iv10mswindspeed2GRe) & newdata.av100msabswinddirectionGRe>180 & newdata.av100msabswinddirectionGRe<250;
TurbineData(ii-2).date = filename;
TurbineData(ii-2).iv10mswindspeed2GRe = mean(newdata.iv10mswindspeed2GRe(IsConsidered));
TurbineData(ii-2).ivactivepowerGRe = mean(newdata.ivactivepowerGRe(IsConsidered));
TurbineData(ii-2).CalculatedAirdensity_GRe = mean(newdata.CalculatedAirdensity_GRe(IsConsidered));
TurbineData(ii-2).HWShub1 = mean(newdata.HWShub1(IsConsidered));
TurbineData(ii-2).av100msabswinddirectionGRe = mean(newdata.av100msabswinddirectionGRe(IsConsidered));
end
The function ReadFamosDataIntoTimeTable is to convert the .dat binary data into .mat data optimization, for loop, speed, data conversion MATLAB Answers — New Questions