I’ve got files which contain data multiple from multiple sensors. Each sensor has it’s own timestamp. Not all sensors have the same number of values.
Currently I’m splitting the file into several timetables. Then I’ll merge and synchronize the timetables and finally fill the missing values.
Is there a better way to do this, since the original files contain several hundred thousand lines for 50-60 signals?
txtArray= {‘Sig1_Time’ ‘Sig1_Value’ ‘Signal2_Time’ ‘Sig2_Value’ ‘Sig3_Time’ ‘Sig3_Value’;
‘4/28/2020 6:41:56.555 PM’ ‘92.1822814’ ‘4/28/2020 6:41:56.545 PM’ ‘21.2642456’ ‘4/28/2020 6:40:56.545 PM’ ‘1.26’;
‘4/28/2020 6:42:06.655 PM’ ‘92.2822814’ ‘4/28/2020 6:42:06.645 PM’ ‘22.3538671’ ‘4/28/2020 6:50:06.645 PM’ ‘2.35’;
‘4/28/2020 6:42:07.665 PM’ ‘92.1922814’ ‘4/28/2020 6:42:07.655 PM’ ‘22.2642456’ ” ”;
” ” ‘4/28/2020 6:42:08.665 PM’ ‘23.2822436’ ” ”;
” ” ‘4/28/2020 6:42:20.786 PM’ ‘22.2642456’ ” ”;
};

% find columns with timestamps
TimeCols = contains([txtArray(1,:)],’Time’,’IgnoreCase’,true);

% convert times to numerical values
times = datetime([txtArray(2:end,TimeCols)],’InputFormat’,’M/d/yyyy h:m:ss.SSS a’);

NoVars = sum(~TimeCols);

for j=1:NoVars
% create time column in TimeTable
TC = times(:,j);
% create data column in TimeTable
DC = cellfun(@(s) str2double(s),txtArray(:,2*j));
% merge arrays into Timetable, remove invalied (NaT) times
TT(j).tt = array2timetable(DC(~isnat(TC)),’RowTimes’,TC(~isnat(TC)));
end

% merge timetables
for j=1:NoVars
if ~issorted(TT(j).tt);
TT(j).tt=sortrows(TT(j).tt);
end
if j>1
if j==2
Ttable = TT(j-1).tt;
end
% synchronize removes dublicate times
Ttable = synchronize(Ttable,TT(j).tt); % https://mathworks.com/help/matlab/ref/timetable.synchronize.html
Ttable = fillmissing(Ttable,’previous’); % https://mathworks.com/help/matlab/ref/fillmissing.html
end
end

% fill missing values for the first lines which may still be empty
Ttable = fillmissing(Ttable,’next’);

% rename properties
Ttable.Properties.VariableNames = {‘Var1′,’Var2′,’Var3’};I’ve got files which contain data multiple from multiple sensors. Each sensor has it’s own timestamp. Not all sensors have the same number of values.
Currently I’m splitting the file into several timetables. Then I’ll merge and synchronize the timetables and finally fill the missing values.
Is there a better way to do this, since the original files contain several hundred thousand lines for 50-60 signals?
txtArray= {‘Sig1_Time’ ‘Sig1_Value’ ‘Signal2_Time’ ‘Sig2_Value’ ‘Sig3_Time’ ‘Sig3_Value’;
‘4/28/2020 6:41:56.555 PM’ ‘92.1822814’ ‘4/28/2020 6:41:56.545 PM’ ‘21.2642456’ ‘4/28/2020 6:40:56.545 PM’ ‘1.26’;
‘4/28/2020 6:42:06.655 PM’ ‘92.2822814’ ‘4/28/2020 6:42:06.645 PM’ ‘22.3538671’ ‘4/28/2020 6:50:06.645 PM’ ‘2.35’;
‘4/28/2020 6:42:07.665 PM’ ‘92.1922814’ ‘4/28/2020 6:42:07.655 PM’ ‘22.2642456’ ” ”;
” ” ‘4/28/2020 6:42:08.665 PM’ ‘23.2822436’ ” ”;
” ” ‘4/28/2020 6:42:20.786 PM’ ‘22.2642456’ ” ”;
};

% find columns with timestamps
TimeCols = contains([txtArray(1,:)],’Time’,’IgnoreCase’,true);

% convert times to numerical values
times = datetime([txtArray(2:end,TimeCols)],’InputFormat’,’M/d/yyyy h:m:ss.SSS a’);

NoVars = sum(~TimeCols);

for j=1:NoVars
% create time column in TimeTable
TC = times(:,j);
% create data column in TimeTable
DC = cellfun(@(s) str2double(s),txtArray(:,2*j));
% merge arrays into Timetable, remove invalied (NaT) times
TT(j).tt = array2timetable(DC(~isnat(TC)),’RowTimes’,TC(~isnat(TC)));
end

% merge timetables
for j=1:NoVars
if ~issorted(TT(j).tt);
TT(j).tt=sortrows(TT(j).tt);
end
if j>1
if j==2
Ttable = TT(j-1).tt;
end
% synchronize removes dublicate times
Ttable = synchronize(Ttable,TT(j).tt); % https://mathworks.com/help/matlab/ref/timetable.synchronize.html
Ttable = fillmissing(Ttable,’previous’); % https://mathworks.com/help/matlab/ref/fillmissing.html
end
end

% fill missing values for the first lines which may still be empty
Ttable = fillmissing(Ttable,’next’);

% rename properties
Ttable.Properties.VariableNames = {‘Var1′,’Var2′,’Var3’}; I’ve got files which contain data multiple from multiple sensors. Each sensor has it’s own timestamp. Not all sensors have the same number of values.
Currently I’m splitting the file into several timetables. Then I’ll merge and synchronize the timetables and finally fill the missing values.
Is there a better way to do this, since the original files contain several hundred thousand lines for 50-60 signals?
txtArray= {‘Sig1_Time’ ‘Sig1_Value’ ‘Signal2_Time’ ‘Sig2_Value’ ‘Sig3_Time’ ‘Sig3_Value’;
‘4/28/2020 6:41:56.555 PM’ ‘92.1822814’ ‘4/28/2020 6:41:56.545 PM’ ‘21.2642456’ ‘4/28/2020 6:40:56.545 PM’ ‘1.26’;
‘4/28/2020 6:42:06.655 PM’ ‘92.2822814’ ‘4/28/2020 6:42:06.645 PM’ ‘22.3538671’ ‘4/28/2020 6:50:06.645 PM’ ‘2.35’;
‘4/28/2020 6:42:07.665 PM’ ‘92.1922814’ ‘4/28/2020 6:42:07.655 PM’ ‘22.2642456’ ” ”;
” ” ‘4/28/2020 6:42:08.665 PM’ ‘23.2822436’ ” ”;
” ” ‘4/28/2020 6:42:20.786 PM’ ‘22.2642456’ ” ”;
};

% find columns with timestamps
TimeCols = contains([txtArray(1,:)],’Time’,’IgnoreCase’,true);

% convert times to numerical values
times = datetime([txtArray(2:end,TimeCols)],’InputFormat’,’M/d/yyyy h:m:ss.SSS a’);

NoVars = sum(~TimeCols);

for j=1:NoVars
% create time column in TimeTable
TC = times(:,j);
% create data column in TimeTable
DC = cellfun(@(s) str2double(s),txtArray(:,2*j));
% merge arrays into Timetable, remove invalied (NaT) times
TT(j).tt = array2timetable(DC(~isnat(TC)),’RowTimes’,TC(~isnat(TC)));
end

% merge timetables
for j=1:NoVars
if ~issorted(TT(j).tt);
TT(j).tt=sortrows(TT(j).tt);
end
if j>1
if j==2
Ttable = TT(j-1).tt;
end
% synchronize removes dublicate times
Ttable = synchronize(Ttable,TT(j).tt); % https://mathworks.com/help/matlab/ref/timetable.synchronize.html
Ttable = fillmissing(Ttable,’previous’); % https://mathworks.com/help/matlab/ref/fillmissing.html
end
end

% fill missing values for the first lines which may still be empty
Ttable = fillmissing(Ttable,’next’);

% rename properties
Ttable.Properties.VariableNames = {‘Var1′,’Var2′,’Var3’}; timetable, data import, array2timetable, multiple time columns MATLAB Answers — New Questions

​