Importing files in bulk chnages table columns to NaNs
I have the following code in which I want to replace the column 37 in a table. I want to swap the column of categorical values for a column of numerical values.
% Open file selection dialog and allow multiple selections
[file_list, path_n] = uigetfile(‘*.csv’, ‘Select CSV Files’, ‘Multiselect’, ‘on’);
% Check if any files were selected
if ischar(file_list)
% Convert single file string to cell array
file_list = {file_list};
end
% Initialize a cell array to store the modified tables
modifiedTables = cell(length(file_list), 1);
% Loop through each selected file
for k = 1:length(file_list)
% Construct the full file path
filePath = fullfile(path_n, file_list{k});
% Import the CSV file as a table
T = readtable(filePath);
% Check if the table has at least 37 columns
if width(T) >= 37
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Update only the 37th column with the new double column
T.(T.Properties.VariableNames{37}) = newOrderNumberColumn;
end
% Store the modified table in the cell array
modifiedTables{k} = T;
end
% Display the number of tables processed
disp([‘Number of tables processed: ‘, num2str(length(modifiedTables))]);
Unfortunately, the other columns (for example column 35, 36,38 and 39) also become affected when importing the csv files in bulk and hence my new modified table now contains NaNs even though I would like to only modify the column 37 (see "modifiedtables").
When I import via the user interface (see "imported_by_hand") or when I use the code below(see "T") that doesnt seem to happen.
T_copy = T;
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T_copy{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Replace the 37th column with the new double column by creating a new table
% Get the table variable names
varNames = T_copy.Properties.VariableNames;
How can I keep this from happening when I import the data in bulk?
Thanks!I have the following code in which I want to replace the column 37 in a table. I want to swap the column of categorical values for a column of numerical values.
% Open file selection dialog and allow multiple selections
[file_list, path_n] = uigetfile(‘*.csv’, ‘Select CSV Files’, ‘Multiselect’, ‘on’);
% Check if any files were selected
if ischar(file_list)
% Convert single file string to cell array
file_list = {file_list};
end
% Initialize a cell array to store the modified tables
modifiedTables = cell(length(file_list), 1);
% Loop through each selected file
for k = 1:length(file_list)
% Construct the full file path
filePath = fullfile(path_n, file_list{k});
% Import the CSV file as a table
T = readtable(filePath);
% Check if the table has at least 37 columns
if width(T) >= 37
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Update only the 37th column with the new double column
T.(T.Properties.VariableNames{37}) = newOrderNumberColumn;
end
% Store the modified table in the cell array
modifiedTables{k} = T;
end
% Display the number of tables processed
disp([‘Number of tables processed: ‘, num2str(length(modifiedTables))]);
Unfortunately, the other columns (for example column 35, 36,38 and 39) also become affected when importing the csv files in bulk and hence my new modified table now contains NaNs even though I would like to only modify the column 37 (see "modifiedtables").
When I import via the user interface (see "imported_by_hand") or when I use the code below(see "T") that doesnt seem to happen.
T_copy = T;
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T_copy{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Replace the 37th column with the new double column by creating a new table
% Get the table variable names
varNames = T_copy.Properties.VariableNames;
How can I keep this from happening when I import the data in bulk?
Thanks! I have the following code in which I want to replace the column 37 in a table. I want to swap the column of categorical values for a column of numerical values.
% Open file selection dialog and allow multiple selections
[file_list, path_n] = uigetfile(‘*.csv’, ‘Select CSV Files’, ‘Multiselect’, ‘on’);
% Check if any files were selected
if ischar(file_list)
% Convert single file string to cell array
file_list = {file_list};
end
% Initialize a cell array to store the modified tables
modifiedTables = cell(length(file_list), 1);
% Loop through each selected file
for k = 1:length(file_list)
% Construct the full file path
filePath = fullfile(path_n, file_list{k});
% Import the CSV file as a table
T = readtable(filePath);
% Check if the table has at least 37 columns
if width(T) >= 37
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Update only the 37th column with the new double column
T.(T.Properties.VariableNames{37}) = newOrderNumberColumn;
end
% Store the modified table in the cell array
modifiedTables{k} = T;
end
% Display the number of tables processed
disp([‘Number of tables processed: ‘, num2str(length(modifiedTables))]);
Unfortunately, the other columns (for example column 35, 36,38 and 39) also become affected when importing the csv files in bulk and hence my new modified table now contains NaNs even though I would like to only modify the column 37 (see "modifiedtables").
When I import via the user interface (see "imported_by_hand") or when I use the code below(see "T") that doesnt seem to happen.
T_copy = T;
% Extract the ‘OrderNumber’ column (column 37) and convert it to a string array
orderNumberColumn = string(T_copy{:, 37});
% Initialize the new column as a double array with the same size as the original column
newOrderNumberColumn = zeros(size(orderNumberColumn));
% Convert the string values to doubles
for i = 1:length(orderNumberColumn)
numValue = str2double(orderNumberColumn(i));
if isnan(numValue)
% If conversion fails (non-numeric value), set the value to 0
newOrderNumberColumn(i) = 0;
else
% Otherwise, use the numeric value
newOrderNumberColumn(i) = numValue;
end
end
% Replace the 37th column with the new double column by creating a new table
% Get the table variable names
varNames = T_copy.Properties.VariableNames;
How can I keep this from happening when I import the data in bulk?
Thanks! categorical variable, table, convert, import MATLAB Answers — New Questions