Observations being read as number of columns instead of rows
Before splitting my data into training, validation, and testing, the code worked fine. After the split, trainnet begins reading the observations as the number of columns instead of by the number of rows. I checked the size of the array files and they are the same size the arrays were before I added a data split. I do not understand why trainnet starts reading the arrays differently. Any help would be appreciated.
Error message:
Error using trainnet
Error forming validation data mini-batch.
Error in MainCode6 (line 40)
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts);
Caused by:
Number of observations in predictors (6) and targets (1) must match. Check that the data and network are
consistent.
% Script Number One
all = readtable(excelFile,"Sheet","Nsplit6",’VariableNamingRule’,’preserve’);
train_ratio = 0.8;
val_ratio = 0.1;
test_ratio = 0.1;
num_samples = size(all, 1);
indices = randperm(num_samples);
num_train = floor(train_ratio * num_samples);
num_val = floor(val_ratio * num_samples);
num_test = num_samples – num_train – num_val;
train_indices = indices(1:num_train);
val_indices = indices(num_train+1:num_train+num_val);
test_indices = indices(num_train+num_val+1:end);
train_data = all(train_indices, :);
val_data = all(val_indices, :);
test_data = all(test_indices, :);
X_train = train_data(:, 1:end-1);
Y_train = train_data(:, end);
X_val = val_data(:, 1:end-1);
Y_val = val_data(:, end);
X_test = test_data(:, 1:end-1);
Y_test = test_data(:, end);
stan_x_train = table2array(X_train);
stan_y_train = table2array(Y_train);
stan_x_val = table2array(X_val);
stan_y_val = table2array(Y_val);
stan_x_test = table2array(X_test);
stan_y_test = table2array(Y_test);
%Script number two
inputSize = 6;
hiddenLayerSize1 = 40;
hiddenLayerSize2 = 20;
hiddenLayerSize3 = 10;
outputSize = 1;
layers = [
featureInputLayer(inputSize)
fullyConnectedLayer(hiddenLayerSize1, ‘Name’, ‘fc1’)
reluLayer(‘Name’, ‘relu1’)
fullyConnectedLayer(hiddenLayerSize2, ‘Name’, ‘fc2’)
reluLayer(‘Name’, ‘relu2’)
fullyConnectedLayer(hiddenLayerSize3, ‘Name’, ‘fc3’)
reluLayer(‘Name’, ‘relu3’)
fullyConnectedLayer(outputSize, ‘Name’, ‘output’)
];
net = dlnetwork(layers);
opts = trainingOptions(‘adam’, …
‘InitialLearnRate’, 0.01, …
‘MaxEpochs’, 200, …
‘MiniBatchSize’, 20, …
‘ValidationData’, {stan_x_val’, stan_y_val’}, …
‘ValidationFrequency’, 20, …
‘Verbose’, true …
);
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts);Before splitting my data into training, validation, and testing, the code worked fine. After the split, trainnet begins reading the observations as the number of columns instead of by the number of rows. I checked the size of the array files and they are the same size the arrays were before I added a data split. I do not understand why trainnet starts reading the arrays differently. Any help would be appreciated.
Error message:
Error using trainnet
Error forming validation data mini-batch.
Error in MainCode6 (line 40)
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts);
Caused by:
Number of observations in predictors (6) and targets (1) must match. Check that the data and network are
consistent.
% Script Number One
all = readtable(excelFile,"Sheet","Nsplit6",’VariableNamingRule’,’preserve’);
train_ratio = 0.8;
val_ratio = 0.1;
test_ratio = 0.1;
num_samples = size(all, 1);
indices = randperm(num_samples);
num_train = floor(train_ratio * num_samples);
num_val = floor(val_ratio * num_samples);
num_test = num_samples – num_train – num_val;
train_indices = indices(1:num_train);
val_indices = indices(num_train+1:num_train+num_val);
test_indices = indices(num_train+num_val+1:end);
train_data = all(train_indices, :);
val_data = all(val_indices, :);
test_data = all(test_indices, :);
X_train = train_data(:, 1:end-1);
Y_train = train_data(:, end);
X_val = val_data(:, 1:end-1);
Y_val = val_data(:, end);
X_test = test_data(:, 1:end-1);
Y_test = test_data(:, end);
stan_x_train = table2array(X_train);
stan_y_train = table2array(Y_train);
stan_x_val = table2array(X_val);
stan_y_val = table2array(Y_val);
stan_x_test = table2array(X_test);
stan_y_test = table2array(Y_test);
%Script number two
inputSize = 6;
hiddenLayerSize1 = 40;
hiddenLayerSize2 = 20;
hiddenLayerSize3 = 10;
outputSize = 1;
layers = [
featureInputLayer(inputSize)
fullyConnectedLayer(hiddenLayerSize1, ‘Name’, ‘fc1’)
reluLayer(‘Name’, ‘relu1’)
fullyConnectedLayer(hiddenLayerSize2, ‘Name’, ‘fc2’)
reluLayer(‘Name’, ‘relu2’)
fullyConnectedLayer(hiddenLayerSize3, ‘Name’, ‘fc3’)
reluLayer(‘Name’, ‘relu3’)
fullyConnectedLayer(outputSize, ‘Name’, ‘output’)
];
net = dlnetwork(layers);
opts = trainingOptions(‘adam’, …
‘InitialLearnRate’, 0.01, …
‘MaxEpochs’, 200, …
‘MiniBatchSize’, 20, …
‘ValidationData’, {stan_x_val’, stan_y_val’}, …
‘ValidationFrequency’, 20, …
‘Verbose’, true …
);
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts); Before splitting my data into training, validation, and testing, the code worked fine. After the split, trainnet begins reading the observations as the number of columns instead of by the number of rows. I checked the size of the array files and they are the same size the arrays were before I added a data split. I do not understand why trainnet starts reading the arrays differently. Any help would be appreciated.
Error message:
Error using trainnet
Error forming validation data mini-batch.
Error in MainCode6 (line 40)
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts);
Caused by:
Number of observations in predictors (6) and targets (1) must match. Check that the data and network are
consistent.
% Script Number One
all = readtable(excelFile,"Sheet","Nsplit6",’VariableNamingRule’,’preserve’);
train_ratio = 0.8;
val_ratio = 0.1;
test_ratio = 0.1;
num_samples = size(all, 1);
indices = randperm(num_samples);
num_train = floor(train_ratio * num_samples);
num_val = floor(val_ratio * num_samples);
num_test = num_samples – num_train – num_val;
train_indices = indices(1:num_train);
val_indices = indices(num_train+1:num_train+num_val);
test_indices = indices(num_train+num_val+1:end);
train_data = all(train_indices, :);
val_data = all(val_indices, :);
test_data = all(test_indices, :);
X_train = train_data(:, 1:end-1);
Y_train = train_data(:, end);
X_val = val_data(:, 1:end-1);
Y_val = val_data(:, end);
X_test = test_data(:, 1:end-1);
Y_test = test_data(:, end);
stan_x_train = table2array(X_train);
stan_y_train = table2array(Y_train);
stan_x_val = table2array(X_val);
stan_y_val = table2array(Y_val);
stan_x_test = table2array(X_test);
stan_y_test = table2array(Y_test);
%Script number two
inputSize = 6;
hiddenLayerSize1 = 40;
hiddenLayerSize2 = 20;
hiddenLayerSize3 = 10;
outputSize = 1;
layers = [
featureInputLayer(inputSize)
fullyConnectedLayer(hiddenLayerSize1, ‘Name’, ‘fc1’)
reluLayer(‘Name’, ‘relu1’)
fullyConnectedLayer(hiddenLayerSize2, ‘Name’, ‘fc2’)
reluLayer(‘Name’, ‘relu2’)
fullyConnectedLayer(hiddenLayerSize3, ‘Name’, ‘fc3’)
reluLayer(‘Name’, ‘relu3’)
fullyConnectedLayer(outputSize, ‘Name’, ‘output’)
];
net = dlnetwork(layers);
opts = trainingOptions(‘adam’, …
‘InitialLearnRate’, 0.01, …
‘MaxEpochs’, 200, …
‘MiniBatchSize’, 20, …
‘ValidationData’, {stan_x_val’, stan_y_val’}, …
‘ValidationFrequency’, 20, …
‘Verbose’, true …
);
netTrained = trainnet(stan_x_train, stan_y_train, net, "mse", opts); dlnetwork, observation error, ann MATLAB Answers — New Questions