The indices for validation and test sets are not being assigned correctly by ‘divideind’
I am new in matlab and I am trying to use ‘divideind’ to divide my data in order, but unfortunately it gives me this:
trainInd: [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 … ] (1×1418 double)
valInd: [1×0 double]
testInd: [1×0 double]
I would appreciate it if you tell me where is the problem?
this is my code:
X = data.inputs;
% Define the dependent variable
y = data.output;
Q = length(y);
% Define the split points
train_end = 1418;
val_end = 1773;
test_end = Q;
% Assign the indices
train_indices = 1:train_end;
val_indices = (train_end + 1):val_end;
test_indices = (val_end + 1):test_end;
% Use divideind to ensure proper division
[train_indices,val_indices,test_indices] = divideind(Q,train_indices,val_indices,test_indices);
% Divide the data into training and testing sets
x_train = log(X(train_indices, :));
x_valid = log(X(val_indices, :));
x_test = log(X(test_indices, :));
y_train = log(y(train_indices));
y_valid = log(y(val_indices));
y_test = log(y(test_indices));
% Transpose the data to fit the input format of the neural network
inputs_train = x_train’;
outputs_train = y_train’;
inputs_validation = x_valid’ ;
outputs_valid = y_valid’;
inputs_test = x_test’;
outputs_test = y_test’;
% Initialize arrays to store RMSE values
train_rmse = zeros(1, 10);
Valid_rmse = zeros(1, 10);
test_rmse = zeros(1, 10);
for i = 1:10
hiddenLayerSize = [i, i];
net = fitnet(hiddenLayerSize, ‘trainlm’);
net.trainParam.epochs = 200;
net.layers{1}.transferFcn = ‘poslin’;
net.layers{2}.transferFcn = ‘logsig’;
net.divideFcn = ‘divideind’;
net.divideMode = ‘sample’;
net.divideParam.trainInd = train_indices;
net.divideParam.valInd = val_indices;
net.divideParam.testInd = test_indices;
[net,tr] = train(net, inputs_train, outputs_train);
% Predict on training data
train_predictions = exp(net(inputs_train(:,tr.trainInd)));
yTrainTrue = exp(outputs_train(:,tr.trainInd));
train_rmse(i) = sqrt(mean((train_predictions – yTrainTrue).^2));
% Predict on Validation data
Valid_predictions = exp(net(inputs_validation(:,tr.valInd)));
yValTrue = exp(outputs_valid(:,tr.valInd));
Valid_rmse(i) = sqrt(mean((Valid_predictions – yValTrue).^2));
% Predict on Validation data
test_predictions = exp(net(inputs_test(:,tr.testInd)));
yTestTrue = exp(outputs_test(:,tr.testInd));
test_rmse(i) = sqrt(mean((test_predictions – yTestTrue).^2));
endI am new in matlab and I am trying to use ‘divideind’ to divide my data in order, but unfortunately it gives me this:
trainInd: [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 … ] (1×1418 double)
valInd: [1×0 double]
testInd: [1×0 double]
I would appreciate it if you tell me where is the problem?
this is my code:
X = data.inputs;
% Define the dependent variable
y = data.output;
Q = length(y);
% Define the split points
train_end = 1418;
val_end = 1773;
test_end = Q;
% Assign the indices
train_indices = 1:train_end;
val_indices = (train_end + 1):val_end;
test_indices = (val_end + 1):test_end;
% Use divideind to ensure proper division
[train_indices,val_indices,test_indices] = divideind(Q,train_indices,val_indices,test_indices);
% Divide the data into training and testing sets
x_train = log(X(train_indices, :));
x_valid = log(X(val_indices, :));
x_test = log(X(test_indices, :));
y_train = log(y(train_indices));
y_valid = log(y(val_indices));
y_test = log(y(test_indices));
% Transpose the data to fit the input format of the neural network
inputs_train = x_train’;
outputs_train = y_train’;
inputs_validation = x_valid’ ;
outputs_valid = y_valid’;
inputs_test = x_test’;
outputs_test = y_test’;
% Initialize arrays to store RMSE values
train_rmse = zeros(1, 10);
Valid_rmse = zeros(1, 10);
test_rmse = zeros(1, 10);
for i = 1:10
hiddenLayerSize = [i, i];
net = fitnet(hiddenLayerSize, ‘trainlm’);
net.trainParam.epochs = 200;
net.layers{1}.transferFcn = ‘poslin’;
net.layers{2}.transferFcn = ‘logsig’;
net.divideFcn = ‘divideind’;
net.divideMode = ‘sample’;
net.divideParam.trainInd = train_indices;
net.divideParam.valInd = val_indices;
net.divideParam.testInd = test_indices;
[net,tr] = train(net, inputs_train, outputs_train);
% Predict on training data
train_predictions = exp(net(inputs_train(:,tr.trainInd)));
yTrainTrue = exp(outputs_train(:,tr.trainInd));
train_rmse(i) = sqrt(mean((train_predictions – yTrainTrue).^2));
% Predict on Validation data
Valid_predictions = exp(net(inputs_validation(:,tr.valInd)));
yValTrue = exp(outputs_valid(:,tr.valInd));
Valid_rmse(i) = sqrt(mean((Valid_predictions – yValTrue).^2));
% Predict on Validation data
test_predictions = exp(net(inputs_test(:,tr.testInd)));
yTestTrue = exp(outputs_test(:,tr.testInd));
test_rmse(i) = sqrt(mean((test_predictions – yTestTrue).^2));
end I am new in matlab and I am trying to use ‘divideind’ to divide my data in order, but unfortunately it gives me this:
trainInd: [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 … ] (1×1418 double)
valInd: [1×0 double]
testInd: [1×0 double]
I would appreciate it if you tell me where is the problem?
this is my code:
X = data.inputs;
% Define the dependent variable
y = data.output;
Q = length(y);
% Define the split points
train_end = 1418;
val_end = 1773;
test_end = Q;
% Assign the indices
train_indices = 1:train_end;
val_indices = (train_end + 1):val_end;
test_indices = (val_end + 1):test_end;
% Use divideind to ensure proper division
[train_indices,val_indices,test_indices] = divideind(Q,train_indices,val_indices,test_indices);
% Divide the data into training and testing sets
x_train = log(X(train_indices, :));
x_valid = log(X(val_indices, :));
x_test = log(X(test_indices, :));
y_train = log(y(train_indices));
y_valid = log(y(val_indices));
y_test = log(y(test_indices));
% Transpose the data to fit the input format of the neural network
inputs_train = x_train’;
outputs_train = y_train’;
inputs_validation = x_valid’ ;
outputs_valid = y_valid’;
inputs_test = x_test’;
outputs_test = y_test’;
% Initialize arrays to store RMSE values
train_rmse = zeros(1, 10);
Valid_rmse = zeros(1, 10);
test_rmse = zeros(1, 10);
for i = 1:10
hiddenLayerSize = [i, i];
net = fitnet(hiddenLayerSize, ‘trainlm’);
net.trainParam.epochs = 200;
net.layers{1}.transferFcn = ‘poslin’;
net.layers{2}.transferFcn = ‘logsig’;
net.divideFcn = ‘divideind’;
net.divideMode = ‘sample’;
net.divideParam.trainInd = train_indices;
net.divideParam.valInd = val_indices;
net.divideParam.testInd = test_indices;
[net,tr] = train(net, inputs_train, outputs_train);
% Predict on training data
train_predictions = exp(net(inputs_train(:,tr.trainInd)));
yTrainTrue = exp(outputs_train(:,tr.trainInd));
train_rmse(i) = sqrt(mean((train_predictions – yTrainTrue).^2));
% Predict on Validation data
Valid_predictions = exp(net(inputs_validation(:,tr.valInd)));
yValTrue = exp(outputs_valid(:,tr.valInd));
Valid_rmse(i) = sqrt(mean((Valid_predictions – yValTrue).^2));
% Predict on Validation data
test_predictions = exp(net(inputs_test(:,tr.testInd)));
yTestTrue = exp(outputs_test(:,tr.testInd));
test_rmse(i) = sqrt(mean((test_predictions – yTestTrue).^2));
end divideind, ann, split data MATLAB Answers — New Questions
