Multilayer Perceptron with More Than One Output and Data Interpretation?
I haven’t used Matlab’s ML or deep learning toolbox since maybe 2022. In the last year, it seems like MathWorks intentionally made the ML portion of Matlab unusable. I’m hoping someone here can help me out because rolling back my matlab two years will make a lot of my recent work unstable. I’ve done my best to document my process here. Error messages are given in italics for ease of reading.
I have some data that is represented by four doubles that are normalzied to between 0 and 1. I would like to use these four doubles to predict two doubles that are also between 0 and 1. I would like to make a multilayer perceptron that will take in the four values and spit out the two values. However, createMLPNetwork doesn’t appear to support multiple outputs – at least, the documentation for it doesn’t explain how to do so. So I have tried to make a MLP from scratch using the following code:
%% Prep NN architecture
layers = [
inputLayer([4, 1], "CU", name="in")
flattenLayer
fullyConnectedLayer(20, name="fc1")
reluLayer
fullyConnectedLayerB(16, name="fc2")
reluLayer
fullyConnectedLayer(12, name="fc3")
reluLayer
fullyConnectedLayer(numel(data{1,1}), name="output")
softmaxLayer
];
net = dlnetwork(layers);
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Attempting to run this gives me the following results:
Error using trainnet (line 46)
Error forming mini-batch for network input "in". Data interpreted with format "CU". To specify a different format, use the InputDataFormats option.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Caused by:
The data format must have a batch dimension.
I tried to get around using the generic inputLayer function by using the more specific functions like featureInputLayer but replacing the inputLayer with a featureInputLayer throws the following:
Error using trainnet (line 46)
Number of observations in predictors (4) and targets (2) must match. Check that the data and network are consistent.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
So that won’t work because I only have two output data points. The same happened when I tried imageInputLayer. Then I tried replacing "CU" with other values – the following error is for "BU" but the other errors are the same – but got the following:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 104)
net = dlnetwork(layers);
Caused by:
Layer ‘fc1’: Invalid input data for fully connected layer. The input data must have exactly one channel dimension.
So then I tried flattening the output and got the following error:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 105)
net = dlnetwork(layers);
Caused by:
Layer ‘flatten’: Invalid input data. Layer expects data with a channel dimension, but received input data with format "BU".
I’m really not sure what to do here. I have no idea why MathWorks would make this so much more difficult to use and give so much less, and more opaque, documentation. If anyone has any ideas on how to make this work, I’d be happy to hear them. In the meantime, I’m going to take another crack at the createMLPNetwork function and hope that when MathWorks says "a black-box continuous-time or discrete-time neural state-space model with identifiable (estimable) network weights and bias," by "states" they mean "outputs."I haven’t used Matlab’s ML or deep learning toolbox since maybe 2022. In the last year, it seems like MathWorks intentionally made the ML portion of Matlab unusable. I’m hoping someone here can help me out because rolling back my matlab two years will make a lot of my recent work unstable. I’ve done my best to document my process here. Error messages are given in italics for ease of reading.
I have some data that is represented by four doubles that are normalzied to between 0 and 1. I would like to use these four doubles to predict two doubles that are also between 0 and 1. I would like to make a multilayer perceptron that will take in the four values and spit out the two values. However, createMLPNetwork doesn’t appear to support multiple outputs – at least, the documentation for it doesn’t explain how to do so. So I have tried to make a MLP from scratch using the following code:
%% Prep NN architecture
layers = [
inputLayer([4, 1], "CU", name="in")
flattenLayer
fullyConnectedLayer(20, name="fc1")
reluLayer
fullyConnectedLayerB(16, name="fc2")
reluLayer
fullyConnectedLayer(12, name="fc3")
reluLayer
fullyConnectedLayer(numel(data{1,1}), name="output")
softmaxLayer
];
net = dlnetwork(layers);
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Attempting to run this gives me the following results:
Error using trainnet (line 46)
Error forming mini-batch for network input "in". Data interpreted with format "CU". To specify a different format, use the InputDataFormats option.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Caused by:
The data format must have a batch dimension.
I tried to get around using the generic inputLayer function by using the more specific functions like featureInputLayer but replacing the inputLayer with a featureInputLayer throws the following:
Error using trainnet (line 46)
Number of observations in predictors (4) and targets (2) must match. Check that the data and network are consistent.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
So that won’t work because I only have two output data points. The same happened when I tried imageInputLayer. Then I tried replacing "CU" with other values – the following error is for "BU" but the other errors are the same – but got the following:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 104)
net = dlnetwork(layers);
Caused by:
Layer ‘fc1’: Invalid input data for fully connected layer. The input data must have exactly one channel dimension.
So then I tried flattening the output and got the following error:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 105)
net = dlnetwork(layers);
Caused by:
Layer ‘flatten’: Invalid input data. Layer expects data with a channel dimension, but received input data with format "BU".
I’m really not sure what to do here. I have no idea why MathWorks would make this so much more difficult to use and give so much less, and more opaque, documentation. If anyone has any ideas on how to make this work, I’d be happy to hear them. In the meantime, I’m going to take another crack at the createMLPNetwork function and hope that when MathWorks says "a black-box continuous-time or discrete-time neural state-space model with identifiable (estimable) network weights and bias," by "states" they mean "outputs." I haven’t used Matlab’s ML or deep learning toolbox since maybe 2022. In the last year, it seems like MathWorks intentionally made the ML portion of Matlab unusable. I’m hoping someone here can help me out because rolling back my matlab two years will make a lot of my recent work unstable. I’ve done my best to document my process here. Error messages are given in italics for ease of reading.
I have some data that is represented by four doubles that are normalzied to between 0 and 1. I would like to use these four doubles to predict two doubles that are also between 0 and 1. I would like to make a multilayer perceptron that will take in the four values and spit out the two values. However, createMLPNetwork doesn’t appear to support multiple outputs – at least, the documentation for it doesn’t explain how to do so. So I have tried to make a MLP from scratch using the following code:
%% Prep NN architecture
layers = [
inputLayer([4, 1], "CU", name="in")
flattenLayer
fullyConnectedLayer(20, name="fc1")
reluLayer
fullyConnectedLayerB(16, name="fc2")
reluLayer
fullyConnectedLayer(12, name="fc3")
reluLayer
fullyConnectedLayer(numel(data{1,1}), name="output")
softmaxLayer
];
net = dlnetwork(layers);
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Attempting to run this gives me the following results:
Error using trainnet (line 46)
Error forming mini-batch for network input "in". Data interpreted with format "CU". To specify a different format, use the InputDataFormats option.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
Caused by:
The data format must have a batch dimension.
I tried to get around using the generic inputLayer function by using the more specific functions like featureInputLayer but replacing the inputLayer with a featureInputLayer throws the following:
Error using trainnet (line 46)
Number of observations in predictors (4) and targets (2) must match. Check that the data and network are consistent.
Error in trainRutileModel (line 107)
[trainedNet, info] = trainnet(train, layers, "mse", opts);
So that won’t work because I only have two output data points. The same happened when I tried imageInputLayer. Then I tried replacing "CU" with other values – the following error is for "BU" but the other errors are the same – but got the following:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 104)
net = dlnetwork(layers);
Caused by:
Layer ‘fc1’: Invalid input data for fully connected layer. The input data must have exactly one channel dimension.
So then I tried flattening the output and got the following error:
Error using dlnetwork/initialize (line 558)
Invalid network.
Error in dlnetwork (line 167)
net = initialize(net, dlX{:});
Error in trainRutileModel (line 105)
net = dlnetwork(layers);
Caused by:
Layer ‘flatten’: Invalid input data. Layer expects data with a channel dimension, but received input data with format "BU".
I’m really not sure what to do here. I have no idea why MathWorks would make this so much more difficult to use and give so much less, and more opaque, documentation. If anyone has any ideas on how to make this work, I’d be happy to hear them. In the meantime, I’m going to take another crack at the createMLPNetwork function and hope that when MathWorks says "a black-box continuous-time or discrete-time neural state-space model with identifiable (estimable) network weights and bias," by "states" they mean "outputs." multilayer perceptron, mlp, neural network, inputlayer MATLAB Answers — New Questions