Using transformer neural network for classification task

numChannels = inputSize;
maxPosition = 256;
numHeads = 4;
numKeyChannels = numHeads*32;

layers = [
sequenceInputLayer(numChannels,Name="input")
positionEmbeddingLayer(numChannels, maxPosition, Name="pos-emb");
additionLayer(2, Name="add")
selfAttentionLayer(numHeads,numKeyChannels,’AttentionMask’,’causal’)
selfAttentionLayer(numHeads,numKeyChannels)
indexing1dLayer("last")
fullyConnectedLayer(numClasses)
softmaxLayer
classificationLayer];

lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph, "input", "add/in2");

maxEpochs = 100;
miniBatchSize = 32;
learningRate = 0.001;
solver = ‘adam’;
shuffle = ‘every-epoch’;
gradientThreshold = 10;
executionEnvironment = "auto"; % chooses local GPU if available, otherwise CPU

options = trainingOptions(solver, …
‘Plots’,’training-progress’, …
‘MaxEpochs’, maxEpochs, …
‘MiniBatchSize’, miniBatchSize, …
‘Shuffle’, shuffle, …
‘InitialLearnRate’, learningRate, …
‘GradientThreshold’, gradientThreshold, …
‘ExecutionEnvironment’, executionEnvironment);

The input size is 12, so there are 12 features.
numClasses is 4, so I am classifying it into 4 class.

But it gives the following error when I try to run it

"
Error in test123_20240727 (line 195)
net=trainNetwork(XTrain, YTrain, layers, options);

Caused by:
Layer ‘add’: Unconnected input. Each layer input must be connected to the output of another layer.
"

line 195 is "net=trainNetwork(XTrain, YTrain, layers, options);"

Can anyone help me with this?numChannels = inputSize;
maxPosition = 256;
numHeads = 4;
numKeyChannels = numHeads*32;

lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph, "input", "add/in2");

The input size is 12, so there are 12 features.
numClasses is 4, so I am classifying it into 4 class.

But it gives the following error when I try to run it

"
Error in test123_20240727 (line 195)
net=trainNetwork(XTrain, YTrain, layers, options);

Caused by:
Layer ‘add’: Unconnected input. Each layer input must be connected to the output of another layer.
"

line 195 is "net=trainNetwork(XTrain, YTrain, layers, options);"

Can anyone help me with this? numChannels = inputSize;
maxPosition = 256;
numHeads = 4;
numKeyChannels = numHeads*32;

lgraph = layerGraph(layers);
lgraph = connectLayers(lgraph, "input", "add/in2");

The input size is 12, so there are 12 features.
numClasses is 4, so I am classifying it into 4 class.

But it gives the following error when I try to run it

"
Error in test123_20240727 (line 195)
net=trainNetwork(XTrain, YTrain, layers, options);

Caused by:
Layer ‘add’: Unconnected input. Each layer input must be connected to the output of another layer.
"

line 195 is "net=trainNetwork(XTrain, YTrain, layers, options);"

Can anyone help me with this? transformer, neural network, machine learning, classification MATLAB Answers — New Questions

Cart

Cart