I use Matlab 2024b realase to construct lstm-attention-lstm network, the key and value come from the output of the lstm1, and the query comes from lstm2’s output, however when I make this, the Matlab tells me that there is a net circle error. How can I achieve me goal whithout error. Thank you very much.
Here is my code and layers:
net = dlnetwork;
layers = [
sequenceInputLayer(InputChannels,Name="input")
lstmLayer(hidden_num_lstm1, ‘OutputMode’, ‘sequence’, Name=’lstm_encoder’)
dropoutLayer(0.2, Name=’dropout_1′)
layerNormalizationLayer(Name=’batchnormal_1′)
fullyConnectedLayer(AttentionChannels, Name="key") % Key
attentionLayer(NumHeads, Name="cross-attention")
layerNormalizationLayer
lstmLayer(hidden_num_lstm2, ‘OutputMode’, ‘sequence’, Name=’lstm_decoder’)
dropoutLayer(0.2, Name=’dropout_2′)
layerNormalizationLayer(Name=’batchnormal_2′)
fullyConnectedLayer(64, Name=’fc’)
fullyConnectedLayer(OutputChannels, Name=’fc_out’)
];
net = addLayers(net,layers);
net = connectLayers(net,’key’,’cross-attention/key’);
net = disconnectLayers(net,’key’,’cross-attention/query’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="value") % Value
];
net = addLayers(net,layers);
net = connectLayers(net,’batchnormal_1′,’value’);
net = connectLayers(net,’value’,’cross-attention/value’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="query") % Query
];
net = addLayers(net,layers);
net = connectLayers(net,’lstm_decoder’,’query’);
net = connectLayers(net,’query’,’cross-attention/query’);
plot(net);I use Matlab 2024b realase to construct lstm-attention-lstm network, the key and value come from the output of the lstm1, and the query comes from lstm2’s output, however when I make this, the Matlab tells me that there is a net circle error. How can I achieve me goal whithout error. Thank you very much.
Here is my code and layers:
net = dlnetwork;
layers = [
sequenceInputLayer(InputChannels,Name="input")
lstmLayer(hidden_num_lstm1, ‘OutputMode’, ‘sequence’, Name=’lstm_encoder’)
dropoutLayer(0.2, Name=’dropout_1′)
layerNormalizationLayer(Name=’batchnormal_1′)
fullyConnectedLayer(AttentionChannels, Name="key") % Key
attentionLayer(NumHeads, Name="cross-attention")
layerNormalizationLayer
lstmLayer(hidden_num_lstm2, ‘OutputMode’, ‘sequence’, Name=’lstm_decoder’)
dropoutLayer(0.2, Name=’dropout_2′)
layerNormalizationLayer(Name=’batchnormal_2′)
fullyConnectedLayer(64, Name=’fc’)
fullyConnectedLayer(OutputChannels, Name=’fc_out’)
];
net = addLayers(net,layers);
net = connectLayers(net,’key’,’cross-attention/key’);
net = disconnectLayers(net,’key’,’cross-attention/query’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="value") % Value
];
net = addLayers(net,layers);
net = connectLayers(net,’batchnormal_1′,’value’);
net = connectLayers(net,’value’,’cross-attention/value’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="query") % Query
];
net = addLayers(net,layers);
net = connectLayers(net,’lstm_decoder’,’query’);
net = connectLayers(net,’query’,’cross-attention/query’);
plot(net); I use Matlab 2024b realase to construct lstm-attention-lstm network, the key and value come from the output of the lstm1, and the query comes from lstm2’s output, however when I make this, the Matlab tells me that there is a net circle error. How can I achieve me goal whithout error. Thank you very much.
Here is my code and layers:
net = dlnetwork;
layers = [
sequenceInputLayer(InputChannels,Name="input")
lstmLayer(hidden_num_lstm1, ‘OutputMode’, ‘sequence’, Name=’lstm_encoder’)
dropoutLayer(0.2, Name=’dropout_1′)
layerNormalizationLayer(Name=’batchnormal_1′)
fullyConnectedLayer(AttentionChannels, Name="key") % Key
attentionLayer(NumHeads, Name="cross-attention")
layerNormalizationLayer
lstmLayer(hidden_num_lstm2, ‘OutputMode’, ‘sequence’, Name=’lstm_decoder’)
dropoutLayer(0.2, Name=’dropout_2′)
layerNormalizationLayer(Name=’batchnormal_2′)
fullyConnectedLayer(64, Name=’fc’)
fullyConnectedLayer(OutputChannels, Name=’fc_out’)
];
net = addLayers(net,layers);
net = connectLayers(net,’key’,’cross-attention/key’);
net = disconnectLayers(net,’key’,’cross-attention/query’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="value") % Value
];
net = addLayers(net,layers);
net = connectLayers(net,’batchnormal_1′,’value’);
net = connectLayers(net,’value’,’cross-attention/value’);
plot(net);
layers = [
fullyConnectedLayer(AttentionChannels, Name="query") % Query
];
net = addLayers(net,layers);
net = connectLayers(net,’lstm_decoder’,’query’);
net = connectLayers(net,’query’,’cross-attention/query’);
plot(net); attentionlayer, network, deeplearning MATLAB Answers — New Questions

​