Weights and Biases Not Updating in Custom MATLAB dlnetwork Training Loop

Hello MATLAB Community,
I am currently working on training a custom autoencoder network using MATLAB’s dlnetwork framework. Despite setting up a manual training loop with gradient computation and parameter updates using adamupdate, I’ve observed that the weights and biases of the network do not change between iterations. Additionally, all biases remain zero throughout training. I am using Matlab R2024a. Here are the relevant parts of my code:

trailingAvgG = [];
trailingAvgSqG = [];
trailingAvgR = [];
trailingAvgSqR = [];
miniBatchSize = 600;
learnRate = 0.01;
layers = [
sequenceInputLayer(1,MinLength = 2048)
modwtLayer(‘Level’,5,’IncludeLowpass’,false,’SelectedLevels’,2:5,"Wavelet","sym2")
flattenLayer
convolution1dLayer(128,8,Padding="same",Stride=8)
batchNormalizationLayer()
tanhLayer
maxPooling1dLayer(2,Padding="same")
convolution1dLayer(32,8,Padding="same",Stride=4)
batchNormalizationLayer
tanhLayer
maxPooling1dLayer(2,Padding="same")
transposedConv1dLayer(32,8,Cropping="same",Stride=4)
tanhLayer
transposedConv1dLayer(128,8,Cropping="same",Stride=8)
tanhLayer
bilstmLayer(8)
fullyConnectedLayer(8)
dropoutLayer(0.2)
fullyConnectedLayer(4)
dropoutLayer(0.2)
fullyConnectedLayer(1)];

net = dlnetwork(layers);
numEpochs = 200;

%dataMat = 1x2048x22275
dldata = arrayDatastore(dataMat,IterationDimension=3);

mbq = minibatchqueue(dldata,…
MiniBatchSize=miniBatchSize, …
OutputEnvironment= "cpu");

iteration = 0;

for epoch = 1:numEpochs

shuffle(mbq);

while hasdata(mbq)
iteration = iteration+1;
[XTrain] = next(mbq);
XTrain = dlarray(XTrain,"TBC"); % 1(C)x600(B)x2048(T)
[datafromRNN,lossR] = RNN_model(XTrain,net);

[gradientsR] = dlfeval(@gradientFunction,mean(lossR), net);

[net,trailingAvgR,trailingAvgSqR] = adamupdate(net,gradientsR, …
trailingAvgR,trailingAvgSqR,iteration,learnRate);
disp([‘Iteration ‘, num2str(iteration), ‘, Loss: ‘, num2str(extractdata(lossR))]);
end
end
function [gradientsR] = gradientFunction(lossR, net)
gradientsR = dlgradient(lossR, net.Learnables);
end

function [datafromRNN,loss] = RNN_model(data,net)

z = data;
[coder, last] = forward(net, z, ‘Outputs’, {‘maxpool1d_2’, ‘fc_3’});
loss = mse(last,z);
end

Questions:
Why are the weights and biases not updating, and why do the biases remain zero?
How can I ensure that the gradients computed are correct and being applied effectively?
Are there any specific settings or modifications I should consider to resolve this issue?
Any insights or suggestions would be greatly appreciated!Hello MATLAB Community,
I am currently working on training a custom autoencoder network using MATLAB’s dlnetwork framework. Despite setting up a manual training loop with gradient computation and parameter updates using adamupdate, I’ve observed that the weights and biases of the network do not change between iterations. Additionally, all biases remain zero throughout training. I am using Matlab R2024a. Here are the relevant parts of my code:

net = dlnetwork(layers);
numEpochs = 200;

%dataMat = 1x2048x22275
dldata = arrayDatastore(dataMat,IterationDimension=3);

mbq = minibatchqueue(dldata,…
MiniBatchSize=miniBatchSize, …
OutputEnvironment= "cpu");

iteration = 0;

for epoch = 1:numEpochs

shuffle(mbq);

while hasdata(mbq)
iteration = iteration+1;
[XTrain] = next(mbq);
XTrain = dlarray(XTrain,"TBC"); % 1(C)x600(B)x2048(T)
[datafromRNN,lossR] = RNN_model(XTrain,net);

[gradientsR] = dlfeval(@gradientFunction,mean(lossR), net);

function [datafromRNN,loss] = RNN_model(data,net)

z = data;
[coder, last] = forward(net, z, ‘Outputs’, {‘maxpool1d_2’, ‘fc_3’});
loss = mse(last,z);
end

Questions:
Why are the weights and biases not updating, and why do the biases remain zero?
How can I ensure that the gradients computed are correct and being applied effectively?
Are there any specific settings or modifications I should consider to resolve this issue?
Any insights or suggestions would be greatly appreciated! Hello MATLAB Community,
I am currently working on training a custom autoencoder network using MATLAB’s dlnetwork framework. Despite setting up a manual training loop with gradient computation and parameter updates using adamupdate, I’ve observed that the weights and biases of the network do not change between iterations. Additionally, all biases remain zero throughout training. I am using Matlab R2024a. Here are the relevant parts of my code:

net = dlnetwork(layers);
numEpochs = 200;

%dataMat = 1x2048x22275
dldata = arrayDatastore(dataMat,IterationDimension=3);

mbq = minibatchqueue(dldata,…
MiniBatchSize=miniBatchSize, …
OutputEnvironment= "cpu");

iteration = 0;

for epoch = 1:numEpochs

shuffle(mbq);

while hasdata(mbq)
iteration = iteration+1;
[XTrain] = next(mbq);
XTrain = dlarray(XTrain,"TBC"); % 1(C)x600(B)x2048(T)
[datafromRNN,lossR] = RNN_model(XTrain,net);

[gradientsR] = dlfeval(@gradientFunction,mean(lossR), net);

function [datafromRNN,loss] = RNN_model(data,net)

z = data;
[coder, last] = forward(net, z, ‘Outputs’, {‘maxpool1d_2’, ‘fc_3’});
loss = mse(last,z);
end

Cart

Cart