Optimizing Hyperparameters for trainnet fucntion

Hi there I have built my own TCN model in matlab to predict a contunious output and am trying to figure out the best way to optimize the hyperparmeters: Filter Size, Number of Filters, Number of Blocks, and Drop out Factor. I am attempting to use the bayespot function, but am not sure what to use as my function handel and if this is the best method for this kind of network. Should I be using the experiment manger to do this instead (https://www.mathworks.com/help/deeplearning/ug/tune-experiment-hyperparameters-using-bayesian-optimization.html) Does anyone have an suggestions for my code or if there is another way to perform hyperparmeter optimization for this type of network achetecture? Thanks so much.
%Network
numFilters = 64;
filterSize = 5;
droupoutFactor = 0.005;
numBlocks = 5;

net = dlnetwork;

layer = sequenceInputLayer(numFeatures,Normalization="rescale-symmetric",Name="input");

net = addLayers(net,layer);

for i = 1:numBlocks
dilationFactor = 2^(i-1);

layers = [
convolution1dLayer(filterSize,numFilters,DilationFactor=dilationFactor,Padding="causal",Name="conv1_"+i)
layerNormalizationLayer
spatialDropoutLayer(Name= "spat_drop_"+i,Probability=droupoutFactor)
convolution1dLayer(filterSize,numFilters,DilationFactor=dilationFactor,Padding="causal")
layerNormalizationLayer
reluLayer
spatialDropoutLayer(Name="spat_drop2_"+i,Probability=droupoutFactor)
additionLayer(2,Name="add_"+i)];

% Add and connect layers.
net = addLayers(net,layers);
net = connectLayers(net,outputName,"conv1_"+i);

end

net = connectLayers(net,outputName,"fc");

%Training Options
options = trainingOptions("adam", …
‘MaxEpochs’, 60, …
‘MiniBatchSize’, 1, …
‘InputDataFormat’, "CTB", …
‘Metrics’, "rmse", …
‘Verbose’, 0);

filtsize = optimizableVariable(‘filterSize’,[1,10],’Type’,’integer’)
numfilt = optimizableVariable(‘numFilters’,[20,60],’Type’,’integer’)
numblock = optimizableVariable(‘numBlocks’,[1,10],’Type’,’integer’)
dropout = optimizableVariable(‘dropoutfactor’,[0.001,0.01],’Type’,’integer’)

net = trainnet(traningdataX,trainingdataY,net,"mse",options);
fun = (x)@ %Not sure what to put here!

reuslts = bayesopt(,[filtsize, numfilt, numblock, droupout])Hi there I have built my own TCN model in matlab to predict a contunious output and am trying to figure out the best way to optimize the hyperparmeters: Filter Size, Number of Filters, Number of Blocks, and Drop out Factor. I am attempting to use the bayespot function, but am not sure what to use as my function handel and if this is the best method for this kind of network. Should I be using the experiment manger to do this instead (https://www.mathworks.com/help/deeplearning/ug/tune-experiment-hyperparameters-using-bayesian-optimization.html) Does anyone have an suggestions for my code or if there is another way to perform hyperparmeter optimization for this type of network achetecture? Thanks so much.
%Network
numFilters = 64;
filterSize = 5;
droupoutFactor = 0.005;
numBlocks = 5;

net = dlnetwork;

layer = sequenceInputLayer(numFeatures,Normalization="rescale-symmetric",Name="input");

net = addLayers(net,layer);

for i = 1:numBlocks
dilationFactor = 2^(i-1);

% Add and connect layers.
net = addLayers(net,layers);
net = connectLayers(net,outputName,"conv1_"+i);

end

net = connectLayers(net,outputName,"fc");

%Training Options
options = trainingOptions("adam", …
‘MaxEpochs’, 60, …
‘MiniBatchSize’, 1, …
‘InputDataFormat’, "CTB", …
‘Metrics’, "rmse", …
‘Verbose’, 0);

net = trainnet(traningdataX,trainingdataY,net,"mse",options);
fun = (x)@ %Not sure what to put here!

reuslts = bayesopt(,[filtsize, numfilt, numblock, droupout]) Hi there I have built my own TCN model in matlab to predict a contunious output and am trying to figure out the best way to optimize the hyperparmeters: Filter Size, Number of Filters, Number of Blocks, and Drop out Factor. I am attempting to use the bayespot function, but am not sure what to use as my function handel and if this is the best method for this kind of network. Should I be using the experiment manger to do this instead (https://www.mathworks.com/help/deeplearning/ug/tune-experiment-hyperparameters-using-bayesian-optimization.html) Does anyone have an suggestions for my code or if there is another way to perform hyperparmeter optimization for this type of network achetecture? Thanks so much.
%Network
numFilters = 64;
filterSize = 5;
droupoutFactor = 0.005;
numBlocks = 5;

net = dlnetwork;

layer = sequenceInputLayer(numFeatures,Normalization="rescale-symmetric",Name="input");

net = addLayers(net,layer);

for i = 1:numBlocks
dilationFactor = 2^(i-1);

% Add and connect layers.
net = addLayers(net,layers);
net = connectLayers(net,outputName,"conv1_"+i);

end

net = connectLayers(net,outputName,"fc");

%Training Options
options = trainingOptions("adam", …
‘MaxEpochs’, 60, …
‘MiniBatchSize’, 1, …
‘InputDataFormat’, "CTB", …
‘Metrics’, "rmse", …
‘Verbose’, 0);

net = trainnet(traningdataX,trainingdataY,net,"mse",options);
fun = (x)@ %Not sure what to put here!

reuslts = bayesopt(,[filtsize, numfilt, numblock, droupout]) optimization, machine learning, hyperparameters MATLAB Answers — New Questions

Cart

Cart