I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with ‘m’.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval – Memory consumption piling up – Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here

% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantageI am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with ‘m’.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval – Memory consumption piling up – Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here

% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantage I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with ‘m’.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval – Memory consumption piling up – Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here

% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantage parfeval, fetchnext MATLAB Answers — New Questions

​