How can I efficiently save and access large arrays generated in nested loops?
I need to run nested for-loops over the variables J1 and J2. The range for J1 is 1 to 41, and the range for J2 is 1 to 9. Inside these loops, I evaluate 16 functions, each of which returns an array of complex numbers with a size of 500 by 502.
I used the following given method to save the data, and it produced an 11 GB file, which seems very large. Is this normal? What is an efficient way to save this data at the end of the calculation?
What I want to do with this data afterward:
I will need to access the 16 arrays, A1 to A16, within the same J1 and J2 loop to perform other operations. Therefore, I want to store the data in a way that allows easy access to these 16 arrays within the loops.
My method to store data:
all_data = cell(41,9);
for J1 = 1:41
for J2 = 1:9
%evaluate 16 function to get 16 arrays (A1 to A16) of size 500 x 502:
all_data{J1,J2} = struct("A1", A1,…
"A2", A2,…
"A3", A3,…
"A4", A4,…
"A5", A5,…
"A6", A6,…
"A7", A7,…
"A8", A8,…
"A9", A9,…
"A10", A10,…
"A11", A11,…
"A12", A12,…
"A13", A13,…
"A14", A14,…
"A15", A15,…
"A16", A16);
end
end
save(‘Saved_Data.mat’,’-v7.3′);I need to run nested for-loops over the variables J1 and J2. The range for J1 is 1 to 41, and the range for J2 is 1 to 9. Inside these loops, I evaluate 16 functions, each of which returns an array of complex numbers with a size of 500 by 502.
I used the following given method to save the data, and it produced an 11 GB file, which seems very large. Is this normal? What is an efficient way to save this data at the end of the calculation?
What I want to do with this data afterward:
I will need to access the 16 arrays, A1 to A16, within the same J1 and J2 loop to perform other operations. Therefore, I want to store the data in a way that allows easy access to these 16 arrays within the loops.
My method to store data:
all_data = cell(41,9);
for J1 = 1:41
for J2 = 1:9
%evaluate 16 function to get 16 arrays (A1 to A16) of size 500 x 502:
all_data{J1,J2} = struct("A1", A1,…
"A2", A2,…
"A3", A3,…
"A4", A4,…
"A5", A5,…
"A6", A6,…
"A7", A7,…
"A8", A8,…
"A9", A9,…
"A10", A10,…
"A11", A11,…
"A12", A12,…
"A13", A13,…
"A14", A14,…
"A15", A15,…
"A16", A16);
end
end
save(‘Saved_Data.mat’,’-v7.3′); I need to run nested for-loops over the variables J1 and J2. The range for J1 is 1 to 41, and the range for J2 is 1 to 9. Inside these loops, I evaluate 16 functions, each of which returns an array of complex numbers with a size of 500 by 502.
I used the following given method to save the data, and it produced an 11 GB file, which seems very large. Is this normal? What is an efficient way to save this data at the end of the calculation?
What I want to do with this data afterward:
I will need to access the 16 arrays, A1 to A16, within the same J1 and J2 loop to perform other operations. Therefore, I want to store the data in a way that allows easy access to these 16 arrays within the loops.
My method to store data:
all_data = cell(41,9);
for J1 = 1:41
for J2 = 1:9
%evaluate 16 function to get 16 arrays (A1 to A16) of size 500 x 502:
all_data{J1,J2} = struct("A1", A1,…
"A2", A2,…
"A3", A3,…
"A4", A4,…
"A5", A5,…
"A6", A6,…
"A7", A7,…
"A8", A8,…
"A9", A9,…
"A10", A10,…
"A11", A11,…
"A12", A12,…
"A13", A13,…
"A14", A14,…
"A15", A15,…
"A16", A16);
end
end
save(‘Saved_Data.mat’,’-v7.3′); storage, data, big data, matlab MATLAB Answers — New Questions