Mapreduce with tall array in matrix multiplication
Hi,
I am processing huge dataset (billions rows), so I used tall array in my code. While I found the tall array are not support general matrix multiplication. The ‘*’ only works in a limited conditions.
I am wondering if we can combine mapreduce to perform matrix multiplication with tall arrays ? Below is a example shows the error.
A = tall(ones(200,2));
B = tall(ones(2,3));
values = A*B;
gather(values)
Error using tall/mtimes>iVerifyAtLeastOneScalar
Matrix multiplication of two tall arrays requires one of them to be scalar.
Learn more about errors encountered during GATHER.
Error in * (line 31)
[X,Y] = iVerifyAtLeastOneScalar(X,Y,"MATLAB:bigdata:array:MtimesBothTall");Hi,
I am processing huge dataset (billions rows), so I used tall array in my code. While I found the tall array are not support general matrix multiplication. The ‘*’ only works in a limited conditions.
I am wondering if we can combine mapreduce to perform matrix multiplication with tall arrays ? Below is a example shows the error.
A = tall(ones(200,2));
B = tall(ones(2,3));
values = A*B;
gather(values)
Error using tall/mtimes>iVerifyAtLeastOneScalar
Matrix multiplication of two tall arrays requires one of them to be scalar.
Learn more about errors encountered during GATHER.
Error in * (line 31)
[X,Y] = iVerifyAtLeastOneScalar(X,Y,"MATLAB:bigdata:array:MtimesBothTall"); Hi,
I am processing huge dataset (billions rows), so I used tall array in my code. While I found the tall array are not support general matrix multiplication. The ‘*’ only works in a limited conditions.
I am wondering if we can combine mapreduce to perform matrix multiplication with tall arrays ? Below is a example shows the error.
A = tall(ones(200,2));
B = tall(ones(2,3));
values = A*B;
gather(values)
Error using tall/mtimes>iVerifyAtLeastOneScalar
Matrix multiplication of two tall arrays requires one of them to be scalar.
Learn more about errors encountered during GATHER.
Error in * (line 31)
[X,Y] = iVerifyAtLeastOneScalar(X,Y,"MATLAB:bigdata:array:MtimesBothTall"); tall array, datastore, mapreduce MATLAB Answers — New Questions