Comparing classification performance using Friedman Test
I am trying to compare the performance of three classifiers across four performance metrics using the Friedman test in MATLAB. Since MATLAB does not include a built-in Nemenyi post-hoc test, I used the "multcompare" function as suggested in related discussions. I obtained the following results. If I understand correctly, a high p-value indicates that there is no significant difference between the classifier performances. How should I interpret the values in c and m? Am I doing something wrong? Can Pearson’s r be used to compare the classifiers instead of Friedman and other post-hoc test? Thanks for the help.
PrfMat = [0.9352 0.9697 0.7475 0.9877;
0.9670 0.8713 0.8414 0.7052;
0.6944 0.6841 0.9851 0.9897];
[p,~,stats] = friedman(PrfMat, 1, ‘on’)
[c,m] = multcompare(stats, ‘CType’, ‘tukey-kramer’)
p = 0.8013
c = 1.0000 2.0000 -2.3747 0.3333 3.0413 0.9891
1.0000 3.0000 -2.0413 0.6667 3.3747 0.9216
1.0000 4.0000 -3.0413 -0.3333 2.3747 0.9891
2.0000 3.0000 -2.3747 0.3333 3.0413 0.9891
2.0000 4.0000 -3.3747 -0.6667 2.0413 0.9216
3.0000 4.0000 -3.7080 -1.0000 1.7080 0.7785
m = 2.6667 0.7454
2.3333 0.7454
2.0000 0.7454
3.0000 0.7454I am trying to compare the performance of three classifiers across four performance metrics using the Friedman test in MATLAB. Since MATLAB does not include a built-in Nemenyi post-hoc test, I used the "multcompare" function as suggested in related discussions. I obtained the following results. If I understand correctly, a high p-value indicates that there is no significant difference between the classifier performances. How should I interpret the values in c and m? Am I doing something wrong? Can Pearson’s r be used to compare the classifiers instead of Friedman and other post-hoc test? Thanks for the help.
PrfMat = [0.9352 0.9697 0.7475 0.9877;
0.9670 0.8713 0.8414 0.7052;
0.6944 0.6841 0.9851 0.9897];
[p,~,stats] = friedman(PrfMat, 1, ‘on’)
[c,m] = multcompare(stats, ‘CType’, ‘tukey-kramer’)
p = 0.8013
c = 1.0000 2.0000 -2.3747 0.3333 3.0413 0.9891
1.0000 3.0000 -2.0413 0.6667 3.3747 0.9216
1.0000 4.0000 -3.0413 -0.3333 2.3747 0.9891
2.0000 3.0000 -2.3747 0.3333 3.0413 0.9891
2.0000 4.0000 -3.3747 -0.6667 2.0413 0.9216
3.0000 4.0000 -3.7080 -1.0000 1.7080 0.7785
m = 2.6667 0.7454
2.3333 0.7454
2.0000 0.7454
3.0000 0.7454 I am trying to compare the performance of three classifiers across four performance metrics using the Friedman test in MATLAB. Since MATLAB does not include a built-in Nemenyi post-hoc test, I used the "multcompare" function as suggested in related discussions. I obtained the following results. If I understand correctly, a high p-value indicates that there is no significant difference between the classifier performances. How should I interpret the values in c and m? Am I doing something wrong? Can Pearson’s r be used to compare the classifiers instead of Friedman and other post-hoc test? Thanks for the help.
PrfMat = [0.9352 0.9697 0.7475 0.9877;
0.9670 0.8713 0.8414 0.7052;
0.6944 0.6841 0.9851 0.9897];
[p,~,stats] = friedman(PrfMat, 1, ‘on’)
[c,m] = multcompare(stats, ‘CType’, ‘tukey-kramer’)
p = 0.8013
c = 1.0000 2.0000 -2.3747 0.3333 3.0413 0.9891
1.0000 3.0000 -2.0413 0.6667 3.3747 0.9216
1.0000 4.0000 -3.0413 -0.3333 2.3747 0.9891
2.0000 3.0000 -2.3747 0.3333 3.0413 0.9891
2.0000 4.0000 -3.3747 -0.6667 2.0413 0.9216
3.0000 4.0000 -3.7080 -1.0000 1.7080 0.7785
m = 2.6667 0.7454
2.3333 0.7454
2.0000 0.7454
3.0000 0.7454 classification, friedman MATLAB Answers — New Questions









