[SOLVED]The prediction score are different between the Rapidminer and Libsvm

huaiyanggongzi

I ran the SVM classification using both Rapidminer and Libsvm against the same data set. In the rapidminer, I also used the libsvm implementation. The document processing scheme and svm parameter setting are exactly the same for both experiment. However, the generated score are different. Are there any possible reasons to cause this kind of different?

MariusHelf

What exactly do you mean by scoring? How did you evaluate it? If you have a RapidMiner process, please post it here.
Maybe it's also a version issue: I am not absolutely sure, but by a first quick glance it seems that the version of libSVM integrated into RapidMiner is based on v2.84.
Finally, what are the actual scores that you retrieved?

Best,
Marius

MariusHelf

Hi,

your processes look fine. Actually I don't know why exactly the confidences are different. Which version of libSVM are you using?
Did you use exactly the same training and test data with the same preprocessing for training and applying the SVM from the library?

Best, Marius

huaiyanggongzi

Hi,
Besides using libsvm module in Rapidminer, I used libsvm(version 3.12) as the comparison. Both experiments used the same training and test data set. The data pre-processing steps are also the same for both experiments.

MariusHelf

Then you are using a different major version of libSVM than is integrated into RapidMiner. That could play a role in the different values. Furthermore, I saw that there are different methods for getting predictions, like predict_values, predict_sigmoid_values etc. Maybe you (or the tool you are using) use different methods than RapidMiner does. I am not an expert for the details of the libSVM though, but you can have a look at the implementation in the class LibSVMModel in the RapidMiner source code.

huaiyanggongzi

Marius, Thanks. I changed the option from "confidence to multiclass" to "confidence calculation".

MariusHelf

And, just for the records, now the results are equal?

huaiyanggongzi

yes, the SVM score gotten from Rapidminer matches the LIBSVM command line very well. Thanks.