Options
Different Results on Different Versions
Contributor I
I have performed an experiment using auto model on two different versions i.e. RapidMiner Studio 9.0.003 and 9.3.001. My input data, task, target, and features were exactly the same. I ran the auto model process with keeping the default settings, however, I got a different set of results. For instance, Naïve Base Accuracy and AUC values on version 9.0.003 were 40% and 0.836 respectively, while on the version 9.1.003 they were 64% and 0.535.
It will be helpful to get the possible reasons of such discrepancies.
It will be helpful to get the possible reasons of such discrepancies.
0
Best Answer
-
Options
BalazsBarany
Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert Posts: 955 
Unicorn
Hi,
AutoModel was improved a lot between 9.0 and 9.3. It is able to try new attribute combinations, optimize parameters etc.
Also, with a small example set, your results depend a lot on the random selection into training and test sets.
The model and the results should be identical or very similar between the version. The AutoModel result is not expected to be.
You can always inspect the generated processes and compare the training set and the resulting model.
Regards,
Balázs9
Answers
Lindon Ventures
Data Science Consulting from Certified RapidMiner Experts