Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Simulator and test-training data in Automodel
Chemical_eng
Member Posts: 16 Contributor II
in Help
Hello,
I am using AutoModel. I have some questions :
1. Is the simulator based on test data, training data or all data ?
2. How do I ensure that my test dataset is balanced, I have a lot of categorical variables, how do I ensure then the test dataset is balanced ?
3. Can I see metrics for both training and test error rates ? I think I only see for test.
Thanks
I am using AutoModel. I have some questions :
1. Is the simulator based on test data, training data or all data ?
2. How do I ensure that my test dataset is balanced, I have a lot of categorical variables, how do I ensure then the test dataset is balanced ?
3. Can I see metrics for both training and test error rates ? I think I only see for test.
Thanks
Tagged:
0
Best Answers
-
MartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,528 RM Data ScientistHi @Chemical_eng ,1. Simulator uses mostly none of the two data sets. It is in the end applying the model on the data set you configure. The data is only used to determine min and max.2. Usually you won't balance on attributes, but only on labels? You can do this by changing the costs/gains matrix3. Only test error rates are reported, because train error rates are rarely of any use.BR,Martin- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany0 -
MartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,528 RM Data ScientistHi @Chemical_eng ,i suppose not all random seeds are set to a fixed seed, so that the split and also the randomness in some algorithms (or the randomness introduced by parallel computation) does change the results slightly.The results should not differ much, right?BR,Martin- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany0
Answers
3. I wanted the error on train dataset to compare to test for overfitting/underfitting purposes, but is ok.
Also another question : 1. I am getting different result when I run train the algorithm in the same dataset, I assume is because of random errors by the algorithm parameters or by choosing the data for train-test split. I am using the model for optimization and I see it gives me different recommendations every time I train, any ideas on how to keep the model fixed ? can we fix these random parameters or choose and average or best combination?