Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
"[SOLVED] Using user set confidence for AUC performance, not working.."
Contributor II
Hi!
What I'm trying to do is use a polynominal model's confidence for a single class to plot AUC with a binominal label corresponding to that same class.
What I did was:
1. Rename confidence(class) to confidence(1).
2. Generate attribute confidence(0) = 0.
3. Set roles, confidence(1) to confidence_1 and 0 to 0.
4. Rename prediction(multiclass) to prediction(singleclass).
5. Map classofinterest to 1 and default to 0.
6. Transform the polynominal prediction to binominal.
This gives me the 3 prediction roles (prediction, confidence_0 and 1) that look exactly the same in metadata as the model specifically done for the binominal label.
Yet the AUC isn't using confidence, because AUC gives 0.5, optimistic 1 and pessimistic 0.
On the other hand, the prediction label works because I can get accuracy.
I guess there's some other meta-metadata that I can't see. Any way to achieve my goal?
Thanks a lot!
What I'm trying to do is use a polynominal model's confidence for a single class to plot AUC with a binominal label corresponding to that same class.
What I did was:
1. Rename confidence(class) to confidence(1).
2. Generate attribute confidence(0) = 0.
3. Set roles, confidence(1) to confidence_1 and 0 to 0.
4. Rename prediction(multiclass) to prediction(singleclass).
5. Map classofinterest to 1 and default to 0.
6. Transform the polynominal prediction to binominal.
This gives me the 3 prediction roles (prediction, confidence_0 and 1) that look exactly the same in metadata as the model specifically done for the binominal label.
Yet the AUC isn't using confidence, because AUC gives 0.5, optimistic 1 and pessimistic 0.
On the other hand, the prediction label works because I can get accuracy.
I guess there's some other meta-metadata that I can't see. Any way to achieve my goal?
Thanks a lot!
Tagged:
0
Answers
Solution: instead of (2) generate attribute confidence(0) = 0, generate it equal to 1- confidence_1...