Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
How important is the confidence interval? Should I worry if the confidence is low?
Learner I
Hi,
I have made two final models, both are performing equally good when cross validating. When it comes to deployment in a business setting, one of them shows very low confidence in its prediction (the most confident prediction is around .6 and even when its confidence is 0.25 or more, it decides to predict it as true.) How come? When it is predicting on the training and test set, it has much higher confidence in its predictions.
My other model is equally good in terms of cross validation, but when deploying it, it has much higher confidence in its prediction (most ranging from .6 to .9
Should I ignore this or can I conclude based on this, that the latter model is better to use?
I have made two final models, both are performing equally good when cross validating. When it comes to deployment in a business setting, one of them shows very low confidence in its prediction (the most confident prediction is around .6 and even when its confidence is 0.25 or more, it decides to predict it as true.) How come? When it is predicting on the training and test set, it has much higher confidence in its predictions.
My other model is equally good in terms of cross validation, but when deploying it, it has much higher confidence in its prediction (most ranging from .6 to .9
Should I ignore this or can I conclude based on this, that the latter model is better to use?
Tagged:
0
Answers
Ingo