Confidence of a predeiction

Papad

Hi everyone,
When I see a prediction in my result like this 

in line 4 I have confidence 1 for loyal , but the prediction is wrong. Does that mean that my model is not so good?
Thanks in advance.

varunm1

Hello @Papad

The predicted output of a model is based on the confidence value. The model checks the confidence and assigns a label (churn or loyal) based on a threshold of 0.5. In your line 4 we can see that your model thinks that the sample belongs to loyal as the confidence is 1. We can see that your model also gave wrong predictions for line 1 and 3.

A model goodness depends on its performance metrics like AUC, Kappa etc. Its difficult to determine worth of a model based on single sample.

MartinLiebig

Hi,

my 2 cents: https://towardsdatascience.com/why-you-should-ignore-data-science-performance-measures-785e9b56e941

BR,

Martin