RapidMiner

F-Score for Multiclass Problems?

Elite II

F-Score for Multiclass Problems?

hi,

in binomial classification, f-measure exists, but not in Performance (Classification) operator for multiclass data, 

does there exist a f-measure for multiclass problems? is it in some way possible to built or generate an attribute f-score (multiclass) with the existing fields like precision / recall etc.?

 

what about 2* (Precision(class1+class2+...+class_n)*Recall(class1+..+class_n)) / (Sum of precision + recall over all classes) ?

does that make sense in some way? what do you guys think? 

Is this still comparative? I know with more classes it is likely that precision and recall will go down, do the multiplication factors tend to weight too much and take bad performance too much into account, regarding the division by the sum of terms in the denominator? or is it still accurately enough?

 

I just want to get a better performance measure overall for the whole result if one class is just massively underrepresented...

3 REPLIES
Moderator

Re: F-Score for Multiclass Problems?

Hi,

 

isn't logloss a good option for you?


~Martin

--------------------------------------------------------------------------
Head of Data Science Services at RapidMiner
Elite II

Re: F-Score for Multiclass Problems?

hm I don't know how to interpret logistic loss..

it is ln(1+exp(-[conf(CC)])), so it calculates that formula with the value of confidence for the class predicted which (should) be the correct class, and adds it up for all training examples and then averages it, is that correct?

so the smaller the confidence, the bigger the logistic loss.. but in what range are the values and how should I interpret them? what is a small/ big loss?

Elite III

Re: F-Score for Multiclass Problems?

Here's a good simple explanation of logloss from Kaggle, where it is a popular performance metric:

https://www.kaggle.com/wiki/LogarithmicLoss

 

The value range is dependent on the dataset you have and the predictive power of your attributes, there is no absolute answer that says what a "good" logloss is vs a "bad" one.

 

Brian T., Lindon Ventures - www.lindonventures.com
Analytics Consulting by Certified RapidMiner Analysts