Positive class assigning logic

kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 291 Unicorn
edited December 2018 in Help

Hi rapidminers, 

 

Alike topic has been discussed already in many threads here (yes I've read them all...) but still this is something which becomes pretty unclear eventually. 

 

Say, we have a binary classifier and two classes, 1/0 or Yes/No or true/false, whatever.

 

We optimize the performnace towards recall, which is true positive rate:

 

Screenshot 2017-10-20 10.42.12.png

There's an internal mapping of labels to positive and negative class, but which is not known beforehand.

So we end up with recall where we see that positive class has been assigned to label 0: 

 

Screenshot 2017-10-20 10.41.48.png

In my case 0 is 'good', so we optimized for best prediction of good cases (in real life problem, for example, this might mean to correctly detect maximum number of good transactions at the cost of maybe letting also some bad transactions through). But in fact, this is known only after the performance evaluation when we see what ecacly label is assigned to a positive class. 

 

How can we know BEFOREHAND wht's the logic of assigning the classes to positive and negative, regardless of actual labels? 

I know about 'REMAP BINOMIALS' which is a bit tricky operator as it changes INTERNAL mapping but has no effect on the visible results; hence, if I apply it here in any way, I still will ALWAYS get that 'positive class: 0' regardless of remapping. 

 

This starts to drive me nuts as I feel like not understanding apriori, WHAT EXACTLY CLASS RECALL I am optimizing for. 

 

Can anyone explain the certain logic of assigning classes once again and if there's any way of intentionally changing it? Or RapidMiner logic always chooses positive class in some certain way for each certain dataset / process and we just have to live with it once we found out what is an actual positive class in this case?

 

Thanks a lot.

Best Answer

Answers

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    This is a great question, I have also wondered whether there is a certain way to force RapidMiner to treat a given class as positive for its performance metrics.

     

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    I need to call the bullpen on this one.  @IngoRM ?

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    Thanks @IngoRM, it's always great to learn a new RapidMiner trick!

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 291 Unicorn

    Thanks @IngoRM!

     

    That all seems logical and correct, with the exception of the fact that (I swear!) for some reason 'REMAP BINOMIALS' hadn't any effect on the class mapping when I previously tried it today, or maybe it's just the end of the hard week... don't know. I just tried it now once again and magically it works and remaps classes. So yes, this is what I wanted to achieve. 

     

    Good weekend everyone! :) 

  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder

    Suuuuuure :smileytongue:

    Maybe you have accidentally swapped the two classes in the parameter settings or something like that.  Anyway, I am glad it works now.

    Enjoy your weekend :smileywink:

Sign In or Register to comment.