🎉 🎉 RAPIDMINER 9.10 IS OUT!!! 🎉🎉

Download the latest version helping analytics teams accelerate time-to-value for streaming and IIOT use cases.

CLICK HERE TO DOWNLOAD

AUC in rapidminer results in 0

NonaNona Member Posts: 15 Contributor I
edited September 2019 in Help
hii all,
I have a dataset with 2 columns, the first one is label and the second column is prediction i.e. i have stored results of my algorithm in a column and now i want to evaluate performance of that algorithm with rapidminer. i have imported data and specified the columns as label and prediction and  evaluating the performance using binomial classification performance evaluation. it gives correct results on confusion matrix and f-measure but in AUC it shows 0. Please suggest me how should i use rapidminer to get the correct AUC score on my data.
Tagged:

Answers

  • RalfKlinkenbergRalfKlinkenberg Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member, Unconfirmed, University Professor Posts: 68  RM Founder
    Hi Nona,
    in order to be able to compute the ROC curve and the Area Under the ROC Curve (AUC), RapidMiner needs to be able to sort the classified examples/predictions by their confidence of being positive examples. Hence you also need the confidence column(s) predicted by your model.
    Best regards,
    Ralf
  • NonaNona Member Posts: 15 Contributor I
    Hi Ralf
    Thanks for the reply. But my algorithm does not return any probability or confidence value. How could I evaluate its performance?  My dataset is imbalanced.
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,097  RM Data Scientist
    Hi Nona,

    which algorithm are you using? I think every predictive algorithm in RM returns confidences. AUC is a measure which is only possible if you have some kind of confidence in place.

    You could try to emulate a similar thing using different weights for the classes. given that you can get different recalls and precisions. Not sure how to do the calculation though.

    ~Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • NonaNona Member Posts: 15 Contributor I
    Hii Martin
    I am not using any rapidminer algorithm. .I have designed a new algorithm and I want to compare its performance with others. I am doing all other experiments in rapidminer and that's why I imported the results of my algorithm in rapidminer to compare the results.
Sign In or Register to comment.