Options

Compare ROC Parameter

PlatyQPlatyQ Member Posts: 4 Contributor I
edited September 2020 in Help

Hello,

I have some questions but I am not anglophone and may be it will difficult to you to understand my questions and difficult to me to understand your answer.

Please if I am not understandable say it to me.

I try to make a learning machine to a stage in computer science and I want compare some models to this.

I have very few examples (20) but I have to try anyway.

My troubles are to adjust the parameters of "compare ROC", I think understand what is a ROC curve and how interpreted but may be I make a mistake cause I don't understand why they are two parameters :
  • number_of_folds.
  • split_ratio.

According to me if i put 4 in number_of_folds for each models Rapidminer compares 4 training machine with 15 examples to learn and evaluate them with the last 5 and after create for each model the ROC Curves. And it match with that I want (compare some models to my 20 examples).

But split ratio can not be disable so it can play a role in the roc curve but i don't understand which. May be if I put 1 in this parameter it work as I think?

 

Thank for your help
Tagged:

Answers

  • Options
    lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, Member Posts: 1,195 Unicorn

    Hi @PlatyQ,

     

    Good question.

    I asked myself the same question when I used this operator for the first time, but I never .......dared to ask the question.

    Your post prompted me to look for an answer in the documentation : 

    According to the documentation, Compare ROC behaves like a Split Validation or a Cross Validation operator : 

     - if number of folds = -1, then Compare ROC behaves like a Split Validation with the defined split ratio.

     - if number of folds is in Range ]1, infinite[, then Compare ROC behaves like a Cross Validation with the defined number of folds.       (and in this case, the defined split ratio is not taken into account).

     -  if number of folds = 0 or 1 , then RapidMiner raise an error .... and it's logic.

     

    NB : That's what I understand from the documentation, but thanks to the experts to confirm or..... invalidate this explanation of the behaviour of Compare ROC operator.

     

    I hope it helps,

     

    Regards,

     

    Lionel

  • Options
    PlatyQPlatyQ Member Posts: 4 Contributor I

    Hi @lionelderkrikor and thank for your response

    I also search in documentation before ask the question but I read and read again and no see that "if number of folds is in Range ]1, infinite[ [...] (and in this case, the defined split ratio is not taken into account)". I think your right cause it seems more logic than use the two options in the same time but I regret than it is not well-defined I the documentation.

    I am reassure by your response but I hope we can have a second opinion.

     

    (And may be somebody know how to ask modifications in the documentation? If two people have doubts may be we are not the only ones.)

     

    thanks

Sign In or Register to comment.