RapidMiner 9.7 is Now Available

Lots of amazing new improvements including true version control! Learn more about what's new here.

CLICK HERE TO DOWNLOAD

Classification by Regression Operator vs. Polynomial By Binomial Classification

mbukombuko Member Posts: 10 Contributor I
edited November 2019 in Help
Hi,

I have a multi class approach by using SVM (mySVM) and a operator to enable the multi classification. The problem is, that the two possible operators "Classification by Regression Operator" and "Polynomial By Binomial Classification" lead to different kind of results for the confidence values:

1) Classification by Regression Operator: confidence value element of (-∞, 1]. This seems to be the signed distance to the hyperplane. Is this correct? Why are there no values higher than 1? (1 would mean that it is on the edge of the margin. Might it depend on the kernel function?)

2) Polynomial By Binomial Classification: confidence value element of [0,1]. Is this any kind of probability? Definition?

Unfortunately, I don't find any hint how the confidence values are defined (with regard to the used operators or SVM implementation).

In order to use RapidMiner and the output, I need a clear understanding of the parameters, confidence values and dependencies to the RapidMiner operators.

I would be pleased, if you would help me with these issues!

Best regards,
Mark

PS: I have already opened a similar thread in another category (https://rapid-i.com/rapidforum/index.php/topic,9418.msg31536.html)

Answers

  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,514  RM Data Scientist
    confidences should usually be [0,1]. So the case in 1) seems odd to me. I never used classification by regression though.

    ~martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • mbukombuko Member Posts: 10 Contributor I
    @Martin thank you for your reply!

    Unfortunately, a clear description and definition of the output and operators are missing, but I need it in order to work scientifically correct. I am looking forward to an official answer from RapidMiner, but at the moment there is an absence of any reaction.
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,514  RM Data Scientist
    Hi,

    i guess there will be more offical answer than my postings, except if you ask at support.rapidminer.com, but this requieres a licence. I am employed at RapidMiner; That makes it kind of offical?

    What you can do is have a look at the source. The classification by Regression operator can be found at:

    https://github.com/rapidminer/rapidminer-studio/blob/master/src/main/java/com/rapidminer/operator/learner/meta/ClassificationByRegression.java

    There we find the comment:

    /**
    * For a classified dataset (with possibly more than two classes) builds a classifier using a
    * regression method which is specified by the inner operator. For each class {@rapidminer.math i} a
    * regression model is trained after setting the label to {@rapidminer.math +1} if the label equals
    * {@rapidminer.math i} and to {@rapidminer.math -1} if it is not. Then the regression models are
    * combined into a classification model. In order to determine the prediction for an unlabeled
    * example, all models are applied and the class belonging to the regression model which predicts
    * the greatest value is chosen.
    *
    * @author Ingo Mierswa, Simon Fischer
    */
    I checked the attached example process. Apperently confidence(Mine) is [0,1] but confidence(Rock) not. For some reason only one confidence is set "correctly". I will check the code deeper once I have more time

    ~Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • mbukombuko Member Posts: 10 Contributor I
    @Martin, thank you for your help!

    The comment is also written in the documentation. I have the same problem with the different confidence outputs: only one is set "correctly" (but only with 1 ?).

    What is the definition of the confidence value if it is in [0,1]? (I wanted to avoid looking into the implementations, but it seems to be necessary)
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,514  RM Data Scientist
    usually this is a measure on how much the algorithm trusts it's own calculation. Usually the higher the value the more likely is the algorithm to be right.

    The value is calculated for every algorithm differently. For a k-nn it is the fraction of neighbours with this class (in the unweighted case). For the SVM the value depends on the distance to the seperating hyperplane.
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • mbukombuko Member Posts: 10 Contributor I
    "For the SVM the value depends on the distance to the seperating hyperplane"
    - usually yes, but it also depends on the SVM implementation (mySVM vs. libSVM) and the wrapping operators like the two mentioned in the thread's title.
    Why are the confidence values for the same SVM algorithm but different operators within a different range (which is not documented)?
    In the case of libSVM there is the possibility to estimate the probability as confidence instead of the distance to the hyperplane. Unfortunately, the specific meanings of the confidence values regarding the algorithms etc. are not documented.

    I think I have to look into the implementions since the academic support does not help either.

    Thanks so far, I would be very pleased about any further support.
Sign In or Register to comment.