RapidMiner 9.7 is Now Available

Lots of amazing new improvements including true version control! Learn more about what's new here.

CLICK HERE TO DOWNLOAD

"SVM vs Logistic Regression"

gokugoku Member Posts: 4 Contributor I
edited June 2019 in Help
I'd like some help.
I'm performing document classifiaction using SVM and Simple Logistic and I want to choose which one is better for my case.
From what I see, SVM model is build faster than simple logistic, but simple logistic seems to have better precision and recall than SVM.
So, I have some silly questions that I want to ask, because I need to explain why simple  logistic can be chosen.

SVM avoids overfitting using hyperplanes but I don't know if simple logistic does something to avoid overfitting.
And my second question is if simple logistic is linear logistic regression model or gets a linear model and fits it in a logistic regression model.

Any advice would be helpful because simple logistic is getting me confused  ???
Tagged:

Answers

  • kasper2304kasper2304 Member Posts: 28 Contributor II
    Hi goku.

    To decide what which method are the best one need to know a bit more about your dataset. So if you can please explain a bit more. I dont really know what you mean by "Linear" logistic regression. The name "log"istic regression implies that the regression is based on a line that comes from the logarithm rather than being linear. The only three regressions i ever heard about is "simple" linear regression = 1 DV vs 1 IV,  "multiple" regression = 1 DV vs. + 2 IV's and "logistic" regression = 1 categorical DV vs. + 1 IV.

    I eralise that my answer comes late but if you still need help i would provide you with some hints.

    Kasper
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869   Unicorn
    Hi,

    if you cross validated your results, then in your case probably the logistic regression works better than the svm... \

    Btw, in RapidMiner the logistic regression internally uses an SVM with a modified loss function. There is a paper which proves that both methods are equivalent. If you are interested, I can search for the title and the author of the paper.

    Best regards,
    Marius
  • earmijoearmijo Member Posts: 263   Unicorn
    Marius:

    Could you post the reference please?

    I have never quite understood what RM fits when one uses the Logistic Regression Operator.

    I'm a statistician and for me Logistic Regression is what the Weka operator W-Logistic fits.

    Thanks in advance,

    \E.
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869   Unicorn
    Hi E.,

    as the documentation of the Logistic Regression states, it is based on myKLR and the respective paper by Keerthi. Here you go:
    http://www-ai.cs.uni-dortmund.de/SOFTWARE/MYKLR/index.html
    http://link.springer.com/content/pdf/10.1007%2Fs10994-005-0768-5

    I have to admit that I have never read the paper, so I can't give you any details about its contents :)

    Best regards,
    Marius
Sign In or Register to comment.