"SVM vs Logistic Regression"

gokugoku Member Posts: 4 Contributor I
edited June 2019 in Help
I'd like some help.
I'm performing document classifiaction using SVM and Simple Logistic and I want to choose which one is better for my case.
From what I see, SVM model is build faster than simple logistic, but simple logistic seems to have better precision and recall than SVM.
So, I have some silly questions that I want to ask, because I need to explain why simple  logistic can be chosen.

SVM avoids overfitting using hyperplanes but I don't know if simple logistic does something to avoid overfitting.
And my second question is if simple logistic is linear logistic regression model or gets a linear model and fits it in a logistic regression model.

Any advice would be helpful because simple logistic is getting me confused  ???
Tagged:

Answers

  • kasper2304kasper2304 Member Posts: 28 Contributor II
    Hi goku.

    To decide what which method are the best one need to know a bit more about your dataset. So if you can please explain a bit more. I dont really know what you mean by "Linear" logistic regression. The name "log"istic regression implies that the regression is based on a line that comes from the logarithm rather than being linear. The only three regressions i ever heard about is "simple" linear regression = 1 DV vs 1 IV,  "multiple" regression = 1 DV vs. + 2 IV's and "logistic" regression = 1 categorical DV vs. + 1 IV.

    I eralise that my answer comes late but if you still need help i would provide you with some hints.

    Kasper
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi,

    if you cross validated your results, then in your case probably the logistic regression works better than the svm... \

    Btw, in RapidMiner the logistic regression internally uses an SVM with a modified loss function. There is a paper which proves that both methods are equivalent. If you are interested, I can search for the title and the author of the paper.

    Best regards,
    Marius
  • earmijoearmijo Member Posts: 270 Unicorn
    Marius:

    Could you post the reference please?

    I have never quite understood what RM fits when one uses the Logistic Regression Operator.

    I'm a statistician and for me Logistic Regression is what the Weka operator W-Logistic fits.

    Thanks in advance,

    \E.
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi E.,

    as the documentation of the Logistic Regression states, it is based on myKLR and the respective paper by Keerthi. Here you go:
    http://www-ai.cs.uni-dortmund.de/SOFTWARE/MYKLR/index.html
    http://link.springer.com/content/pdf/10.1007%2Fs10994-005-0768-5

    I have to admit that I have never read the paper, so I can't give you any details about its contents :)

    Best regards,
    Marius
Sign In or Register to comment.