GLM: weights vs. coefficients

kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 280   Unicorn
edited July 29 in Help
Hi miners, 

I am training GLM model for binary classification, so basically I perform logistic regression. 

My question is, how do I interpret the relation between GLM model weights output and regression coefficients?

In many cases, they are exactly the same, but some differ, and some on a very high magnitude. For example, for one feature weight and regression coefficient both equal 1.841; then for another feature I observe weight 0.328 while regression coefficient is 0.0002; yet for another feature weight is -0.617 and coefficient is -0.001.

(I use regularisation so the whole coefficients / weights range is not that big, let's say roughly between 2 and -2).
Tghadially

Best Answers

Answers

  • kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 280   Unicorn
    Hi @Telcontar120

    Thanks for the example; it makes a clear point, however this is not exactly what I was asking for. I am not trying to compare GLM and LR, but actually I have just one GLM model where I am comparing model coefficients with feature weights. I think Ingo's answer cleared it pretty well. 
    Tghadially
  • kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 280   Unicorn
    Hi @IngoRM

    Thanks for an advise, I was looking at the first column of coefficients (not standardized). In fact, std. coefficients and weights from GLM weights output port are the same, so I have my question answered.

    However, I have now the second question: if I use derived coefficients for a regression equation (which for example I then put into code to make predictions on new data), should I actually use normal or standardized coefficients, or it won't make a difference? What I exactly mean, I am using the following formulas to calculate probability on new data: 
    p = exp(y*)/(1 + exp(y*)), 
    where y* = Log ( p / (1-p) ) = b0 + b1*x1 + b2*x2 + ... + bk*xk

    Tghadially
  • kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 280   Unicorn
    Thanks @Telcontar120 - pretty clear. 
    Tghadially
Sign In or Register to comment.