RapidMiner 9.8 Beta is now available

Be one of the first to get your hands on the new features. More details and downloads here:

GET RAPIDMINER 9.8 BETA

LibSVM Parameter Gamma = Sigma?

Fred12Fred12 Member Posts: 344   Unicorn
edited November 2018 in Help

hi,

I was just wondering,  what is the formula used for the RBF Kernel, is it the first or second (Gauss-RBF) formula used from here:

https://en.wikipedia.org/wiki/Radial_basis_function_kernel

 

 because I noticed, when I set parameter Gamma, the training data will usually overfit if my gamma is chosen very small, but in other literatures, I read that if u make gamma big enough, the training point influence will get closer to each point, which means it overfits...

 

but I guess, as my data gets overfit if gamma is smaller, what actually is meant by gamma in the LibSVM Parameter is Sigma... and probably the formula:   gamma = 1/(2*sigma^2) was used, is that correct?

 

Tagged:

Best Answer

  • Fred12Fred12 Member Posts: 344   Unicorn
    Solution Accepted

    yes I did, it is: K(xi , xj ) = exp(−γ*||_xi − x_j||^2 ) where gamma :  y=1/(2*sigma^2)

     

    so I guess I was wrong, a smaller gamma then actually means a bigger sigma, which means broader influence of data points and smoothed (less hard) decision boundaries, and otherwise a bigger gamma means smaller sigma in the formula and therefore smaller influence of a data point such that only those support vectors very close to the decision boundary are considered and therefore a bigger chance for overfitting the training data...

     

    I was just wondering because I got better scores with usually smaller gamma, so I thought that overfitted my data but its actually the other way around

    Thomas_Ott

Answers

Sign In or Register to comment.