LibSVM Parameter Gamma = Sigma?

Fred12

hi,

I was just wondering,  what is the formula used for the RBF Kernel, is it the first or second (Gauss-RBF) formula used from here:

https://en.wikipedia.org/wiki/Radial_basis_function_kernel

 

 because I noticed, when I set parameter Gamma, the training data will usually overfit if my gamma is chosen very small, but in other literatures, I read that if u make gamma big enough, the training point influence will get closer to each point, which means it overfits...

 

but I guess, as my data gets overfit if gamma is smaller, what actually is meant by gamma in the LibSVM Parameter is Sigma... and probably the formula:   gamma = 1/(2*sigma^2) was used, is that correct?

 

Fred12

yes I did, it is: K(xi , xj ) = exp(−γ*||_xi − x_j||^2 ) where gamma :  y=1/(2*sigma^2)

 

so I guess I was wrong, a smaller gamma then actually means a bigger sigma, which means broader influence of data points and smoothed (less hard) decision boundaries, and otherwise a bigger gamma means smaller sigma in the formula and therefore smaller influence of a data point such that only those support vectors very close to the decision boundary are considered and therefore a bigger chance for overfitting the training data...

 

I was just wondering because I got better scores with usually smaller gamma, so I thought that overfitted my data but its actually the other way around

Thomas_Ott

Did you check out http://www.csie.ntu.edu.tw/~cjlin/libsvm to see how they implemented the RBF kernel?

Thomas_Ott

This is a great graphic on the trade off between gamma and C. You can see how they influence the data. 

 

http://community.rapidminer.com/t5/RapidMiner-Studio-Forum/Financial-Time-Series-Prediction/m-p/33456?lightbox-message-images-33515=551i694A2F729EAC22A8