LibSVM Parameter Gamma = Sigma?
hi,
I was just wondering, what is the formula used for the RBF Kernel, is it the first or second (GaussRBF) formula used from here:
https://en.wikipedia.org/wiki/Radial_basis_function_kernel
because I noticed, when I set parameter Gamma, the training data will usually overfit if my gamma is chosen very small, but in other literatures, I read that if u make gamma big enough, the training point influence will get closer to each point, which means it overfits...
but I guess, as my data gets overfit if gamma is smaller, what actually is meant by gamma in the LibSVM Parameter is Sigma... and probably the formula: gamma = 1/(2*sigma^2) was used, is that correct?
Best Answer

Fred12 Member Posts: 344 Unicorn
yes I did, it is: K(xi , xj ) = exp(−γ*_xi − x_j^2 ) where gamma : y=1/(2*sigma^2)
so I guess I was wrong, a smaller gamma then actually means a bigger sigma, which means broader influence of data points and smoothed (less hard) decision boundaries, and otherwise a bigger gamma means smaller sigma in the formula and therefore smaller influence of a data point such that only those support vectors very close to the decision boundary are considered and therefore a bigger chance for overfitting the training data...
I was just wondering because I got better scores with usually smaller gamma, so I thought that overfitted my data but its actually the other way around
1
Answers
Did you check out http://www.csie.ntu.edu.tw/~cjlin/libsvm to see how they implemented the RBF kernel?
This is a great graphic on the trade off between gamma and C. You can see how they influence the data.
http://community.rapidminer.com/t5/RapidMinerStudioForum/FinancialTimeSeriesPrediction/mp/33456?lightboxmessageimages33515=551i694A2F729EAC22A8