General guidelines to the use of Parameter Optimization for SVM.

pblack476

Certain learners have infinite possibilities of configuration and the task seems daunting.

So what are some guidelines on how to proceed with this step of model creation? Right now I am trying to optmize a SVM and while AutoModel gave me a starting point on that, it seems there is much more to test. How should I tackle this? What are usually effective parameters to tune in a SVM and are there recommended ranges for them?

Bear in mind that for my application I could leave a model running overnight without problems. But I am trying to understand the best I can the problem and try to avoid brute forcing it if possible.

I am on a 8750h CPU w/ 16 gb RAM

varunm1

Hello @pblack476

The three parameters I try to tune in SVM are kernel, C and gamma. Specific Range for C values is hard to inform. But very large C values might over fit the model and also takes more time for processing. You can try C 0 to 20 and check where you are getting best model if its at 20 try to increase C and see if you are getting best model above 20 as well. 

Incase of gamma, lower gamma will have high variance and high gamma will have low variance. You can search for gamma values as well similar to above.

Their maximum allowed ranges are given in SVM help text in RM

BalazsBarany

Hi,

there's also the recommendation to try C in logarithmic steps from e. g. 0.0001 to 100. 

Example with Optimize Parameters (Grid):
Min: 0.0001      Max: 100     Steps: 6          Scale: logarithmic

Changing C in these steps likely gives you a better information on the optimum value. But you should also try the default setting of 0, because that uses some heuristics to determine a good value. 

Regards,

Balázs