RapidMiner 9.8 Beta is now available

Be one of the first to get your hands on the new features. More details and downloads here:

GET RAPIDMINER 9.8 BETA

ParameterOptimization of W-SMOreg kernel parameters

raveliteravelite Member Posts: 2 Contributor I
edited November 2018 in Help
Greetings!

I was recently working with SVMs for regression, and working specifically with the Weka class SMOreg. (somehow I haven't yet gotten the native SVRs to perform comparably...).

To properly select parameters for the regression (complexity, epsilon-width, and especially kernal parameters), I tried using the EvolutionaryParameterOptimizer. However, since the kernel parameters are contained in a string, they don't seem to show up in the parameter settings dialog for the Optimizer.

Is there a way to add the kernel parameters to this optimization?

cheers,

Graham

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531   Unicorn
    Hi Graham,
    this is not possible, but you simply could wrapp the Evolutionary Parameteroptimization with an outer GridParameterOptimization testing all three possible values.
    Non-Continoous values cannot be optimized properly by an evolutionary approach.

    Greetings,
      Sebastian
  • raveliteravelite Member Posts: 2 Contributor I
    Sebastian,

    Thanks for your reply.

    The parameter that I would wish to add to the optimization (for example, gamma from the RBFKernel function) is indeed a continuous parameter. It just doesn't show up because the kernel options for that learner (W-SMOreg) are represented all together as a string. But the option itself is continuous (say 1e-4 to 10?) meaning basically the size of the blob over each support vector.

    best,

    Graham
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531   Unicorn
    Hi Graham,
    thanks to  the weka guys' nearly cryptographic formulation of parameters this is not possible.
    But you should get similar performance if you use the LibSVM with svm_type epsilon_SVR or nu_SVR. As far as I know the SMOreg just uses another optimization algorithm but the same model. This should result in at least an equal performance with proper parameter settings.


    Greetings,
      Sebastian
  • Legacy UserLegacy User Member Posts: 0 Newbie
    Sebastian,

    Again, thanks for your explanation.

    Yet I haven't been able to get comparable results on SV Regression using the LibSVM models. I'm not sure what the problem is exactly yet.

    Perhaps I have a parameter miss-set. I'll try to open a new thread comparing my results with the libraries, since theoretically you would be using the same model (and thus could expect the same result) in both libraries.

    Graham
Sign In or Register to comment.