"SVM (libSVM) Missing Function Value"

B_MinerB_Miner Member Posts: 72 Contributor II
edited June 2019 in Help
Hi guys,

Is it possible to get the decision function value from your implementation of libSVM?
There is a field called 'function value' in the support vector table of the model but in the case I show below, it is all zero. Maybe this is not it though as we'd expect the function to be defined for all cases, not just the support vectors.


The decision function is:

f(x)= sgn(sum_over_all_i {a*yk(x_i,x) } ) where a* is the optimized Lagrangian multipliers, y is the observed target value and k() is the kernel).

Thanks!

Data and Code are below:


I used a small data set:

x1 x2 class
0 0.7 One
0.7 0 One
0 -0.7 One
-0.7 0 One
0.5 0.5 One
-0.5 0.5 One
-0.5 -0.5 One
0.5 -0.5 One
0 2.8 Two
-2.8 0 Two
0 -2.8 Two
2.8 0 Two
2 2 Two
-2 2 Two
-2 -2 Two
2 -2 Two


Here is the code I ran:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="341" width="547">
      <operator activated="true" class="retrieve" expanded="true" height="60" name="Retrieve" width="90" x="45" y="75">
        <parameter key="repository_entry" value="//Default/svmTest"/>
      </operator>
      <operator activated="true" class="nominal_to_binominal" expanded="true" height="94" name="Nominal to Binominal" width="90" x="246" y="30">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="class"/>
        <parameter key="include_special_attributes" value="true"/>
      </operator>
      <operator activated="true" class="support_vector_machine_libsvm" expanded="true" height="76" name="SVM (2)" width="90" x="45" y="210">
        <parameter key="kernel_type" value="poly"/>
        <parameter key="degree" value="2"/>
        <parameter key="gamma" value="1.0"/>
        <parameter key="C" value="10.0"/>
        <list key="class_weights"/>
        <parameter key="confidence_for_multiclass" value="false"/>
      </operator>
      <operator activated="true" class="apply_model" expanded="true" height="76" name="Apply Model" width="90" x="246" y="255">
        <list key="application_parameters"/>
      </operator>
      <operator activated="true" class="performance_classification" expanded="true" height="76" name="Performance" width="90" x="380" y="165">
        <list key="class_weights"/>
      </operator>
      <connect from_op="Retrieve" from_port="output" to_op="Nominal to Binominal" to_port="example set input"/>
      <connect from_op="Nominal to Binominal" from_port="example set output" to_op="SVM (2)" to_port="training set"/>
      <connect from_op="SVM (2)" from_port="model" to_op="Apply Model" to_port="model"/>
      <connect from_op="SVM (2)" from_port="exampleSet" to_op="Apply Model" to_port="unlabelled data"/>
      <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
      <connect from_op="Apply Model" from_port="model" to_port="result 1"/>
      <connect from_op="Performance" from_port="performance" to_port="result 2"/>
      <connect from_op="Performance" from_port="example set" to_port="result 3"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
    </process>
  </operator>



Tagged:

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    isn't that exactly the outcome the model applier will calculate if you apply the model?

    Greetings,
      Sebastian
  • B_MinerB_Miner Member Posts: 72 Contributor II
    Oops! Yes, that is the decision isnt it. :o I meant to leave off the sgn and just ask about getting access to what is inside:

    sum_over_all_i {a*yk(x_i,x) }

    I know that libsvm will give this information.
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Hi,

    isn't this exactly the thing the "Create Formula" operator performs?

    Cheers,
    Ingo
  • B_MinerB_Miner Member Posts: 72 Contributor II
    Perhaps, I was not famiiliar with this operator. Trying it out, it gives a strange result for the sample I posted:

    which is not correct (the result is close but does not match libsvm or hand calculations).

    The "correct answer" for the vector [2.8,0] is -1.0001512 where the below code will give 1.145649441

    So, I'm not sure....

      0.0 * (pow((1.0 * (-0.5 * x1 - 0.5 * x2) + 0.0), 2)) + 0.0 * (pow((1.0 * (0.5 * x1 - 0.5 * x2) + 0.0), 2)) + 0.037111961591350995 * (pow((1.0 * (-2.8 * x2) + 0.0), 2)) + 0.037121838508423564 * (pow((1.0 * (2.8 * x1) + 0.0), 2)) - 1.136066635903084
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Hi,

    the flipped sign could simply derive from the fact which class (One or Two) is internally mapped to -1 and +1. And the derivation between 1.000... and 1.145... could derive from different parameters, rescaling, shuffled data, ...  I would have to check the source code of LibSVM, of RapidMiner, your hand calculations and compare everything to come up with a definite answer.

    Cheers,
    Ingo
Sign In or Register to comment.