# "How are L pos and L neg set in an SVM?"

Member Posts: 1 Contributor I
edited June 2019 in Help
Noob here.  I am working on a basic classifcation problem (responders vs non-responders), where the cost of missing a responder is much higher than the cost of including a non-responder.  Specifically, the average sale is \$115 and the average cost to mail somebody is \$0.68.  We want to bias in favor of maximizing sales over minimizing cost to mail.

I can't find any documentation anywhere that shows how the "L pos" and "L neg" settings in an SVM are actually set.  Are they numbers between 0 and 1?  Are they relative weights for mis-classifying a positive "L pos" and a negative "L neg"?  In my case, I would interpret that as setting "L pos" much higher than "L neg", since I want to ensure that we avoid as many false negatives as possible.

Can somebody help me understand how this works?

Eric
Tagged:

• Member Posts: 4 Contributor I
I know this is a few months old, but I'm working on a very similar problem, so if anyone has any insight on this, that would be greatly appreciated.

Additionally, I'm still exploring classification methods for what I'm working on.  Is it possible to set similar loss criteria in other classification methods?  (False positives aren't too big of a deal, but false negatives are very costly--also, the test set includes very few responders compared to the number of non-responders, so a lot of methods that just look to improve accuracy without weighing false positives against false negatives classify almost everyone as a non-responder.)
• Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,453 RM Data Scientist
Hi,

i don't know what it is but i got eager to find it out.

So apprently it is used in /src/main/java/com/rapidminer/operator/learner/functions/kernel/jmysvm/svm/SVM.java

I figured out there that the balance cost option sets the same values.
`				double generalCpos = paramOperator.getParameterAsDouble(JMySVMLearner.PARAMETER_L_POS);				double generalCneg = paramOperator.getParameterAsDouble(JMySVMLearner.PARAMETER_L_NEG);				if (paramOperator.getParameterAsBoolean(JMySVMLearner.PARAMETER_BALANCE_COST)) {					generalCpos *= the_examples.count_examples()							/ (2.0d * (the_examples.count_examples() - the_examples.count_pos_examples()));					generalCneg *= ((the_examples.count_examples()) / (2.0d * the_examples.count_pos_examples()));				}				for (int i = 0; i < cPos.length; i++) {					cPos = generalCpos;					cNeg = generalCneg;`
So i did a quick test (process XML below). I classified golf with and without balance cost. Further i added weights to ensure class balance this way.

Weights w/o balance cost and w/o class balancing
`Temperature	-0.32830007919309123Outlook = sunny	-0.2990901852444362Humidity	-0.2759359389650189Wind = true	-0.23094010767584963Outlook = rain	0.0Wind = false	0.17320508075688723Outlook = overcast	0.35248116362113585`
w balance cost but w/o class balancing
`Outlook = sunny	-0.33391948463996746Humidity	-0.2776748983378322Wind = true	-0.25278262899833226Outlook = rain	-0.12521980673998773Temperature	-0.060810818269701084Wind = false	0.18958697174874925Outlook = overcast	0.5411008440732543`
and finallyw/ scaling but w/o balancing
`Outlook = sunny	-0.33391948463996746Humidity	-0.2776748983378322Wind = true	-0.25278262899833226Outlook = rain	-0.12521980673998773Temperature	-0.060810818269701084Wind = false	0.18958697174874925Outlook = overcast	0.5411008440732543`
So apprently the balancing here does the same thing than balancing via weights. Thus I expect LPos and LNeg to be class_weights.

Cheers,
Martin
`<?xml version="1.0" encoding="UTF-8" standalone="no"?><process version="6.5.002">  <context>    <input/>    <output/>    <macros/>  </context>  <operator activated="true" class="process" compatibility="6.5.002" expanded="true" name="Process">    <process expanded="true">      <operator activated="true" class="retrieve" compatibility="6.5.002" expanded="true" height="60" name="Retrieve Golf" width="90" x="45" y="120">        <parameter key="repository_entry" value="//Samples/data/Golf"/>      </operator>      <operator activated="true" class="nominal_to_numerical" compatibility="6.5.002" expanded="true" height="94" name="Nominal to Numerical" width="90" x="179" y="120">        <list key="comparison_groups"/>      </operator>      <operator activated="true" class="generate_weight_stratification" compatibility="6.5.002" expanded="true" height="76" name="Generate Weight (Stratification)" width="90" x="313" y="120">        <parameter key="total_weight" value="14.0"/>      </operator>      <operator activated="true" class="support_vector_machine" compatibility="6.5.002" expanded="true" height="112" name="SVM" width="90" x="514" y="120"/>      <connect from_op="Retrieve Golf" from_port="output" to_op="Nominal to Numerical" to_port="example set input"/>      <connect from_op="Nominal to Numerical" from_port="example set output" to_op="Generate Weight (Stratification)" to_port="example set input"/>      <connect from_op="Generate Weight (Stratification)" from_port="example set output" to_op="SVM" to_port="training set"/>      <connect from_op="SVM" from_port="weights" to_port="result 1"/>      <portSpacing port="source_input 1" spacing="0"/>      <portSpacing port="sink_result 1" spacing="0"/>      <portSpacing port="sink_result 2" spacing="0"/>    </process>  </operator></process>`
- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany
• RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 Unicorn
For other classifiers (& SVM too) another way to tackle this with different costs is to use cost based modelling.
Here's an example on how it works.  Please note, I chose to use Naive Bayes because it demonstrated the extemes well chosing to send to the entire dataset to 'maximise profit' (Have the least cost)

You can chose to either use the Meta Cost operator or use the Performance (Costs) operators separately.
Meta cost builds several models to favour the cost.
Performance (Costs) tells you the expected cost of the model.  (I tend to combine this one with an Optimise operator and then build my model parameters with the ones that decrease the cost the most).
`<?xml version="1.0" encoding="UTF-8" standalone="no"?><process version="7.0.000">  <context>    <input/>    <output/>    <macros/>  </context>  <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process">    <process expanded="true">      <operator activated="true" class="generate_direct_mailing_data" compatibility="7.0.000" expanded="true" height="68" name="Generate Direct Mailing Data" width="90" x="45" y="34">        <parameter key="number_examples" value="100000"/>      </operator>      <operator activated="true" class="multiply" compatibility="7.0.000" expanded="true" height="124" name="Multiply" width="90" x="112" y="136"/>      <operator activated="true" class="split_validation" compatibility="7.0.000" expanded="true" height="124" name="Validation" width="90" x="246" y="34">        <process expanded="true">          <operator activated="true" class="metacost" compatibility="7.0.000" expanded="true" height="82" name="MetaCost" width="90" x="45" y="34">            <parameter key="cost_matrix" value="[0.0 115.0;0.68 -115.0]"/>            <process expanded="true">              <operator activated="true" class="naive_bayes" compatibility="7.0.000" expanded="true" height="82" name="Naive Bayes" width="90" x="179" y="136"/>              <connect from_port="training set" to_op="Naive Bayes" to_port="training set"/>              <connect from_op="Naive Bayes" from_port="model" to_port="model"/>              <portSpacing port="source_training set" spacing="0"/>              <portSpacing port="sink_model" spacing="0"/>            </process>          </operator>          <connect from_port="training" to_op="MetaCost" to_port="training set"/>          <connect from_op="MetaCost" from_port="model" to_port="model"/>          <portSpacing port="source_training" spacing="0"/>          <portSpacing port="sink_model" spacing="0"/>          <portSpacing port="sink_through 1" spacing="0"/>        </process>        <process expanded="true">          <operator activated="true" class="apply_model" compatibility="7.0.000" expanded="true" height="82" name="Apply Model" width="90" x="45" y="30">            <list key="application_parameters"/>          </operator>          <operator activated="true" class="performance_costs" compatibility="7.0.000" expanded="true" height="82" name="Performance (Cost Meta)" width="90" x="112" y="187">            <parameter key="cost_matrix" value="[0.0 115.0;0.68 -115.0]"/>            <enumeration key="class_order_definition"/>          </operator>          <operator activated="true" class="performance_classification" compatibility="7.0.000" expanded="true" height="82" name="Performance" width="90" x="179" y="30">            <list key="class_weights"/>          </operator>          <connect from_port="model" to_op="Apply Model" to_port="model"/>          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance (Cost Meta)" to_port="example set"/>          <connect from_op="Performance (Cost Meta)" from_port="example set" to_op="Performance" to_port="labelled data"/>          <connect from_op="Performance (Cost Meta)" from_port="performance" to_op="Performance" to_port="performance"/>          <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>          <portSpacing port="source_model" spacing="0"/>          <portSpacing port="source_test set" spacing="0"/>          <portSpacing port="source_through 1" spacing="0"/>          <portSpacing port="sink_averagable 1" spacing="0"/>          <portSpacing port="sink_averagable 2" spacing="0"/>        </process>      </operator>      <operator activated="true" class="create_lift_chart" compatibility="7.0.000" expanded="true" height="103" name="Create Lift Chart" width="90" x="447" y="289">        <parameter key="target_class" value="response"/>      </operator>      <operator activated="true" class="split_validation" compatibility="7.0.000" expanded="true" height="124" name="Validation (2)" width="90" x="246" y="187">        <process expanded="true">          <operator activated="true" class="naive_bayes" compatibility="7.0.000" expanded="true" height="82" name="Naive Bayes (2)" width="90" x="45" y="34"/>          <connect from_port="training" to_op="Naive Bayes (2)" to_port="training set"/>          <connect from_op="Naive Bayes (2)" from_port="model" to_port="model"/>          <portSpacing port="source_training" spacing="0"/>          <portSpacing port="sink_model" spacing="0"/>          <portSpacing port="sink_through 1" spacing="0"/>        </process>        <process expanded="true">          <operator activated="true" class="apply_model" compatibility="7.0.000" expanded="true" height="82" name="Apply Model (2)" width="90" x="45" y="30">            <list key="application_parameters"/>          </operator>          <operator activated="true" class="performance_costs" compatibility="7.0.000" expanded="true" height="82" name="Performance (Cost Without Meta)" width="90" x="45" y="238">            <parameter key="cost_matrix" value="[0.0 115.0;0.68 -115.0]"/>            <enumeration key="class_order_definition"/>          </operator>          <operator activated="true" class="performance_classification" compatibility="7.0.000" expanded="true" height="82" name="Performance (2)" width="90" x="179" y="30">            <list key="class_weights"/>          </operator>          <connect from_port="model" to_op="Apply Model (2)" to_port="model"/>          <connect from_port="test set" to_op="Apply Model (2)" to_port="unlabelled data"/>          <connect from_op="Apply Model (2)" from_port="labelled data" to_op="Performance (Cost Without Meta)" to_port="example set"/>          <connect from_op="Performance (Cost Without Meta)" from_port="example set" to_op="Performance (2)" to_port="labelled data"/>          <connect from_op="Performance (Cost Without Meta)" from_port="performance" to_op="Performance (2)" to_port="performance"/>          <connect from_op="Performance (2)" from_port="performance" to_port="averagable 1"/>          <portSpacing port="source_model" spacing="0"/>          <portSpacing port="source_test set" spacing="0"/>          <portSpacing port="source_through 1" spacing="0"/>          <portSpacing port="sink_averagable 1" spacing="0"/>          <portSpacing port="sink_averagable 2" spacing="0"/>        </process>      </operator>      <operator activated="true" class="subprocess" compatibility="7.0.000" expanded="true" height="124" name="Just playing around" width="90" x="380" y="85">        <process expanded="true">          <operator activated="true" class="performance_to_data" compatibility="7.0.000" expanded="true" height="82" name="Performance to Data" width="90" x="45" y="34"/>          <operator activated="true" class="performance_to_data" compatibility="7.0.000" expanded="true" height="82" name="Performance to Data (2)" width="90" x="45" y="238"/>          <operator activated="true" class="extract_macro" compatibility="7.0.000" expanded="true" height="68" name="Extract Macro" width="90" x="179" y="136">            <parameter key="macro" value="CostWith"/>            <parameter key="macro_type" value="data_value"/>            <parameter key="attribute_name" value="Value"/>            <parameter key="example_index" value="1"/>            <list key="additional_macros"/>          </operator>          <operator activated="true" class="extract_macro" compatibility="7.0.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="179" y="238">            <parameter key="macro" value="CostWithout"/>            <parameter key="macro_type" value="data_value"/>            <parameter key="attribute_name" value="Value"/>            <parameter key="example_index" value="1"/>            <list key="additional_macros"/>          </operator>          <operator activated="true" class="generate_data_user_specification" compatibility="7.0.000" expanded="true" height="68" name="Gen Data" width="90" x="313" y="289">            <list key="attribute_values">              <parameter key="CostDifference" value="parse(%{CostWith})-parse(%{CostWithout})"/>            </list>            <list key="set_additional_roles"/>          </operator>          <operator activated="true" class="format_numbers" compatibility="7.0.000" expanded="true" height="82" name="Saving by using meta model" width="90" x="447" y="238">            <parameter key="format_type" value="currency"/>            <parameter key="locale" value="English (United Kingdom)"/>            <parameter key="use_grouping" value="true"/>          </operator>          <connect from_port="in 1" to_op="Performance to Data" to_port="performance vector"/>          <connect from_port="in 2" to_op="Performance to Data (2)" to_port="performance vector"/>          <connect from_op="Performance to Data" from_port="example set" to_op="Extract Macro" to_port="example set"/>          <connect from_op="Performance to Data" from_port="performance vector" to_port="out 2"/>          <connect from_op="Performance to Data (2)" from_port="example set" to_op="Extract Macro (2)" to_port="example set"/>          <connect from_op="Performance to Data (2)" from_port="performance vector" to_port="out 3"/>          <connect from_op="Gen Data" from_port="output" to_op="Saving by using meta model" to_port="example set input"/>          <connect from_op="Saving by using meta model" from_port="example set output" to_port="out 1"/>          <portSpacing port="source_in 1" spacing="0"/>          <portSpacing port="source_in 2" spacing="0"/>          <portSpacing port="source_in 3" spacing="0"/>          <portSpacing port="sink_out 1" spacing="0"/>          <portSpacing port="sink_out 2" spacing="0"/>          <portSpacing port="sink_out 3" spacing="0"/>          <portSpacing port="sink_out 4" spacing="0"/>        </process>      </operator>      <connect from_op="Generate Direct Mailing Data" from_port="output" to_op="Multiply" to_port="input"/>      <connect from_op="Multiply" from_port="output 1" to_op="Validation" to_port="training"/>      <connect from_op="Multiply" from_port="output 2" to_op="Validation (2)" to_port="training"/>      <connect from_op="Multiply" from_port="output 3" to_op="Create Lift Chart" to_port="example set"/>      <connect from_op="Validation" from_port="model" to_op="Create Lift Chart" to_port="model"/>      <connect from_op="Validation" from_port="averagable 1" to_op="Just playing around" to_port="in 1"/>      <connect from_op="Create Lift Chart" from_port="model" to_port="result 5"/>      <connect from_op="Create Lift Chart" from_port="lift pareto chart" to_port="result 6"/>      <connect from_op="Validation (2)" from_port="model" to_port="result 1"/>      <connect from_op="Validation (2)" from_port="averagable 1" to_op="Just playing around" to_port="in 2"/>      <connect from_op="Just playing around" from_port="out 1" to_port="result 2"/>      <connect from_op="Just playing around" from_port="out 2" to_port="result 3"/>      <connect from_op="Just playing around" from_port="out 3" to_port="result 4"/>      <portSpacing port="source_input 1" spacing="0"/>      <portSpacing port="sink_result 1" spacing="84"/>      <portSpacing port="sink_result 2" spacing="0"/>      <portSpacing port="sink_result 3" spacing="0"/>      <portSpacing port="sink_result 4" spacing="0"/>      <portSpacing port="sink_result 5" spacing="0"/>      <portSpacing port="sink_result 6" spacing="0"/>      <portSpacing port="sink_result 7" spacing="0"/>    </process>  </operator></process>`
• Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,453 RM Data Scientist
While i totally support you i would like to add, that in a best case scenario you can give each example (e.g. customer) a cost (or revenue). Than you can aggregate on this and optimize on the costs. This is very similar but even more detailed. I think this is a better way - if the data is available.

~Martin
- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany