Options

# Speculative Rounds In Forward Selection

Member Posts: 23 Contributor II
edited November 2018 in Help
Hello all,

I've got a short question:
What feature/attribue chooses the Forward Selection Operator in a speculative round?

Let's assume we have 5 features/attributes, "without increase" - stopping criterium, maximal number of attributes set to 5 and speculative rounds = 2.

1.Round :
- att1 = 70 %
- att2 = 60 %
- att3 = 50 %
- att4 = 45 %
- att5 = 80 %
=> attribute 5 will be chosen

2.Round :
- (att5 + att1) = 60 %
- (att5 + att2) = 70 %
- (att5 + att3) = 80 %
- (att5 + att4) = 90 %
=> attribute 4 will be chosen

3.Round :
- (att5 + att4 + att1) = 70 %
- (att5 + att4 + att2) = 60 %
- (att5 + att4 + att3) = 50 %
=> No increase => speculative round

So here's my question again: what attribute will now be chosen?
And what will happen next?

Sorry if this is a stupid question but I didn't find any answer neither here in this forum nor with google.

Sasch

• Options
RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
Hi Sasch,

actually, your third round is *not* a speculative round - the Forward Selection always adds one attribute and looks for increases. In your case, the best result was obtained in round 2. Since round 3 does not deliver a better result, the algorithm stops and delivers the best results found, i.e. att5+att4 from round 2.

If you configure one speculative round, the Forward Selection would try a 4th round, even though no increase could be achieved in round 3.

In any case, the attributes that delivered the best performance are returned.

Best regards,
Marius
• Options
RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
Experiment with this process to get a better feeling for what is happening:
`<?xml version="1.0" encoding="UTF-8" standalone="no"?><process version="5.3.013">  <context>    <input/>    <output/>    <macros/>  </context>  <operator activated="true" class="process" compatibility="5.3.013" expanded="true" name="Process">    <process expanded="true">      <operator activated="true" class="retrieve" compatibility="5.3.013" expanded="true" height="60" name="Retrieve Iris" width="90" x="45" y="30">        <parameter key="repository_entry" value="//Samples/data/Iris"/>      </operator>      <operator activated="true" class="optimize_selection_forward" compatibility="5.3.013" expanded="true" height="94" name="Forward Selection" width="90" x="179" y="30">        <process expanded="true">          <operator activated="true" class="x_validation" compatibility="5.3.013" expanded="true" height="112" name="Validation" width="90" x="45" y="30">            <description>A cross-validation evaluating a decision tree model.</description>            <process expanded="true">              <operator activated="true" class="decision_tree" compatibility="5.3.013" expanded="true" height="76" name="Decision Tree" width="90" x="45" y="30"/>              <connect from_port="training" to_op="Decision Tree" to_port="training set"/>              <connect from_op="Decision Tree" from_port="model" to_port="model"/>              <portSpacing port="source_training" spacing="0"/>              <portSpacing port="sink_model" spacing="0"/>              <portSpacing port="sink_through 1" spacing="0"/>            </process>            <process expanded="true">              <operator activated="true" class="apply_model" compatibility="5.3.013" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">                <list key="application_parameters"/>              </operator>              <operator activated="true" class="performance" compatibility="5.3.013" expanded="true" height="76" name="Performance" width="90" x="179" y="30"/>              <connect from_port="model" to_op="Apply Model" to_port="model"/>              <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>              <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>              <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>              <portSpacing port="source_model" spacing="0"/>              <portSpacing port="source_test set" spacing="0"/>              <portSpacing port="source_through 1" spacing="0"/>              <portSpacing port="sink_averagable 1" spacing="0"/>              <portSpacing port="sink_averagable 2" spacing="0"/>            </process>          </operator>          <operator activated="true" class="log" compatibility="5.3.013" expanded="true" height="76" name="Log" width="90" x="246" y="30">            <list key="log">              <parameter key="performance" value="operator.Validation.value.performance"/>              <parameter key="attributes" value="operator.Forward Selection.value.feature_names"/>            </list>          </operator>          <connect from_port="example set" to_op="Validation" to_port="training"/>          <connect from_op="Validation" from_port="averagable 1" to_op="Log" to_port="through 1"/>          <connect from_op="Log" from_port="through 1" to_port="performance"/>          <portSpacing port="source_example set" spacing="0"/>          <portSpacing port="sink_performance" spacing="0"/>        </process>      </operator>      <connect from_op="Retrieve Iris" from_port="output" to_op="Forward Selection" to_port="example set"/>      <connect from_op="Forward Selection" from_port="attribute weights" to_port="result 1"/>      <portSpacing port="source_input 1" spacing="0"/>      <portSpacing port="sink_result 1" spacing="0"/>      <portSpacing port="sink_result 2" spacing="0"/>    </process>  </operator></process>`
• Options
Member Posts: 23 Contributor II
Hello Marius,

I understood everything so far but I'm still interested in what attribute the Forward Selection picks in round 3
if I configure one speculative round?
=>
3.Round :
- (att5 + att4 + att1) = 70 %
- (att5 + att4 + att2) = 60 %
- (att5 + att4 + att3) = 50 %
=> No increase => speculative round

Will it be att1 because it has the highest rate?

Do have any links to some literature about Forward Selection with speculative rounds?

Thanks again,
Sasch