Window Slider Validation Model Problem

Cleo

Hello,

I have a process which has a Sliding Window Validation operator (training window width of 30) and within the training area I support vector machine learner connect to the .  The SVM operator  is connect to the  “tra” and “mod” ports.  On the testing I have an Apply Model operator and a write model operator.  Also I have a write model operator (writing to a different location then the one connected to the SVM), connected to the “mod” port on the Sliding Window Validation.

My problem is when I use any combination of the models written on the exact same data used in the Testing area of the Sliding Window Validation operator I get very different results.

Do I have to preprocess the data before it goes it connects to the model, and if so how?

Within the documentation under parameters it is noted that:
create complete model
Indicates if a model of the complete data set should be additionally build after estimation.
Default value: false

I not find a check box anywhere to set its value to true.  However the model writer operator connected to the “mod” port of the sliding window validation operator is createing a file. 

Any suggestions?

Thanks in advance,
Cleo

land

Hi Cleo,
the parameter has been removed, since you now select by connecting the port of the complete model if it should be built. Unfortunately we didn't manage to update all documentation yet. There are soooo many operators, which documentation needs to be rewritten...
I must admit, that I didn't understand your setting and your problem. Could you please post a process and describe in detail what you are going to achieve with it?

Greetings,
  Sebastian

Cleo

Thank you for the offer of looking over my process.

When you run the process Sliding Window Trainer: you should notice two different performance vectors, but I was expecting them to be equal.  When I hover over the “ave” port on the Validation operator rapidminer indicates that it is: Validation.averagable 1 ← Write Performance.through ← Performance.performance

However when I read the performance vector I get a different result.  It seems to me that the “ave” port is outputting the performance of the training vector and not the testing vector. 1) How do I get the performance vector from the test portion of the validation?

In the Sliding window Trainer, I added two model writers, attached to the validation mod port and the second attached to the SVM mod port.  In the sliding window reader, I load the testing data, then multiply the data, and run it through both of the models.  The validation mod results gives me the actual label, while the SVM model gives me the same results as the exa from the performance operator within the trainer.  2)Therefore should I use the SVM model for future data?

3)If possible could you please add any preprocessing operator to the process?  I have a disabled attempt but cannot seem to get it to work correctly.

Thanks again,
Cleo


Sliding Training
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input>
      <location/>
    </input>
    <output>
      <location/>
      <location/>
      <location/>
    </output>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="302" width="975">
      <operator activated="true" class="read_excel" expanded="true" height="60" name="Read Excel" width="90" x="45" y="30">
        <parameter key="excel_file" value="C:\Projects\RM5\Test\raw_data.xls"/>
      </operator>
      <operator activated="true" class="set_role" expanded="true" height="76" name="Set ID" width="90" x="180" y="30">
        <parameter key="name" value="id"/>
        <parameter key="target_role" value="TheID"/>
      </operator>
      <operator activated="true" class="set_role" expanded="true" height="76" name="Set Label" width="90" x="315" y="30">
        <parameter key="name" value="Label"/>
        <parameter key="target_role" value="label"/>
      </operator>
      <operator activated="true" class="set_role" expanded="true" height="76" name="Set Date" width="90" x="447" y="30">
        <parameter key="name" value="date"/>
        <parameter key="target_role" value="date"/>
      </operator>
      <operator activated="true" class="series:sliding_window_validation" expanded="true" height="112" name="Validation" width="90" x="581" y="30">
        <parameter key="training_window_width" value="50"/>
        <parameter key="test_window_width" value="50"/>
        <process expanded="true" height="302" width="279">
          <operator activated="false" class="series:process_series" expanded="true" height="76" name="ValueSeriesPreprocessing" width="90" x="45" y="120">
            <parameter key="start_attribute" value="1"/>
            <parameter key="end_attribute" value="9"/>
            <process expanded="true" height="302" width="443">
              <operator activated="false" class="series:normalize" expanded="true" height="60" name="SeriesNormalization" width="90" x="112" y="30"/>
              <connect from_port="series" to_op="SeriesNormalization" to_port="series"/>
              <connect from_op="SeriesNormalization" from_port="series" to_port="series"/>
              <portSpacing port="source_series" spacing="0"/>
              <portSpacing port="source_input 1" spacing="0"/>
              <portSpacing port="sink_series" spacing="0"/>
            </process>
          </operator>
          <operator activated="true" class="support_vector_machine" expanded="true" height="112" name="SVM_Radial" width="90" x="179" y="30">
            <parameter key="kernel_type" value="radial"/>
            <parameter key="C" value="4847.123204531062"/>
          </operator>
          <connect from_port="training" to_op="SVM_Radial" to_port="training set"/>
          <connect from_op="SVM_Radial" from_port="model" to_port="model"/>
          <portSpacing port="source_training" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true" height="460" width="532">
          <operator activated="true" class="apply_model" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
            <list key="application_parameters"/>
          </operator>
          <operator activated="true" class="write_model" expanded="true" height="60" name="Write Model" width="90" x="246" y="30">
            <parameter key="model_file" value="C:\Projects\RM5\Test\SVM_model.mod"/>
          </operator>
          <operator activated="true" class="performance_binominal_classification" expanded="true" height="76" name="Performance" width="90" x="45" y="210">
            <parameter key="main_criterion" value="true_positive"/>
          </operator>
          <operator activated="true" class="write_performance" expanded="true" height="60" name="Write Performance" width="90" x="179" y="165">
            <parameter key="performance_file" value="C:\Projects\RM5\Test\performance.per"/>
          </operator>
          <operator activated="true" class="write_excel" expanded="true" height="60" name="Write Excel (2)" width="90" x="45" y="390">
            <parameter key="excel_file" value="C:\Projects\RM5\Test\Training_output2.xls"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Apply Model" from_port="model" to_op="Write Model" to_port="input"/>
          <connect from_op="Performance" from_port="performance" to_op="Write Performance" to_port="input"/>
          <connect from_op="Performance" from_port="example set" to_op="Write Excel (2)" to_port="input"/>
          <connect from_op="Write Performance" from_port="through" to_port="averagable 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_averagable 1" spacing="0"/>
          <portSpacing port="sink_averagable 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="write_model" expanded="true" height="60" name="Write Model Validation" width="90" x="720" y="30">
        <parameter key="model_file" value="C:\Projects\RM5\Test\Validation.mod"/>
      </operator>
      <operator activated="true" class="read_performance" expanded="true" height="60" name="Read Performance" width="90" x="715" y="165">
        <parameter key="performance_file" value="C:\Projects\RM5\Test\performance.per"/>
      </operator>
      <connect from_op="Read Excel" from_port="output" to_op="Set ID" to_port="example set input"/>
      <connect from_op="Set ID" from_port="example set output" to_op="Set Label" to_port="example set input"/>
      <connect from_op="Set Label" from_port="example set output" to_op="Set Date" to_port="example set input"/>
      <connect from_op="Set Date" from_port="example set output" to_op="Validation" to_port="training"/>
      <connect from_op="Validation" from_port="model" to_op="Write Model Validation" to_port="input"/>
      <connect from_op="Validation" from_port="averagable 1" to_port="result 1"/>
      <connect from_op="Read Performance" from_port="output" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>







Reader:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.0">
  <context>
    <input>
      <location/>
    </input>
    <output>
      <location/>
    </output>
    <macros/>
  </context>
  <operator activated="true" class="process" expanded="true" name="Process">
    <process expanded="true" height="325" width="947">
      <operator activated="true" class="read_excel" expanded="true" height="60" name="Read Excel" width="90" x="45" y="30">
        <parameter key="excel_file" value="C:\Projects\RM5\Test\raw_data.xls"/>
        <parameter key="sheet_number" value="2"/>
      </operator>
      <operator activated="true" class="set_role" expanded="true" height="76" name="Set ID" width="90" x="180" y="30">
        <parameter key="name" value="id"/>
        <parameter key="target_role" value="TheID"/>
      </operator>
      <operator activated="true" class="read_model" expanded="true" height="60" name="Read Model SVM" width="90" x="447" y="255">
        <parameter key="model_file" value="C:\Projects\RM5\Test\SVM_model.mod"/>
      </operator>
      <operator activated="true" class="read_model" expanded="true" height="60" name="Read Model Validation" width="90" x="447" y="165">
        <parameter key="model_file" value="C:\Projects\RM5\Test\Validation.mod"/>
      </operator>
      <operator activated="true" class="set_role" expanded="true" height="76" name="Set Date" width="90" x="315" y="30">
        <parameter key="name" value="date"/>
        <parameter key="target_role" value="date"/>
      </operator>
      <operator activated="true" class="multiply" expanded="true" height="94" name="Multiply" width="90" x="447" y="30"/>
      <operator activated="true" class="apply_model" expanded="true" height="76" name="SVM" width="90" x="648" y="210">
        <list key="application_parameters"/>
      </operator>
      <operator activated="true" class="write_excel" expanded="true" height="60" name="Write Excel (2)" width="90" x="782" y="210">
        <parameter key="excel_file" value="C:\Projects\RM5\Test\SVM.xls"/>
      </operator>
      <operator activated="true" class="apply_model" expanded="true" height="76" name="Validation" width="90" x="648" y="30">
        <list key="application_parameters"/>
      </operator>
      <operator activated="true" class="write_excel" expanded="true" height="60" name="Write Excel" width="90" x="782" y="30">
        <parameter key="excel_file" value="C:\Projects\RM5\Test\validation.xls"/>
      </operator>
      <connect from_op="Read Excel" from_port="output" to_op="Set ID" to_port="example set input"/>
      <connect from_op="Set ID" from_port="example set output" to_op="Set Date" to_port="example set input"/>
      <connect from_op="Read Model SVM" from_port="output" to_op="SVM" to_port="model"/>
      <connect from_op="Read Model Validation" from_port="output" to_op="Validation" to_port="model"/>
      <connect from_op="Set Date" from_port="example set output" to_op="Multiply" to_port="input"/>
      <connect from_op="Multiply" from_port="output 1" to_op="Validation" to_port="unlabelled data"/>
      <connect from_op="Multiply" from_port="output 2" to_op="SVM" to_port="unlabelled data"/>
      <connect from_op="SVM" from_port="labelled data" to_op="Write Excel (2)" to_port="input"/>
      <connect from_op="Validation" from_port="labelled data" to_op="Write Excel" to_port="input"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

Cleo

Raw Data:
Trainer:
id date 1 5 6 7 8 9 Label
1 2009.04.05 00:00:00 148.8 8169.0 2.7 141.8 104.9 95.7 false
2 2009.04.06 00:00:00 150.5 46581.0 2.8 142.5 103.4 87.2 false
3 2009.04.07 00:00:00 147.4 48491.0 2.8 143.0 103.2 71.0 false
4 2009.04.08 00:00:00 146.9 41437.0 2.7 143.4 104.9 47.9 false
5 2009.04.09 00:00:00 146.9 39267.0 2.7 143.8 105.5 32.0 false
6 2009.04.10 00:00:00 146.9 7423.0 2.6 144.1 104.4 21.4 true
7 2009.04.12 00:00:00 146.9 4208.0 2.5 144.4 104.2 24.8 false
8 2009.04.13 00:00:00 147.0 25574.0 2.5 144.7 103.6 40.0 true
9 2009.04.14 00:00:00 148.3 43028.0 2.3 145.0 99.7 39.6 true
10 2009.04.15 00:00:00 146.4 46816.0 2.4 145.3 100.2 53.1 false
11 2009.04.16 00:00:00 148.8 43909.0 2.3 145.6 98.3 46.4 false
12 2009.04.17 00:00:00 147.9 31635.0 2.1 145.7 99.5 49.2 false
13 2009.04.19 00:00:00 146.8 9367.0 1.9 145.8 99.1 25.3 false
14 2009.04.20 00:00:00 145.5 41054.0 1.4 145.6 97.3 14.7 false
15 2009.04.21 00:00:00 142.9 42425.0 1.1 145.4 97.7 17.9 false
16 2009.04.22 00:00:00 143.6 44563.0 .7 145.1 96.2 17.5 true
17 2009.04.23 00:00:00 141.3 43898.0 .5 144.9 97.1 22.7 false
18 2009.04.24 00:00:00 142.8 39469.0 .2 144.6 96.0 23.7 false
19 2009.04.26 00:00:00 142.0 7828.0 -.1 144.3 96.3 25.7 true
20 2009.04.27 00:00:00 141.0 41112.0 -.3 144.0 94.5 18.0 true
21 2009.04.28 00:00:00 140.7 40907.0 -.3 143.7 96.4 30.3 true
22 2009.04.29 00:00:00 142.5 30449.0 -.2 143.6 98.4 59.8 true
23 2009.04.30 00:00:00 144.4 32885.0 .0 143.8 100.4 82.7 true
24 2009.05.01 00:00:00 146.1 19640.0 .4 144.1 103.6 92.7 false
25 2009.05.03 00:00:00 148.0 5383.0 .7 144.6 103.6 94.5 false
26 2009.05.04 00:00:00 148.8 24038.0 .8 144.9 104.6 92.7 true
27 2009.05.05 00:00:00 147.9 28211.0 .9 145.3 103.2 82.3 true
28 2009.05.06 00:00:00 147.3 30350.0 1.1 145.6 104.5 72.1 true
29 2009.05.07 00:00:00 148.7 32076.0 1.2 146.0 105.5 59.1 true
30 2009.05.08 00:00:00 148.7 22138.0 1.4 146.3 106.6 65.1 false
31 2009.05.10 00:00:00 150.2 6313.0 1.5 146.7 104.9 61.9 false
32 2009.05.11 00:00:00 149.5 27513.0 1.3 146.8 101.9 50.5 false
33 2009.05.12 00:00:00 147.1 30191.0 1.2 146.9 100.7 31.3 false
34 2009.05.13 00:00:00 147.2 36016.0 .9 146.8 97.5 12.7 false
35 2009.05.14 00:00:00 144.4 44701.0 .8 146.6 98.3 23.1 false
36 2009.05.15 00:00:00 146.2 37431.0 .5 146.5 97.6 20.8 true
37 2009.05.17 00:00:00 143.7 8478.0 .2 146.2 97.5 21.4 true
38 2009.05.18 00:00:00 143.6 39476.0 .4 146.1 99.4 34.5 true
39 2009.05.19 00:00:00 147.9 46622.0 .5 146.4 99.6 57.9 true
40 2009.05.20 00:00:00 148.1 47786.0 .7 146.6 99.5 87.2 true
41 2009.05.21 00:00:00 149.2 51994.0 .8 146.8 99.9 86.0 true
42 2009.05.22 00:00:00 149.3 35143.0 1.0 147.1 102.5 92.5 false
43 2009.05.24 00:00:00 150.6 6175.0 1.1 147.4 102.1 90.2 true
44 2009.05.25 00:00:00 150.2 22616.0 1.2 147.8 104.1 85.6 true
45 2009.05.26 00:00:00 150.2 42310.0 1.4 148.1 104.2 84.7 true
46 2009.05.27 00:00:00 152.4 48549.0 1.6 148.5 106.1 89.8 true
47 2009.05.28 00:00:00 153.1 47781.0 1.8 149.1 107.0 90.9 true
48 2009.05.29 00:00:00 153.7 38171.0 2.0 149.6 104.2 81.5 true
49 2009.05.31 00:00:00 154.1 6941.0 2.1 150.1 104.0 74.3 true
50 2009.06.01 00:00:00 154.1 47195.0 2.5 150.7 106.2 79.6 false
51 2009.06.02 00:00:00 158.5 47791.0 2.8 151.4 106.2 84.9 false
52 2009.06.03 00:00:00 158.6 53801.0 2.8 152.1 103.7 73.9 true
53 2009.06.04 00:00:00 156.4 54965.0 2.8 152.6 103.8 54.6 true
54 2009.06.05 00:00:00 156.0 41565.0 2.9 153.0 105.1 46.1 false
55 2009.06.07 00:00:00 157.2 6575.0 2.8 153.4 102.9 44.3 true
56 2009.06.08 00:00:00 156.9 44823.0 2.8 153.8 102.5 45.0 true
57 2009.06.09 00:00:00 156.9 46871.0 2.9 154.2 103.7 54.2 true
58 2009.06.10 00:00:00 159.4 49857.0 3.0 154.8 104.3 73.8 true
59 2009.06.11 00:00:00 160.6 45543.0 3.2 155.4 105.0 93.8 false
60 2009.06.12 00:00:00 161.8 36976.0 3.3 156.0 101.9 90.1 false
61 2009.06.14 00:00:00 161.7 5065.0 3.3 156.6 101.6 85.6 false
62 2009.06.15 00:00:00 161.1 45319.0 2.9 156.8 100.4 57.1 false
63 2009.06.16 00:00:00 157.0 48336.0 2.7 157.0 101.7 38.1 false
64 2009.06.17 00:00:00 158.6 47684.0 2.4 157.0 99.4 19.8 true
65 2009.06.18 00:00:00 157.0 42714.0 2.2 157.0 100.6 31.6 false
66 2009.06.19 00:00:00 157.8 31360.0 2.1 157.1 101.1 39.3 false
67 2009.06.21 00:00:00 158.8 5637.0 1.9 157.3 99.0 50.8 false
68 2009.06.22 00:00:00 157.8 41347.0 1.6 157.2 96.8 44.3 true
69 2009.06.23 00:00:00 155.5 46829.0 1.4 157.1 97.2 43.1 true
70 2009.06.24 00:00:00 157.3 45874.0 1.3 157.1 97.8 49.3 false
71 2009.06.25 00:00:00 158.0 42608.0 1.2 157.1 97.8 67.0 false
72 2009.06.26 00:00:00 157.6 32514.0 1.1 157.1 100.2 71.4 true
73 2009.06.28 00:00:00 157.2 5351.0 1.0 157.1 99.2 71.7 true
74 2009.06.29 00:00:00 157.2 33749.0 1.1 157.3 101.6 76.5 false
75 2009.06.30 00:00:00 159.5 43372.0 1.1 157.5 100.9 78.4 false
76 2009.07.01 00:00:00 159.2 37880.0 1.1 157.6 100.2 77.2 false
77 2009.07.02 00:00:00 159.1 41510.0 .9 157.6 99.5 53.2 false
78 2009.07.03 00:00:00 157.1 16873.0 .8 157.6 100.9 30.4 false
79 2009.07.05 00:00:00 156.9 6332.0 .5 157.4 98.8 11.6 false
80 2009.07.06 00:00:00 155.3 38447.0 .3 157.1 98.3 19.2 false
81 2009.07.07 00:00:00 155.3 42313.0 -.2 156.8 96.2 13.6 false
82 2009.07.08 00:00:00 151.6 52563.0 -.7 156.0 95.2 20.3 false
83 2009.07.09 00:00:00 149.7 47107.0 -.9 155.5 96.4 27.2 false
84 2009.07.10 00:00:00 151.6 37966.0 -1.3 155.0 93.9 35.5 true
85 2009.07.12 00:00:00 149.7 6500.0 -1.5 154.5 93.7 36.5 true
86 2009.07.13 00:00:00 149.1 46095.0 -1.6 154.0 95.1 42.2 true
87 2009.07.14 00:00:00 151.3 45251.0 -1.5 153.8 97.3 62.3 true
88 2009.07.15 00:00:00 152.8 42123.0 -1.3 153.8 98.3 88.6 false
89 2009.07.16 00:00:00 154.2 42772.0 -1.1 153.8 98.7 86.5 true
90 2009.07.17 00:00:00 153.3 31508.0 -1.0 153.8 99.3 84.7 true
91 2009.07.19 00:00:00 154.0 7589.0 -.8 153.9 102.4 85.5 false
92 2009.07.20 00:00:00 155.2 39221.0 -.6 154.0 103.6 84.6 true
93 2009.07.21 00:00:00 155.0 36478.0 -.6 154.1 101.3 64.9 true
94 2009.07.22 00:00:00 153.6 71587.0 -.4 154.0 103.6 55.0 true
95 2009.07.23 00:00:00 155.2 72190.0 -.2 154.3 104.6 57.4 false
96 2009.07.24 00:00:00 156.0 50149.0 -.1 154.4 102.9 67.1 true
97 2009.07.26 00:00:00 156.1 7118.0 .0 154.6 102.2 67.7 false
98 2009.07.27 00:00:00 156.1 55048.0 .2 154.8 101.6 72.1 false
99 2009.07.28 00:00:00 156.7 66385.0 .2 154.9 101.0 62.6 true
100 2009.07.29 00:00:00 154.8 74955.0 .2 154.9 100.8 52.9 true
101 2009.07.30 00:00:00 155.3 61819.0 .4 155.1 101.5 51.6 true
102 2009.07.31 00:00:00 157.5 36084.0 .5 155.3 101.9 74.3 true
103 2009.08.02 00:00:00 158.4 7372.0 .7 155.7 103.1 88.4 true
104 2009.08.03 00:00:00 158.4 43384.0 1.1 156.1 104.2 91.1 false
105 2009.08.04 00:00:00 161.6 42448.0 1.3 156.6 103.3 89.4 false
106 2009.08.05 00:00:00 161.1 45734.0 1.5 157.1 103.7 87.1 true
107 2009.08.06 00:00:00 161.4 43552.0 1.5 157.5 102.4 72.0 true
108 2009.08.07 00:00:00 159.9 39975.0 1.8 157.8 103.8 72.5 false
109 2009.08.09 00:00:00 162.9 8344.0 1.9 158.3 105.0 73.6 false
110 2009.08.10 00:00:00 162.6 40598.0 1.7 158.5 102.6 64.8 false
111 2009.08.11 00:00:00 159.4 43125.0 1.4 158.5 100.0 30.3 true
112 2009.08.12 00:00:00 157.5 56369.0 1.2 158.5 100.1 16.8 false
113 2009.08.13 00:00:00 158.1 48090.0 1.0 158.5 99.5 20.6 false
114 2009.08.14 00:00:00 157.7 33811.0 .8 158.4 97.0 22.5 false
115 2009.08.16 00:00:00 156.3 7537.0 .5 158.1 96.6 14.6 true
116 2009.08.17 00:00:00 155.6 46275.0 .2 157.8 96.0 13.8 true
117 2009.08.18 00:00:00 155.0 43583.0 .1 157.6 97.8 24.9 false
118 2009.08.19 00:00:00 156.3 48098.0 .0 157.4 95.8 37.2 false
119 2009.08.20 00:00:00 155.9 42372.0 -.3 157.1 94.7 37.0 true
120 2009.08.21 00:00:00 153.9 37039.0 -.4 156.9 97.6 38.6 false
121 2009.08.23 00:00:00 155.8 7277.0 -.3 156.9 99.5 47.0 false
122 2009.08.24 00:00:00 156.7 41327.0 -.5 156.7 97.5 52.2 false
123 2009.08.25 00:00:00 154.2 42733.0 -.6 156.5 97.5 41.6 false
124 2009.08.26 00:00:00 153.8 47839.0 -.9 156.1 96.9 12.3 true
125 2009.08.27 00:00:00 151.8 47357.0 -1.1 155.7 97.9 14.9 false
126 2009.08.28 00:00:00 152.4 32945.0 -1.2 155.4 98.2 16.4 false
127 2009.08.30 00:00:00 151.7 12387.0 -1.4 155.0 96.3 17.2 false
128 2009.08.31 00:00:00 150.5 35396.0 -1.5 154.6 97.4 21.7 false
129 2009.09.01 00:00:00 151.8 51864.0 -1.6 154.2 97.4 20.2 true
130 2009.09.02 00:00:00 149.9 48149.0 -1.7 153.8 96.6 28.1 true
131 2009.09.03 00:00:00 150.2 38670.0 -1.7 153.5 96.4 31.9 true
132 2009.09.04 00:00:00 151.1 30680.0 -1.6 153.3 98.9 56.5 false
133 2009.09.06 00:00:00 152.3 5135.0 -1.4 153.2 99.2 80.3 false
134 2009.09.07 00:00:00 152.6 23202.0 -1.4 153.1 99.9 81.7 true
135 2009.09.08 00:00:00 151.7 40463.0 -1.3 153.0 100.0 75.5 true
136 2009.09.09 00:00:00 152.4 38050.0 -1.2 153.0 100.0 65.2 false
137 2009.09.10 00:00:00 152.2 41741.0 -1.1 152.9 101.4 65.8 false
138 2009.09.11 00:00:00 152.5 35683.0 -1.1 152.8 99.6 44.9 true
139 2009.09.13 00:00:00 150.6 7344.0 -1.2 152.6 100.2 26.2 false
140 2009.09.14 00:00:00 150.2 36866.0 -1.1 152.4 100.7 24.2 false
141 2009.09.15 00:00:00 151.3 40433.0 -1.2 152.2 99.2 20.6 false
142 2009.09.16 00:00:00 149.9 46982.0 -1.2 151.9 98.4 30.7 false
143 2009.09.17 00:00:00 150.1 42288.0 -1.3 151.8 98.0 25.8 false
144 2009.09.18 00:00:00 149.4 32910.0 -1.4 151.5 97.8 23.2 true
145 2009.09.20 00:00:00 148.1 4311.0 -1.4 151.2 97.5 14.0 true
146 2009.09.21 00:00:00 148.5 35055.0 -1.4 150.9 97.9 15.4 false
147 2009.09.22 00:00:00 149.0 37869.0 -1.4 150.7 97.5 21.8 false
148 2009.09.23 00:00:00 148.8 42841.0 -1.4 150.6 98.4 26.8 false
149 2009.09.24 00:00:00 148.6 49136.0 -1.7 150.2 96.4 16.0 false
150 2009.09.25 00:00:00 144.8 36165.0 -2.0 149.6 94.6 9.2 true
151 2009.09.27 00:00:00 143.0 16361.0 -2.4 148.8 94.1 9.2 true
152 2009.09.28 00:00:00 141.1 40118.0 -2.6 148.1 95.5 18.4 true
153 2009.09.29 00:00:00 143.3 46141.0 -2.6 147.6 96.3 30.0 false
154 2009.09.30 00:00:00 143.8 45479.0 -2.6 147.2 96.8 45.6 false
155 2009.10.01 00:00:00 143.6 38576.0 -2.7 146.8 95.8 53.3 false
156 2009.10.02 00:00:00 142.2 34040.0 -2.6 146.3 95.8 58.6 false
157 2009.10.05 00:00:00 143.5 32545.0 -2.6 146.0 95.6 48.7 false
158 2009.10.06 00:00:00 142.3 37698.0 -2.7 145.6 95.0 35.1 true
159 2009.10.07 00:00:00 141.2 44281.0 -2.7 145.1 97.5 28.9 true
160 2009.10.08 00:00:00 141.3 40436.0 -2.6 144.8 99.8 41.1 false
161 2009.10.09 00:00:00 142.8 30367.0 -2.5 144.5 100.8 55.6 false
162 2009.10.11 00:00:00 141.8 5143.0 -2.3 144.3 99.5 71.9 false
163 2009.10.12 00:00:00 142.6 30689.0 -2.3 144.1 98.6 67.6 true
164 2009.10.13 00:00:00 141.8 42672.0 -2.2 143.9 98.8 63.6 true
165 2009.10.14 00:00:00 141.8 41078.0 -1.9 143.8 101.0 64.3 true
166 2009.10.15 00:00:00 143.6 46573.0 -1.4 144.0 104.0 85.4 false
167 2009.10.16 00:00:00 148.5 33382.0 -.9 144.5 104.4 94.0 false
168 2009.10.18 00:00:00 148.3 7610.0 -.5 144.9 105.1 92.5 true
169 2009.10.19 00:00:00 148.4 37806.0 -.2 145.2 104.9 88.9 true
170 2009.10.20 00:00:00 148.2 38500.0 .1 145.5 104.1 87.7 true
171 2009.10.21 00:00:00 148.6 42558.0 .5 146.0 106.4 89.0 false
172 2009.10.22 00:00:00 151.4 41148.0 .9 146.5 106.8 92.2 false
173 2009.10.23 00:00:00 152.4 36914.0 1.0 147.0 105.9 77.7 true
174 2009.10.25 00:00:00 150.1 7896.0 1.0 147.3 105.3 57.9 false
175 2009.10.26 00:00:00 149.3 35524.0 1.1 147.6 104.7 39.9 false
176 2009.10.27 00:00:00 150.4 40619.0 1.1 147.9 100.5 25.0 true
177 2009.10.28 00:00:00 149.2 43941.0 .9 147.9 99.5 14.4 false
178 2009.10.29 00:00:00 147.9 39624.0 1.1 148.1 102.0 29.3 false
179 2009.10.30 00:00:00 151.3 37802.0 .9 148.3 99.7 29.4 true
180 2009.11.01 00:00:00 147.1 11166.0 .8 148.1 99.7 43.7 false
181 2009.11.02 00:00:00 148.2 38216.0 .7 148.1 97.6 27.4 true
182 2009.11.03 00:00:00 147.7 38338.0 .6 148.1 97.3 37.4 true
183 2009.11.04 00:00:00 148.2 41429.0 .6 148.2 99.4 44.1 true
184 2009.11.05 00:00:00 149.2 45288.0 .7 148.4 100.9 62.2 false
185 2009.11.06 00:00:00 150.6 49627.0 .7 148.5 99.4 69.9 true
186 2009.11.08 00:00:00 149.9 11927.0 .8 148.6 101.0 78.1 false
187 2009.11.09 00:00:00 150.6 52454.0 .8 148.9 101.9 73.5 false
188 2009.11.10 00:00:00 150.7 66159.0 .8 149.0 98.8 67.7 true
189 2009.11.11 00:00:00 149.5 55694.0 .7 149.1 100.7 40.8 true
190 2009.11.12 00:00:00 148.8 60191.0 .6 149.1 101.0 29.9 false
191 2009.11.13 00:00:00 149.7 43920.0 .6 149.1 101.4 33.1 true
192 2009.11.15 00:00:00 149.5 9181.0 .6 149.2 101.0 44.6 true
193 2009.11.16 00:00:00 149.7 54946.0 .6 149.2 100.4 48.5 false
194 2009.11.17 00:00:00 149.9 59307.0 .6 149.3 99.5 58.0 false
195 2009.11.18 00:00:00 149.9 53567.0 .5 149.3 99.6 48.8 false
196 2009.11.19 00:00:00 148.8 57502.0 .3 149.2 98.3 36.7 false
197 2009.11.20 00:00:00 148.1 43983.0 .1 149.0 97.5 17.2 true
198 2009.11.22 00:00:00 146.6 13037.0 -.1 148.8 98.3 18.0 false
199 2009.11.23 00:00:00 146.9 52616.0 -.2 148.6 99.0 19.1 false
200 2009.11.24 00:00:00 147.4 84177.0 -.3 148.5 98.0 20.4 false
201 2009.11.25 00:00:00 146.7 154191.0 -.6 148.2 96.2 15.2 false
202 2009.11.26 00:00:00 144.0 85021.0 -1.0 147.6 94.6 18.0 true
203 2009.11.27 00:00:00 141.6 56714.0 -1.3 147.0 95.2 25.9 true
204 2009.11.29 00:00:00 142.9 24244.0 -1.4 146.6 95.5 35.9 true
205 2009.11.30 00:00:00 143.0 100275.0 -1.5 146.2 96.5 44.5 true



Reader:
id date 1 5 6 7 8 9
151 2009.09.27 00:00:00 143.0 16361.0 -2.4 148.8 94.1 9.2
152 2009.09.28 00:00:00 141.1 40118.0 -2.6 148.1 95.5 18.4
153 2009.09.29 00:00:00 143.3 46141.0 -2.6 147.6 96.3 30.0
154 2009.09.30 00:00:00 143.8 45479.0 -2.6 147.2 96.8 45.6
155 2009.10.01 00:00:00 143.6 38576.0 -2.7 146.8 95.8 53.3
156 2009.10.02 00:00:00 142.2 34040.0 -2.6 146.3 95.8 58.6
157 2009.10.05 00:00:00 143.5 32545.0 -2.6 146.0 95.6 48.7
158 2009.10.06 00:00:00 142.3 37698.0 -2.7 145.6 95.0 35.1
159 2009.10.07 00:00:00 141.2 44281.0 -2.7 145.1 97.5 28.9
160 2009.10.08 00:00:00 141.3 40436.0 -2.6 144.8 99.8 41.1
161 2009.10.09 00:00:00 142.8 30367.0 -2.5 144.5 100.8 55.6
162 2009.10.11 00:00:00 141.8 5143.0 -2.3 144.3 99.5 71.9
163 2009.10.12 00:00:00 142.6 30689.0 -2.3 144.1 98.6 67.6
164 2009.10.13 00:00:00 141.8 42672.0 -2.2 143.9 98.8 63.6
165 2009.10.14 00:00:00 141.8 41078.0 -1.9 143.8 101.0 64.3
166 2009.10.15 00:00:00 143.6 46573.0 -1.4 144.0 104.0 85.4
167 2009.10.16 00:00:00 148.5 33382.0 -.9 144.5 104.4 94.0
168 2009.10.18 00:00:00 148.3 7610.0 -.5 144.9 105.1 92.5
169 2009.10.19 00:00:00 148.4 37806.0 -.2 145.2 104.9 88.9
170 2009.10.20 00:00:00 148.2 38500.0 .1 145.5 104.1 87.7
171 2009.10.21 00:00:00 148.6 42558.0 .5 146.0 106.4 89.0
172 2009.10.22 00:00:00 151.4 41148.0 .9 146.5 106.8 92.2
173 2009.10.23 00:00:00 152.4 36914.0 1.0 147.0 105.9 77.7
174 2009.10.25 00:00:00 150.1 7896.0 1.0 147.3 105.3 57.9
175 2009.10.26 00:00:00 149.3 35524.0 1.1 147.6 104.7 39.9
176 2009.10.27 00:00:00 150.4 40619.0 1.1 147.9 100.5 25.0
177 2009.10.28 00:00:00 149.2 43941.0 .9 147.9 99.5 14.4
178 2009.10.29 00:00:00 147.9 39624.0 1.1 148.1 102.0 29.3
179 2009.10.30 00:00:00 151.3 37802.0 .9 148.3 99.7 29.4
180 2009.11.01 00:00:00 147.1 11166.0 .8 148.1 99.7 43.7
181 2009.11.02 00:00:00 148.2 38216.0 .7 148.1 97.6 27.4
182 2009.11.03 00:00:00 147.7 38338.0 .6 148.1 97.3 37.4
183 2009.11.04 00:00:00 148.2 41429.0 .6 148.2 99.4 44.1
184 2009.11.05 00:00:00 149.2 45288.0 .7 148.4 100.9 62.2
185 2009.11.06 00:00:00 150.6 49627.0 .7 148.5 99.4 69.9
186 2009.11.08 00:00:00 149.9 11927.0 .8 148.6 101.0 78.1
187 2009.11.09 00:00:00 150.6 52454.0 .8 148.9 101.9 73.5
188 2009.11.10 00:00:00 150.7 66159.0 .8 149.0 98.8 67.7
189 2009.11.11 00:00:00 149.5 55694.0 .7 149.1 100.7 40.8
190 2009.11.12 00:00:00 148.8 60191.0 .6 149.1 101.0 29.9
191 2009.11.13 00:00:00 149.7 43920.0 .6 149.1 101.4 33.1
192 2009.11.15 00:00:00 149.5 9181.0 .6 149.2 101.0 44.6
193 2009.11.16 00:00:00 149.7 54946.0 .6 149.2 100.4 48.5
194 2009.11.17 00:00:00 149.9 59307.0 .6 149.3 99.5 58.0
195 2009.11.18 00:00:00 149.9 53567.0 .5 149.3 99.6 48.8
196 2009.11.19 00:00:00 148.8 57502.0 .3 149.2 98.3 36.7
197 2009.11.20 00:00:00 148.1 43983.0 .1 149.0 97.5 17.2
198 2009.11.22 00:00:00 146.6 13037.0 -.1 148.8 98.3 18.0
199 2009.11.23 00:00:00 146.9 52616.0 -.2 148.6 99.0 19.1
200 2009.11.24 00:00:00 147.4 84177.0 -.3 148.5 98.0 20.4
201 2009.11.25 00:00:00 146.7 154191.0 -.6 148.2 96.2 15.2
202 2009.11.26 00:00:00 144.0 85021.0 -1.0 147.6 94.6 18.0
203 2009.11.27 00:00:00 141.6 56714.0 -1.3 147.0 95.2 25.9
204 2009.11.29 00:00:00 142.9 24244.0 -1.4 146.6 95.5 35.9
205 2009.11.30 00:00:00 143.0 100275.0 -1.5 146.2 96.5 44.5

land

Hi Cleo,
the Validation's subprocesses are executed several times. Each one on a different SUBset of the data. When writing results inside the validation, you are simply overwriting all of them again and again, until the last round. So the model and performance are both generated on a subset of the data! If you retrieve the performance and model from the validation itself, the model is rebuilt from all training data and the performance is averaged. So this both vectors and models must differ!

Greetings,
  Sebastian

Cleo

Hello Sebastian,

Thank you for clarifying the sliding window operator for me.  I added an iterating macro and followed your advice and the process is a little clearer for me.  I would like to ask a couple of follow up questions.

1) Should the training window width = the test window width?
2) If I add a threshold finder/threshold applier and model applier to the Testing portion of the Sliding Window Validation, does it get included in the “mod”?  When I have added the threshold I get much better predictions, however when I use the model on unseen data I get a uniform prediction of false.  Any suggestions on how to use the threshold operator with the sliding windows.

Thanks again,
Cleo

land

Hi Cleo,
if you don't have the same window with, won't you have different attributes? You cannot apply the trained models anyway then.
The threshold operator will not be included into the model automatically. You will have to add an Threshold Applier on the testing side and forward the Threshold.

Greetings,
  Sebastian