Re: Evaluating numeric label
Contributor II
Hey there,
thanks for the micro thread.
Here is the requested code:
This process gives me an error msg (I hope it does at well for you). Some remarks: Even so the "description" for X-Validation says "uses decision tree" it actually is using a neural net. I replaced the decision tree.
thanks for the micro thread.
Here is the requested code:
This process gives me an error msg (I hope it does at well for you). Some remarks: Even so the "description" for X-Validation says "uses decision tree" it actually is using a neural net. I replaced the decision tree.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.1.006">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.1.006" expanded="true" name="Process">
<process expanded="true" height="393" width="748">
<operator activated="true" class="read_excel" compatibility="5.1.006" expanded="true" height="60" name="Read Excel" width="90" x="45" y="30">
<parameter key="excel_file" value="c:\abx.xls"/>
<parameter key="imported_cell_range" value="A1:AM280"/>
<parameter key="first_row_as_names" value="false"/>
<list key="annotations"/>
<list key="data_set_meta_data_information">
<parameter key="0" value="ID.true.polynominal.id"/>
<parameter key="1" value="attribute1.true.polynominal.attribute"/>
<parameter key="2" value="attribute2.true.polynominal.attribute"/>
<parameter key="3" value="attribute3.true.polynominal.attribute"/>
<parameter key="4" value="attribute4.true.polynominal.attribute"/>
<parameter key="5" value="attribute5.true.polynominal.attribute"/>
<parameter key="6" value="attribute6.true.polynominal.attribute"/>
<parameter key="7" value="attribute7.true.polynominal.attribute"/>
<parameter key="8" value="attribute8.true.polynominal.attribute"/>
<parameter key="9" value="attribute9.true.numeric.attribute"/>
<parameter key="10" value="attribute10.true.numeric.attribute"/>
<parameter key="11" value="attribute11.true.numeric.attribute"/>
<parameter key="12" value="attribute12.true.numeric.attribute"/>
<parameter key="13" value="attribute13.true.polynominal.attribute"/>
<parameter key="14" value="attribute14.true.numeric.label"/>
</list>
</operator>
<operator activated="true" class="filter_example_range" compatibility="5.1.006" expanded="true" height="76" name="Filter Example Range" width="90" x="179" y="30">
<parameter key="first_example" value="1"/>
<parameter key="last_example" value="4"/>
<parameter key="invert_filter" value="true"/>
</operator>
<operator activated="true" class="nominal_to_numerical" compatibility="5.1.006" expanded="true" height="94" name="Nominal to Numerical" width="90" x="313" y="30"/>
<operator activated="true" class="x_validation" compatibility="5.1.002" expanded="true" height="112" name="Validation" width="90" x="447" y="30">
<description>A cross-validation evaluating a decision tree model.</description>
%2
0
Answers
You are confused. The operator X-validation does work with both numeric and categorical labels. Here's a quick example (it uses the default measure of performance for numerical labels --rmse):
indeed, confused I am. : )
Thanks for the reply. I just recently started with RM and I am using it for my studies at university. Hope you understand if my question are kind of stupid.
I pasted the code you replied into RM and actually it gave me an error message:
Message: X-Validation cannot handle numerical label.
Fixes:
-Switch sampling to shuffled
-Add discretization operator
Location: Validation.Training
Thats the problem I tried to describe above. : /
If I use the fixe "switch sampling to shuffled" it actually works.Though I cannot understand why. How is the sampling method related to the data types?
You mentioned the X-Validation uses the Root Mean Squared Error for evaluating numeric labels. What is the second Error given (the "mikro" one):
root_mean_squared_error: 23.233 +/- 11.548 (mikro: 25.945 +/- 0.000)
Thanks a lot for further replies!
About your reply: The error you are getting typically means that the Learning operator you are using does not accept numeric labels (for instance if you try to use a Decision Tree with a numerical label). Since the learning operator is inside the X-Validation operator, it sounds as if it is the latter the one complaining. Please upload your process to take a look at it.
For your question about mikro, I'll refer you to an entry by Ingo Mierswa:
http://rapid-i.com/rapidforum/index.php/topic,3718.0.html
thanks for the micro thread.
Here is the requested code:
This process gives me an error msg (I hope it does at well for you). Some remarks: Even so the "description" for X-Validation says "uses decision tree" it actually is using a neural net. I replaced the decision tree. This one actually is error free: Maybe the explanation is the following: In the first process, the numeric label is defined in the "retrieve" operator. In the second process, the label is defined in "set role" and afterwards changed to numeric. Does that makes a difference?
Please understand that I cannot post the excel sheet, since it includes internal data. : /
Thanks for the replies!
/edit: I just noticed that you used a different performance evalutor than I did.. Why did you use the "performance(regression)" operator?
You can still use the Performance Operator which for numerical labels will default to the RSME metric, but if you want control over the exact metric it uses then you have to use the one I used.
I understand you cannot post the data. I tried to run your example with a similar dataset but I have no problem whatsoever. I noticed a small detail too. When you read the Excel sheet you have to declare the first row as NAME (you do this in Step 3 out 4 in the Data Import Wizard Process). Click on the first row (first column labeled Annotation) and select Name. Try doing that.
okay thanks a lot. But actually, the first row of the spreadsheet isnt the names. There are 3 to4 rows almost empty and a couple of attributes arnt named either. So I decided to rename then manually.
In regard to the evaluator: thanks again, I am gonna try around some more. Maybe I can find the error. I am just glad it works now. : )
Thanks!