Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
[SOLVED] Read CSV BUG: Two Consecutive Missing Columns Causes Read Error
Hi Rapid Miner Gurus,
Using RapidMiner 5.2.006x64 on Windows 7x64 I am running into a problem with the Read CSV operator. I do not recall this being a problem in the past versions but somehow it has cropped into the code.
I have a simple example data set test.csv that has 4 columns, two that are missing and next to each other as the last two columns. This problem occurs when there are two or more consecutive missing column's at the right end of the table. Excel reads this CSV fine and so do most other data analysis tools. However RapidMiner throws an error. ???
NOTE: This problem can be somewhat avoided if you EXPLICITLY set the column index and type for every attribute in the csv file. Unfortunately this is not an option because most of the time you are reading a data set that either has too many columns to manually set or has a changing number of columns some of which may be missing. RapidMiner handled missing columns in CSVs very well before so I hope it just a little bug. ;D
Using RapidMiner 5.2.006x64 on Windows 7x64 I am running into a problem with the Read CSV operator. I do not recall this being a problem in the past versions but somehow it has cropped into the code.
I have a simple example data set test.csv that has 4 columns, two that are missing and next to each other as the last two columns. This problem occurs when there are two or more consecutive missing column's at the right end of the table. Excel reads this CSV fine and so do most other data analysis tools. However RapidMiner throws an error. ???
SEVERE: Process failed: An attribute att4 was specified for column 3, but this column does not exist in input data.
A,B,C,D
1,2,,
3,4,,
5,6,,
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.006">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.2.006" expanded="true" name="Process">
<process expanded="true" height="145" width="145">
<operator activated="true" class="read_csv" compatibility="5.2.006" expanded="true" height="60" name="Read CSV" width="90" x="45" y="30">
<parameter key="csv_file" value="C:\test.csv"/>
<parameter key="column_separators" value=","/>
<parameter key="use_quotes" value="false"/>
<list key="annotations"/>
<list key="data_set_meta_data_information"/>
<parameter key="read_not_matching_values_as_missings" value="false"/>
</operator>
<connect from_op="Read CSV" from_port="output" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
NOTE: This problem can be somewhat avoided if you EXPLICITLY set the column index and type for every attribute in the csv file. Unfortunately this is not an option because most of the time you are reading a data set that either has too many columns to manually set or has a changing number of columns some of which may be missing. RapidMiner handled missing columns in CSVs very well before so I hope it just a little bug. ;D
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.006">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="5.2.006" expanded="true" name="Process">
<process expanded="true" height="145" width="145">
<operator activated="true" class="read_csv" compatibility="5.2.006" expanded="true" height="60" name="Read CSV" width="90" x="45" y="30">
<parameter key="csv_file" value="C:\Users\ddrmanac\Desktop\test.csv"/>
<parameter key="column_separators" value=","/>
<parameter key="use_quotes" value="false"/>
<list key="annotations"/>
<list key="data_set_meta_data_information">
<parameter key="0" value=".true.nominal.attribute"/>
<parameter key="1" value=".true.integer.attribute"/>
<parameter key="2" value=".true.integer.attribute"/>
<parameter key="3" value=".true.integer.attribute"/>
</list>
<parameter key="read_not_matching_values_as_missings" value="false"/>
</operator>
<connect from_op="Read CSV" from_port="output" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
0
Answers
thank you for your detailed report. This bug was already fixed in the latest version 5.2.008.
Please update your RapidMiner to this version and reading CSV files with missings at the end of a line shouldn't be a problem anymore.
Best,
Nils
I can confirm that the upgrading to 5.2.008 fixed the issue.
Thanks,
-Gagi
Hi Team,
I get the attribute error while running the process.
Attached the snapshot, additionally the error is "Th attribute att2specified in the old_name parameter doesnot exist in the input data."
hello @jaybusiness86 - welcome to the community. This is an old thread. I would highly recommend reading the info in the "Getting Started Forum" and reposting there with your process etc...
Good luck!
Scott