[SOLVED] Importing 700+ Columns in RapidMiner

Analyticaltim

Dear RapidMiner Community,

I have an odd CSV file that I am attempting to use with RapidMiner. This file has 767 columns and only 256 rows. I have been unable to import all the columns into RapidMiner, (it cuts off after 256). Is there an extension or modification that I can use to expand the available size of RapidMiner's columns. The overall data size is not that big yet the amount of columns is quite large and growing.

Thank you very much for your input.

Best,
Tim

earmijo

What version do you have installed in your computer? I have no problem whatsoever reading a csv file with 770 columns.

Analyticaltim

I am using the free version 5.3.007.

Holy cow, I don't know why mine keeps cutting the CSV at 256 rows. Do you think it is an available memory thing?

Tim

awchisholm

The best thing to do is to post the XML of your process

marcin_blachnik

Please check if you have choosen correct value separator. It may happen that if you choose incorrect one it may combine several columns into single one, leading to just 256 columns.
So I suggest to set the separator to ";" or other unique char and configure the same char in the import wizard.

Analyticaltim

Thanks for all your help Rapid-i community.

I figured it out. Excel was cutting off the sheet at 256 columns. So I put the data into OpenOffice (who does not have that problem) and used the idea of employing a ";" as my separator rather than a comma. It works great and every time.

Thank you all very much once again.!  ::)

Best.
Tim

earmijo

I should have asked "What version of Excel are you using". From Excel 2007 to the present, Excel can handle up to 16,000 columns. You are probably using Excel 97.

Analyticaltim

You got it. I am using a way old version of Excel. I am thinking that I will switch to OpenOffice on a permanent basis. I love the open source stuff.

Thanks for all your help!
Tim