Options

Strangely Large File Sizes for Small Amounts of Data in Example Sets

monoremonore Member Posts: 4 Contributor I
edited May 2020 in Help
I have had several files with file sizes in Store outputs that seem very disproportionate - i.e. 125MB for a 233 row, 29 attribute file. Why is this and how do i fix this? 

My only solution so far has been the cumbersome step of writing to an Excel (which has to be done locally). Then separately reading that Excel. For the file mentioned previously this brought the file in question from 125MB to 6.5MB. 

Is there something with metadata or attribute types to make a change so drastic? Whats the easier way to get these file sizes under control?
Tagged:

Answers

  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,525 RM Data Scientist
    Hi,
    can you try to use a Remove Unused Values operator before Store? This should do it.

    Cheers,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.