Options

Operator nominal to numerical error

BifruanBifruan Member Posts: 2 Newbie
My data is 700,000 records with 8 Attributes.
When I start to operate "nominal to numerical" the error is occur that "need more memory"
but my computer is 8 GB with 512 SSD and Core I5 Gen 11
Is that my spec of my computer is not enough for this operation?
what I can do to fix this problem?

Answers

  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,517 RM Data Scientist
    Hi,
    how many different classes are in your column? If you have a 700.000 different values in one column, than this operator simply creates 700.000 new columns. This means you roughly increase your data set by the factor 700.000.

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • Options
    BifruanBifruan Member Posts: 2 Newbie
    There are 5000 different classes in one column so is it possible that rapidminer can create 5000 new column.
    And Is that student license affect to limit of the operation rows?

    Best Regards
  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,517 RM Data Scientist
    edited February 2021
    no it does come from your edu license.You increased your data set size from
    700.000x8 = 5.6 Million data entries
    to
    700.000x5000 = 3.5 Billion data entries.
    Thats just killing your laptop. I would either choose something like target encoding or try to reduce the number of different classes i.e. with Replace Rare Values or manual mapping

    Best,
    Martin


    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • Options
    MarcoBarradasMarcoBarradas Administrator, Employee, RapidMiner Certified Analyst, Member Posts: 272 Unicorn
    @Bifruan hi what type of model are you trying to build, from the picture you uploaded i see that you are taking a basket ID in the operator. I'm guessing you are using some kind of transactions dataset, may basket analysis, why do you need to convert your ID to individual columns?


Sign In or Register to comment.