memory issues

mjavier99mjavier99 Member Posts: 1 Contributor I
edited November 2018 in Help
Hi, I am running a Clustering Model W-Clope with a table with about 1 million registers. RapiMiner never ends and a have a memory leak. I have configured MAX_JAVA_MEMORY to 2048 because I have 3 GB of memory to use. This RapidMiner build to be used with large table datasets ?

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    weka learners generally needs to copy the data once if used inside RapidMiner. Hence you will have a waste of memory of factor two.

    Then you will have to think that most Datamining algorithms need a runtime equivalent to the number of examples to power of three. So 1 million examples is simply much to much...Even if each computational step would only need 1 clock of your CPU, a 2 GHz CPU would need 2 years to solve such a problem...

    Greetings,
      Sebastian
Sign In or Register to comment.