Options

rapidMiner Studio 6.30 OSX memory surge

ouverzouverz Member Posts: 4 Contributor I
edited August 2019 in Help
Hi there

I am a newbie of rapidMiner and have been trying to find a simple solution to limit rapidMiner's free memory usage, I reckon its more related to java though. I have 4GB RAM on a Macbook pro retina, the data I am processing is not large (20MB ~) and the program freezes up lots of times.

Could someone please provide me with a solution to limit the memory usage ?

Regards
Ofer
Tagged:

Answers

  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering
    Hi,

    to answer that question I must ask: Which version of RapidMiner Studio are you using?

    Regards,
    Marco
  • Options
    ouverzouverz Member Posts: 4 Contributor I
    Thank for replying.

    The version I use is rapidMiner Studio 6.3.00 on Yosemite 10.10.3

  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering
    Hi,

    Studio 6 limits the available memory according to your installed license. Trial license is 8GB (or less if your system has less) and Starter license (after your Trial has expired) is 1GB. Other licenses as stated in the license contract.
    You can see how much memory Studio will use by clicking "View" -> "Show View" -> "System Monitor" in the top menu bar.
    If it does use too much, you can limit it additionally by clicking "Tools" -> "Preferences" and navigating to the "System" tab. Near the bottom is an option called "Maximum amount of memory". You could set that for example to 512 to limit Studio to 512MB of memory.
    However, most of the time Studio freezes are a side effect of Java hitting its defined memory limit. It then frantically tries to free as much of its resources as possible to keep going. So limiting it even further is probably not helping the case.

    Regards,
    Marco
  • Options
    ouverzouverz Member Posts: 4 Contributor I
    Hi thanks for the quick reply.

    My understanding is that rapidMiner is configured to consume 80%-90% ~ of the available memory. So I have seen systems with 8GB RAM struggling with quite mundane tasks of text-mining.
    Setting a max mem in rapidMiner didn't fix my problem of laggy and processes idling.
    Would limiting Java memory utilization eliminate this issue?

    Thanks
  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering
    Hi,

    did it reduce the max memory in the System Monitor after closing and then restarting Studio?
    The option basically does nothing else than cap the -Xmx option given to Java. -Xmx will be set to 90% of your system memory, your available memory according to your license or the Maximum amount of memory value, whichever is lowest.

    Regards,
    Marco
  • Options
    ouverzouverz Member Posts: 4 Contributor I
    Yeah, it seems as if now the memory usage is in check but I need to optimize the right amount of memory as now the process runs 7-8 times as much as before.
    I have to say that I really like the program but I am a little surprised that it consumes so much out of a system for such small data.
  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering
    Hi,

    totally depends on what you're doing. You can work with 200 MB of original data and go to 400 MB, or your can work with 200 MB of original data and go to several GB ;)

    Regards,
    Marco
Sign In or Register to comment.