The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.
Options

main memory limit reached message and restarting the application

User13User13 Member Posts: 155 Maven
edited November 2018 in Help
I'm running the free version of RM Studio on a linux machine (and adding cloud when the capacity is insufficient).



When I'm running largish data sets (about 2M records), after a few runs I get the message: Main Memory Limit Reached, with an encouragement to upgrade.



Yet, if I close the RM application and restart, the process runs just fine.



What's happening here? What can I do to prevent it?



Best Answer

  • Options
    User13User13 Member Posts: 155 Maven
    Solution Accepted
    Hi John,

    this may happen when your process requires an amount of memory which is very close to your maximum memory limit. After a few runs Java may not clean up every entity in memory causing or memory controller to launch the upgrade notification. You may try to force a garbage collection by using the "Free Memory" operator as the first operator of your process.

    Cheers,
    Helge

Answers

  • Options
    User13User13 Member Posts: 155 Maven
    Thanks, Helge - this worked well. I ended up putting the Free Memory operator right after my Retrieve operator, and since Retrieve has no input, also near the end of my process. Appreciate it.

    Is there a plan or timeline to systematically correct the Java garbage collection?
  • Options
    User13User13 Member Posts: 155 Maven
    Hi John,

    I think the problem here is that our upgrade notification can sometimes be activated before the Java GC frees additional memory. There are only those processes affected that use almost the maximum available memory. In those case the "Free Memory" operator can help. I think we will investigate this and maybe try to fine tune this a bit in order to avoid false notifications.

    Cheers,
    Helge
Sign In or Register to comment.