🎉 🎉 RAPIDMINER 9.10 IS OUT!!! 🎉🎉

Download the latest version helping analytics teams accelerate time-to-value for streaming and IIOT use cases.

CLICK HERE TO DOWNLOAD

Memory issue in LOFOutlierDetection operator

ShubhaShubha Member Posts: 139  Guru
edited November 2018 in Help
Hi,

I have a data with 4615 observations and I am applying ' LOFOutlierDetection' on this data. But doing this, I get the error, "OutOfMemoryError Caught: Java heap space". Below i paste the code:
<operator name="Root" class="Process" expanded="yes">
    <operator name="CSVExampleSource" class="CSVExampleSource">
        <parameter key="filename" value="E:\Inputs\Datafile.csv"/>
    </operator>
    <operator name="LOFOutlierDetection" class="LOFOutlierDetection">
        <parameter key="minimal_points_lower_bound" value="3"/>
        <parameter key="minimal_points_upper_bound" value="3"/>
    </operator>
</operator>
I tried to increase the allocated memory in 'RapidMinerGUI.bat' (MAX_JAVA_MEMORY), but this was not useful.

How can I tackle this memory issue?

Details of my system:
2.79 GHz, 0.99GB of RAM.... I can increase my RAM size to 2GB too, but again the problem persists.

Thanks in advance,
Shubha.


Answers

  • homburghomburg Moderator, Employee, Member Posts: 114  RM Data Scientist
    Hi Shubha.

    Using outlier detection operators working with inter-object distances is a memory intensive job, especially for datasets with lots of observations. For your dataset you may need up to 1.5G or even 2G of main memory exclusivly available to RapidMiner to perform the LOF outlier detection. Have a look at the SystemMonitor view in RapidMiner to check the amount of main memory assigned to the program.

    Greetings,
        Helge
Sign In or Register to comment.