Why KNN in rapidminer is giving memory problem ?

kashif_khankashif_khan Member Posts: 19 Contributor II
edited November 2018 in Help
Hi, i am working in text classification on 20NewsGroup dataset with 100 documents in each category. I classify text documents via Naive Bayes using 10-fold cross validation, It runs successfully and give me results at the end.

I tried same with KNN with 10-fold cross validation but it always ends in "Process Failure" which shows that it requires more memory than available. I increase heap space for rapidminer to 2.5G in build.xml as well as rapidminerGUI.bat but nothing improves and it always ends up in demanding more memory.

Kindly help, i am stuck out at it and tried every possible option i could think about

Platform Details:

OS: Windows 7(64 bit)
Software Version: Rapidminer 5.3 (64 bit)
Java: Java 1.7 (64 bit)


  • Options
    kashif_khankashif_khan Member Posts: 19 Contributor II
    No Reply ? :(:(:(
  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering

    some algorithms require more memory than others. How much memory does your system have and what is available to RapidMiner? You can check how much RapidMiner can access by selecting "View" -> "Show View" -> "System Monitor".

  • Options
    kashif_khankashif_khan Member Posts: 19 Contributor II
    System Monitor is showing

    Total: 1.2 G
    Max: 1.2G

    I have total RAM of 3G available in my system. I think Rapid miner use a format for vector which is too heavy. I wrote my example-set after calculating tf-idf in rapidminer and file size was 1.24G.
Sign In or Register to comment.