RapidMiner

Contributor II aruberutou
Contributor II

Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

Hello,

Thanks for producing such a wonderful tool. It has helped my work tremendously.

One minor niggle, however, frustrated me for a quite a bit and I wanted to share my work-around.

I have been using SentiWordNet on an OSX installation, and it tended to die once it hit a certain number of files. Now, I am not sure if its a coding issue (i.e., should it really be opening so many files at once), but it easily fixed by adding "-XX:-MaxFDLimit" to the RapidMiner-Studio-App.sh file, at the appropriate places, as shown below:

# launch RapidMiner Studio
if [ $# -gt 0 ]; then
eval \"$JAVA_HOME\"/bin/java $JVM_OPTIONS -XX:-MaxFDLimit -cp \"${RM_CLASSPATH}\" com.rapidminer.launcher.GUILauncher \"$@\"
else
eval \"$JAVA_HOME\"/bin/java $JVM_OPTIONS -XX:-MaxFDLimit -cp \"${RM_CLASSPATH}\" com.rapidminer.launcher.GUILauncher

Again, I am not sure if it is this way by design, but maybe the RM team should consider including this option by default in the OSX app. The Windows version doesn't seem to suffer from this limitation.

Thanks,
8 REPLIES
RM Staff
RM Staff

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

Hi,

thanks for bringing this to our attention! This looks more like a bug in the SentiWordNet extension to me, though. I have created an internal ticket for it.
We won't globally disable the JVM open file limit of a little more than 10.000 files because that could potentially do more harm than good.

Regards,
Marco
_________________________________________________________
Team Lead Software Engineering | RapidMiner GmbH
Contributor II aruberutou
Contributor II

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

Hi,

Thanks for the quick follow-up. I haven't noticed any issues since disabling the limit; however, I am sure it is probably there for a reason. Smiley Happy

weewey
N/A

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

I have this problem as well. RM always give me an error when doing sentiment analysis to more than 200 over tweets.

I tried the solution that aruberutou mentioned but I'm still facing the problem. ://   ??? ???

This is the error message:
"Error reading /usr/local/WordNet- 3.0/dict:/usr/local/WordNet-3.0/dict/index.verb (Too many open files)"
Contributor II aruberutou
Contributor II

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

Hi, weewey,

Sorry for the late response. In order to resolve this, I also had to increase the respective limits in OSX as well.

Please refer to the following websites for details:

http://blog.mact.me/2014/10/22/yosemite-upgrade-changes-open-file-limit
http://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite/82801...

weewey
N/A

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

@aruberutou

Awesome! Thanks alot!! I got it fixed already.

weewey
N/A

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

@aruberutou

I realise that rapidminer server result in the same problem too.

Do you know how to resolve it?

Appreciate your help
RM Staff
RM Staff

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

Hi,

depending on your OS, it's either the standalone.conf (Linux/Mac) or the standalone.conf.bat file which you can find in the SERVER/bin folder. Look for a line starting with "JAVA_OPTS=" and add your parameters to the existing options.

Regards,
Marco
_________________________________________________________
Team Lead Software Engineering | RapidMiner GmbH
Highlighted
Contributor II rachel_lomasky
Contributor II

Re: Rapidminer OSX app should have "-XX:-MaxFDLimit" by default

The problem is that you have put the "Open WordNet Dictionary" operator inside of the "Process Documents" operator.  You should open it outside, and then use a remember with the type "WordNet Dictionary."

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="wordnetSmiley Surprisedpen_wordnet_dictionary" compatibility="5.3.000" expanded="true" height="68" name="Open WordNet Dictionary" width="90" x="380" y="340">
<parameter key="resource_type" value="directory"/>
<parameter key="directory" value="%{wordnet_directory}"/>
</operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="remember" compatibility="7.4.000" expanded="true" height="68" name="Remember" width="90" x="581" y="340">
<parameter key="name" value="wordnet"/>
<parameter key="io_object" value="WordnetDictionary"/>
<parameter key="store_which" value="1"/>
<parameter key="remove_from_process" value="true"/>
</operator>
</process>

 

And then you can use a "Recall" inside the "Process Documents":

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="recall" compatibility="7.4.000" expanded="true" height="68" name="Recall" width="90" x="313" y="238">
<parameter key="name" value="wordnet"/>
<parameter key="io_object" value="WordnetDictionary"/>
<parameter key="remove_from_store" value="false"/>
</operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
<operator activated="true" class="wordnet:find_hypernym_wordnet" compatibility="5.3.000" expanded="true" height="82" name="Find Hypernyms (WordNet)" width="90" x="514" y="34">
<parameter key="use_prefix" value="true"/>
<parameter key="synset_word_prefix" value="hyper:"/>
<parameter key="maximum_recursion_depth" value="1"/>
<parameter key="multiple_meanings_per_word_policy" value="Take only first meaning"/>
<parameter key="multiple_synsets_policy" value="Take all synsets per meaning"/>
<parameter key="multiple_synset_words_policy" value="Take only first synset word"/>
<parameter key="concatenation" value="Concatenate result per synset"/>
<parameter key="keep_original_tokens" value="true"/>
<parameter key="keep_unmatched_tokens" value="false"/>
<parameter key="take_ID_instead_of_words" value="false"/>
<parameter key="work_on_type_noun" value="true"/>
<parameter key="work_on_type_verb" value="true"/>
<parameter key="work_on_type_adjective" value="true"/>
<parameter key="work_on_type_adverb" value="true"/>
</operator>
</process>

Polls
How can RapidMiner increase participation in our new competitions?
Twitter Feed