RapidMiner

how to properly connect to local mongo DB?

Elite

Re: how to properly connect to local mongo DB?

[ Edited ]

Weird, it isn't really rocket science so this is giving me a serious headache as this is a potential showstopper for us.

 

Did you try running a real process storing data, or did you just test the connection itself? 

The connection works for me also, it just gives a problem when I actually want to use it inside of a process.

 

I have it on 2 seperate machines, one running on windows and another running linux, only difference is that I am running mongo 3.2 on both instead of 3.4

I'll try if I can upgrade to 3.4 and hope that makes a difference, otherwise I'm stuck indeed.

 

This is my test script :

 

<process version="7.3.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve sample" width="90" x="45" y="34">
        <parameter key="repository_entry" value="sampledata"/>
      </operator>
      <operator activated="true" class="text:data_to_json" compatibility="7.4.001" expanded="true" height="82" name="Data To JSON (2)" width="90" x="246" y="34">
        <parameter key="ignore_arrays" value="false"/>
        <parameter key="generate_array" value="false"/>
        <parameter key="include_missing_values" value="false"/>
      </operator>
      <operator activated="true" class="nosql:mongodb_document_writer" compatibility="7.2.000" expanded="true" height="82" name="Write MongoDB (2)" width="90" x="380" y="34">
        <parameter key="mongodb_instance" value="mng_Reviews_models"/>
        <parameter key="write_concern_" value="Acknowledged"/>
        <parameter key="write_concern" value="Acknowledged"/>
        <parameter key="collection" value="sample"/>
        <parameter key="skip_invalid_documents" value="true"/>
      </operator>
      <connect from_op="Retrieve amz_de_LG_tv" from_port="output" to_op="Data To JSON (2)" to_port="example set 1"/>
      <connect from_op="Data To JSON (2)" from_port="documents" to_op="Write MongoDB (2)" to_port="documents 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
    </process>
  </operator>
</process>

Screenshot from 2017-02-15 14-36-12.pngScreenshot from 2017-02-15 14-39-12.png

RMStaff

Re: how to properly connect to local mongo DB?

I used your process (with the Iris data set as source), stored it in the admin home folder and also renamed my MongoDB connection.

No error occurs - regardless of the way I execute the process.

One thing that pops into my mind.

What version of the NoSQL extension are you running on Server & Studio? Are those different?

 

Best,

Edin

Elite

Re: how to properly connect to local mongo DB?

[ Edited ]

Nope, both are 7.3.0 :-(

 

Could also be something I did wrong with the Mongo installation, though as far as I recall that was also just a straight forward process and it seems to be working for any of my other applications using it (and with RM desktop)

 

Could it be related to the Mongo dbpath or so? Not that I see anything wrong with that, for my windows setup that is located on C:\data\db si that should be accessible by the server also I assume?

Contributor

Re: how to properly connect to local mongo DB?

[ Edited ]

I have exactly same issue with Rapidminer server, desktop and NoSQL extenstion in version 7.4. Server runs in Ubuntu 14.04. Mongo is 3.4

RMStaff

Re: how to properly connect to local mongo DB?

[ Edited ]

The standard installation procedure, as well as the MongoDB path, should not be the issue - I used it to.

Did you try to store and run the process not in a user home folder but instead in a generally available folder (e.g. groups)?

Else, I am running out of ideas Smiley Sad

 

In the meantime I try to get some information from our development team.

 

Best regards,

Edin

Elite

Re: how to properly connect to local mongo DB?

[ Edited ]

I can give it a try in a different folder but don't expect much tbh.

 

Could it be related to activation code? Mine is already a fairly old one, before noSQL was an option. Just trying to put additioanl ideas on the table...

 

[UPDATE] Installing the process in another folder (like groups) did not make any difference. The problem remains the same.

RMStaff

Re: how to properly connect to local mongo DB?

The activation code is not the problem.

 

@kayman and @hcha: Is it possible that you send me your server.log (located in InstallationDirectory/standalone/log/) of a day when the connection problem occurred (by PM)?

It helps our developers to further narrow down the issue.

 

Best regards,

Edin

Contributor

Re: how to properly connect to local mongo DB?

Hi @Edin_Klapic

 

I have exactly the same issue listed in the thread.

I am able to execute mongo reads from studio but it fails on the server. I have tried everythin listed in the thread with no avail.
Below is the full server log for the process execution. One interesting part is the warning that reads: 

 

10:25:16,583 WARNING [com.rapidminer.server.tools.config.ServerConfigurationManager] (scheduledprocess_1491747916577) User null is not allowed to view configurationEntry: Access denied for entry Localhost of mongodb_instance

It seems the server configuration manager doesn't have a user thus it can't read the connection (my connections is called "Localhost").

 

Is there source code for the noSQL connector open and if so where is it hosted to take a look at what is going on?

 

Full log:

10:25:15,695 INFO [de.rapidanalytics.web.tools.ProcessServiceDelegate] (http-/0.0.0.0:8080-7) Scheduled process execution /home/admin/processes/Retrieve Data with parameter ID 22. Will fire first at Sun Apr 09 10:25:15 EDT 2017.
10:25:16,577 INFO [de.rapidanalytics.ejb.ProcessJobEJBImpl] (EJB default - 28) Job /home/admin/processes/Retrieve Data was assigned id 27
10:25:16,577 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Dispatching job 27 to queue DEFAULT. (SimpleProcessExecutor)
10:25:16,577 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Starting process /home/admin/processes/Retrieve Data in queue DEFAULT.
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process context is: Macros: [_ra_user : admin, _ra_jobID : 27]; Input: []; Output: []
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) No filename given for result file, using stdout for logging results!
10:25:16,580 INFO [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process //_LOCAL/home/admin/processes/Retrieve Data starts
10:25:16,583 WARNING [com.rapidminer.server.tools.config.ServerConfigurationManager] (scheduledprocess_1491747916577) User null is not allowed to view configurationEntry: Access denied for entry Localhost of mongodb_instance
10:25:16,583 SEVERE [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Process failed: com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.: com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.
at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:169)
at com.rapidminer.operator.Operator.execute(Operator.java:1005) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:807) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.8.0_51]
at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.operator.Operator.execute(Operator.java:1005) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1205) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1101) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1054) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.Process.run(Process.java:1049) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at de.rapidanalytics.execution.AbstractProcessExecutor.runProcessNow(AbstractProcessExecutor.java:91) [rapidminer-server-ejb.jar:]
at de.rapidanalytics.execution.SimpleProcessExecutor.execute(SimpleProcessExecutor.java:31) [rapidminer-server-ejb.jar:]
at de.rapidanalytics.execution.ProcessExecutionDispatcher$ProcessRunnable.run(ProcessExecutionDispatcher.java:89) [rapidminer-server-ejb.jar:]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_51]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_51]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_51]
Caused by: com.rapidminer.tools.config.ConfigurationException: No such configured object of name Localhost of mongodb_instance
at com.rapidminer.server.tools.config.ServerConfigurationManager.logAccessViolationAndThrow(ServerConfigurationManager.java:102) [rapidminer-server-ejb.jar:]
at com.rapidminer.server.tools.config.ServerConfigurationManager.checkAccess(ServerConfigurationManager.java:97) [rapidminer-server-ejb.jar:]
at com.rapidminer.tools.config.ConfigurationManager.lookup(ConfigurationManager.java:340) [rapidminer-studio-core-7.4.0.jar:7.4.0]
at com.rapidminer.extension.nosql.operator.mongodb.MongoDBConnector.doWork(MongoDBConnector.java:161)
... 18 more
10:25:16,584 SEVERE [com.rapidminer.Process.106] (scheduledprocess_1491747916577) Here: Process[1] (Process)
subprocess 'Main Process'
==> +- Read MongoDB[1] (Read MongoDB)
+- JSON To Data[0] (JSON To Data)
+- Select Attributes[0] (Select Attributes)
+- Rename[0] (Rename)
+- Generate Concatenation[0] (Generate Concatenation)
10:25:16,584 INFO [de.rapidanalytics.execution.ProcessExecutionDispatcher] (scheduledprocess_1491747916577) Completed execution of process /home/admin/processes/Retrieve Data
10:25:16,586 INFO [de.rapidanalytics.ejb.ProcessJobEJBImpl] (scheduledprocess_1491747916577) Marking process 27 as completed with exception com.rapidminer.operator.UserError: The database configuration 'Localhost' is invalid.
10:25:16,595 INFO [de.rapidanalytics.execution.LogRecordCollector] (scheduledprocess_1491747916577) Saving records now
10:25:16,617 INFO [de.rapidanalytics.execution.LogRecordCollector] (scheduledprocess_1491747916577) Completed saving records

 

RMStaff

Re: how to properly connect to local mongo DB?

Hi yunior_betancou,

 

thank you for sharing the log file. Especially the User null part is interesting.

We are investigating the problem and hope to be able to fix it soon.

 

Regarding your other question:

The noSQL extension is not publicly available.

 

Best regards,

Edin