Options
"Set MongoDB Instance with a macro, is it possible?"
christian_ploch
Member Posts: 3 
Contributor I
Hello everyone,
long time lurker, first time poster.
I make it fast:
I've got a process, which reads with help of the NoSQL Connectors Plugin from my MongoDB.
So far I was able to set a macro, which says what Collection should be read.
Is it also possible to set a macro, which sets the MongoDB "Instance" (e.g. mongodb://localhost:27017/{%db_name})?
I tried several methods, but it's not working here. Anyone an idea? Thanks!
long time lurker, first time poster.
I make it fast:
I've got a process, which reads with help of the NoSQL Connectors Plugin from my MongoDB.
So far I was able to set a macro, which says what Collection should be read.
Is it also possible to set a macro, which sets the MongoDB "Instance" (e.g. mongodb://localhost:27017/{%db_name})?
I tried several methods, but it's not working here. Anyone an idea? Thanks!
1
Best Answer
-
Options
kayman
Member Posts: 662 
Unicorn
You cannot really set the database name itself with a macro (to my awareness), but the branch operator could be one option. Below example picks the right database based on a macro value.<?xml version="1.0" encoding="UTF-8"?><process version="9.1.000"> <context> <input/> <output/> <macros/> </context> <operator activated="true" class="process" compatibility="9.1.000" expanded="true" name="Process"> <parameter key="logverbosity" value="init"/> <parameter key="random_seed" value="2001"/> <parameter key="send_mail" value="never"/> <parameter key="notification_email" value=""/> <parameter key="process_duration_for_mail" value="30"/> <parameter key="encoding" value="SYSTEM"/> <process expanded="true"> <operator activated="true" class="set_macro" compatibility="9.1.000" expanded="true" height="82" name="Set Macro" width="90" x="112" y="34"> <parameter key="macro" value="database"/> <parameter key="value" value="mongo_1"/> </operator> <operator activated="true" class="branch" compatibility="9.1.000" expanded="true" height="82" name="choose" width="90" x="246" y="34"> <parameter key="condition_type" value="expression"/> <parameter key="expression" value="equals(%{database}, "mongo_1")"/> <parameter key="io_object" value="AHCDescribedCluster"/> <parameter key="return_inner_output" value="false"/> <process expanded="true"> <operator activated="true" class="nosql:mongodb_document_reader" compatibility="9.0.000" expanded="true" height="103" name="Read MongoDB" width="90" x="179" y="34"> <parameter key="mongodb_instance" value="mongo_1"/> <parameter key="collection" value="coll1"/> <parameter key="sort_documents" value="false"/> <parameter key="limit_results" value="false"/> <parameter key="skip" value="0"/> </operator> <portSpacing port="source_condition" spacing="0"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_input 1" spacing="0"/> <portSpacing port="sink_input 2" spacing="0"/> </process> <process expanded="true"> <operator activated="true" class="nosql:mongodb_document_reader" compatibility="9.0.000" expanded="true" height="103" name="Read MongoDB (2)" width="90" x="179" y="34"> <parameter key="mongodb_instance" value="mongo_2"/> <parameter key="collection" value="coll1"/> <parameter key="sort_documents" value="false"/> <parameter key="limit_results" value="false"/> <parameter key="skip" value="0"/> </operator> <portSpacing port="source_condition" spacing="0"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_input 1" spacing="0"/> <portSpacing port="sink_input 2" spacing="0"/> </process> </operator> <connect from_op="Set Macro" from_port="through 1" to_op="choose" to_port="condition"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_result 1" spacing="0"/> </process> </operator> </process>Another, slightly more complex option, is to use the execute mongoDB command operator, where your command contains the 'macronised' collection name.
There are a few issues, one is that the command doesn't like linebreaks which makes it a bit difficult to generate, but this can be fixed as in below example. The example is using an aggregate, but it would probably work fine with a more standard command also.
If full flexibility is really needed you could try python instead<?xml version="1.0" encoding="UTF-8"?><process version="9.1.000"> <context> <input/> <output/> <macros/> </context> <operator activated="true" class="process" compatibility="9.1.000" expanded="true" name="Process"> <parameter key="logverbosity" value="init"/> <parameter key="random_seed" value="2001"/> <parameter key="send_mail" value="never"/> <parameter key="notification_email" value=""/> <parameter key="process_duration_for_mail" value="30"/> <parameter key="encoding" value="SYSTEM"/> <process expanded="true"> <operator activated="true" class="set_macro" compatibility="9.1.000" expanded="true" height="68" name="Set Macro" width="90" x="45" y="34"> <parameter key="macro" value="mycoll"/> <parameter key="value" value="some_collection"/> </operator> <operator activated="true" class="text:create_document" compatibility="8.1.000" expanded="true" height="68" name="Create Document (4)" width="90" x="179" y="34"> <parameter key="text" value="{aggregate: "%{mycoll}",pipeline: [ {$match: { "Something": "something" } }, {$project:{ ... }} ],cursor: {},allowDiskUse: true}"/> <parameter key="add label" value="false"/> <parameter key="label_type" value="nominal"/> </operator> <operator activated="true" class="text:replace_tokens" compatibility="8.1.000" expanded="true" height="68" name="Replace Tokens (8)" width="90" x="313" y="34"> <list key="replace_dictionary"> <parameter key="\n" value=" "/> </list> </operator> <operator activated="true" class="nosql:mongodb_command" compatibility="9.0.000" expanded="true" height="68" name="Execute MongoDB Command (5)" width="90" x="447" y="34"> <parameter key="mongodb_instance" value="my_database"/> </operator> <connect from_op="Create Document (4)" from_port="output" to_op="Replace Tokens (8)" to_port="document"/> <connect from_op="Replace Tokens (8)" from_port="document" to_op="Execute MongoDB Command (5)" to_port="command"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_result 1" spacing="0"/> </process> </operator> </process>
8
Answers
But I will give a try with your solutions @kayman
appreciate it! Thanks!
i think it's technically possible. you can hack the xml and just add %{macroName} at the right setting. Not a nice work around though.
+ We are working on it and the problem will be fixed "soon".
BR,
Martin
Dortmund, Germany
If I play with the latter it throws me "Invalid JSON/BSON Object" (when I add the macro in the xml into the connection string). While the process gives me the "collection is not available".
Thanks anyway