Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Execute Script vs Java Extension coding
sgenzer
Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 
Community Manager
OK I think I am just a glutton for punishment but I keep trying to create a simple extension that executes API GET requests. I thought I had it clinched this weekend using the nice "unirest" library. In fact it works perfectly when using the Execute Script operator:
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="generate_data_user_specification" compatibility="7.6.001" expanded="true" height="68" name="Generate Data by User Specification" width="90" x="179" y="187">
<list key="attribute_values"/>
<list key="set_additional_roles"/>
</operator>
<operator activated="true" class="execute_script" compatibility="7.6.001" expanded="true" height="82" name="Execute Script" width="90" x="313" y="187">
<parameter key="script" value="import com.mashape.unirest.http.*; import com.mashape.unirest.http.exceptions.UnirestException; import com.rapidminer.tools.Ontology; IOObject inputData = input[0]; Attributes attributes = inputData.getAttributes(); Attribute jsonOutput = AttributeFactory.createAttribute("jsonOutput", Ontology.STRING); attributes.addRegular(jsonOutput); inputData.getExampleTable().addAttribute(jsonOutput); for (Example example : inputData) { 	 	HttpResponse<JsonNode> response = Unirest.get("https://community-bitcointy.p.mashape.com/convert/10/USD") 		.header("X-Mashape-Key", "izQxTY0KZ4mshmrBq7DmXQvlC3hcp16709BjsngW8gm71mH39j") 		.header("X-Mashape-Host", "community-bitcointy.p.mashape.com") 		.asJson(); 	JsonNode foo = response.getBody();	 	String jsonText = foo.toString(); 	example.setValue(jsonOutput, jsonText); } return inputData; "/>
<description align="center" color="transparent" colored="false" width="126">this works</description>
</operator>
<connect from_op="Generate Data by User Specification" from_port="output" to_op="Execute Script" to_port="input 1"/>
<connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
However when I try to put this in the "extension template", I can never get this to compile:
package com.rapidminer.apitoolbox;
import java.util.logging.Level;
import com.rapidminer.example.Attribute;
import com.rapidminer.example.Attributes;
import com.rapidminer.example.Example;
import com.rapidminer.example.ExampleSet;
import com.rapidminer.example.table.AttributeFactory;
import com.rapidminer.example.utils.ExampleSetBuilder;
import com.rapidminer.example.utils.ExampleSets;
import com.rapidminer.operator.Operator;
import com.rapidminer.operator.OperatorDescription;
import com.rapidminer.operator.OperatorException;
import com.rapidminer.operator.ports.InputPort;
import com.rapidminer.operator.ports.OutputPort;
import com.rapidminer.operator.ports.metadata.AttributeMetaData;
import com.rapidminer.operator.ports.metadata.ExampleSetMetaData;
import com.rapidminer.operator.ports.metadata.GenerateNewExampleSetMDRule;
import com.rapidminer.operator.ports.metadata.MetaData;
import com.rapidminer.parameter.ParameterType;
import com.rapidminer.parameter.ParameterTypeBoolean;
import com.rapidminer.parameter.ParameterTypeInt;
import com.rapidminer.parameter.UndefinedParameterError;
import com.rapidminer.tools.LogService;
import com.mashape.unirest.http.*;
import com.mashape.unirest.http.exceptions.UnirestException;
import com.rapidminer.tools.Ontology;
/**
* This operator calls the Bitcointy API (via RapidAPI/Mashape)
* and gets the current USD/BTC exchange rate.
*
* @author S. Genzer on 1 Oct 2017
*
*/
public class bitcoin extends Operator {
/**
* @param description
*/
public InputPort exampleSetInput = getInputPorts()
.createPort("exa");
public OutputPort exampleSetOutput = getOutputPorts()
.createPort("exa");
public bitcoin(OperatorDescription description) {
super(description);
}
@Override
public void doWork() throws OperatorException {
LogService.getRoot().log(Level.INFO, "Doing something...");
ExampleSet exampleSet = exampleSetInput.getData(ExampleSet.class);
Attributes attributes = exampleSet.getAttributes();
Attribute jsonOutput = AttributeFactory.createAttribute("jsonOutput", Ontology.STRING);
attributes.addRegular(jsonOutput);
exampleSet.getExampleTable().addAttribute(jsonOutput);
for (Example example : exampleSet) {
HttpResponse<JsonNode> response = Unirest.get("https://community-bitcointy.p.mashape.com/convert/10/USD")
.header("X-Mashape-Key", "key")
.header("X-Mashape-Host", "community-bitcointy.p.mashape.com")
.asJson();
JsonNode foo = response.getBody();
String jsonText = foo.toString();
example.setValue(jsonOutput, jsonText);
}
exampleSetOutput.deliver(exampleSet);
}
}
Help? This is maddening.
Scott
Tagged:
0
Best Answer
-
MartinLiebig
Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,529 RM Data Scientist
Scott,
i've added the class to a branch of operatortoolbox. A few things:
1. Have you added
compile 'com.mashape.unirest:unirest-java:1.3.1'to your gradle dependencies?
2.
response = Unirest.get("https://community-bitcointy.p.mashape.com/convert/10/USD") ...
needs a try catch.
3. you added two different implementations of JsonNode. One from mashable the other from
com.fasterxml.jackson.databind.that screwed it up.
Here the code i was able to compile (but not tested the operator though):
//package com.rapidminer.apitoolbox;
package com.rapidminer.extension.operator.blending;
//import com.fasterxml.jackson.databind.JsonNode;
import com.mashape.unirest.http.*;
import com.mashape.unirest.http.exceptions.UnirestException;
import com.rapidminer.example.Attribute;
import com.rapidminer.example.Attributes;
import com.rapidminer.example.Example;
import com.rapidminer.example.ExampleSet;
import com.rapidminer.example.table.AttributeFactory;
import com.rapidminer.operator.Operator;
import com.rapidminer.operator.OperatorDescription;
import com.rapidminer.operator.OperatorException;
import com.rapidminer.operator.ports.InputPort;
import com.rapidminer.operator.ports.OutputPort;
import com.rapidminer.tools.LogService;
import com.rapidminer.tools.Ontology;
import java.util.logging.Level;
/**
* This operator calls the Bitcointy API (via RapidAPI/Mashape)
* and gets the current USD/BTC exchange rate.
*
* @author S. Genzer on 1 Oct 2017
*
*/
public class BitCoin extends Operator {
/**
* @param description
*/
public InputPort exampleSetInput = getInputPorts()
.createPort("exa");
public OutputPort exampleSetOutput = getOutputPorts()
.createPort("exa");
public BitCoin(OperatorDescription description) {
super(description);
}
@Override
public void doWork() throws OperatorException {
LogService.getRoot().log(Level.INFO, "Doing something...");
ExampleSet exampleSet = exampleSetInput.getData(ExampleSet.class);
Attributes attributes = exampleSet.getAttributes();
Attribute jsonOutput = AttributeFactory.createAttribute("jsonOutput", Ontology.STRING);
attributes.addRegular(jsonOutput);
exampleSet.getExampleTable().addAttribute(jsonOutput);
for (Example example : exampleSet) {
HttpResponse<com.mashape.unirest.http.JsonNode> response = null;
try {
response = Unirest.get("https://community-bitcointy.p.mashape.com/convert/10/USD")
.header("X-Mashape-Key", "key")
.header("X-Mashape-Host", "community-bitcointy.p.mashape.com")
.asJson();
} catch (UnirestException e) {
e.printStackTrace();
}
JsonNode foo = response.getBody();
String jsonText = foo.toString();
example.setValue(jsonOutput, jsonText);
}
exampleSetOutput.deliver(exampleSet);
}
}- Sr. Director Data Solutions, Altair RapidMiner -
Dortmund, Germany0
Answers
@mschmitz, a.k.a. "the man with the cape" comes through again. Thank you. I knew it was some conflict with that JsonNode call but could not find it. Yes I had put the compile statement in the dependencies in build.gradle. Not bad for an EE major, right?
Scott