Creating JUnit tests for new opearotrs

radoneradone RapidMiner Certified Expert, Member Posts: 74 Guru
edited November 2018 in Help
Hello,
I have the following simple TestOperator and MyIOObject.
I am trying to make JUnit tests for this operator.

TestOperator
public class TestOperator extends Operator {
       // input port
       protected InputPort inParam = getInputPorts().createPort("MyIOObject");

       // output port
       protected OutputPort outParam = getOutputPorts().createPort("MyIOObject");

public TestOperator(OperatorDescription description) {
super(description);
}

public void doWork() throws OperatorException {
                // Read input
                MyIOObject ioObj = inParam.getData();

                // Increment value
                ioObj.setValue( ioObj.getValue() + 1 );

                // Deliver output
                outParam.deliver(ioObj);
}
}

IO object
public class MyIOObject  extends ResultObjectAdapter {
private int value = -1;
       public void setValue(int value) {
           this.value = value;
       }

       public int getValue() {
           return this.value;
       }
}
My idea about JUnit test is such as follows:
public class CirclePixelsExtractorTest extends TestCase {

@Test
public void testMeanValue() throws OperatorException {
// create operator
               // ??????????????????????????????????????????????????????
               // ???????????????????????? 1 ???????????????????????????
               // ??????????????????????????????????????????????????????
OperatorDescription od = null; // (1) ??? HOW TO GET AN OperatorDescription ??;
TestOperator ce = new TestOperator (od);

// create IO input data
MyIOObject  io = new MyIOObject();
io.setValue(5);

// provide IO data as an input

               // ??????????????????????????????????????????????????????
               // ???????????????????????? 2 ???????????????????????????
               // ??????????????????????????????????????????????????????
ce.getInputPorts().getPortByIndex(0). // how to set Data ???;  

// execute the operator
ce.doWork();

// read output
MyIOObject io2 = ce.getOutputPorts().getPortByIndex(0).getData();
int res = io2.getValue();

// validate
assertEquals(6, res);
}
}
Please, could anyone help me with:
1) Is there any simple way how to create my operator object (TestOperator)? To use a constructor I am missing the OperatorDescription and its constructor seem to be quite complicated (requires additional classes).
2) How can I set the IO object (io) to the input port: ce.getInputPorts().getPortByIndex(0)


Thank you very much for any advice.

radone
Tagged:

Answers

  • radoneradone RapidMiner Certified Expert, Member Posts: 74 Guru
    Resolved.
    For anyone dealing with the same problem:
    Answer 1)
    The OperatorDescription can be created by method:
            private OperatorDescription createOperatorDescription() {
    GroupTree gt = GroupTree.findGroup("qName", null);
    return new OperatorDescription("key",
    CirclePixelsExtractor.class, gt, CirclePixelsExtractor.class
    .getClassLoader(), "iconName", null, null);
    }
    and the creating of the operator can be done by:
            TestOperator ce = new TestOperator(createOperatorDescription());
    Answer 2)
    		// provide IO data as an input
    ce.getInputPorts().getPortByIndex(0).receive(io);
  • fischerfischer Member Posts: 439 Maven
    Hi,

    you don't need the OperatorDescription. If RapidMiner.init() was called, you can use

    OperatorService.createOperator(YourOpClass.class).

    Best,
    Simon
  • vc126mvc126m Member Posts: 9 Contributor I

    Hello,I am trying to write Junit test case for my operator. Below is my operator code. Could you please help me how to write the test case for it

    import com.att.cmlp.rmcore.util.UtilV2;
    import com.rapidminer.RapidMiner;
    import com.rapidminer.example.Attribute;
    import com.rapidminer.example.table.AttributeFactory;
    import com.rapidminer.example.table.DataRow;
    import com.rapidminer.example.table.DataRowFactory;
    import com.rapidminer.example.table.MemoryExampleTable;
    import com.rapidminer.operator.*;
    import com.rapidminer.operator.nio.file.BufferedFileObject;
    import com.rapidminer.operator.ports.OutputPort;
    import com.rapidminer.parameter.ParameterType;
    import com.rapidminer.parameter.ParameterTypeString;
    import com.rapidminer.tools.GroupTree;
    import com.rapidminer.tools.Ontology;
    import com.rapidminer.tools.OperatorService;
    import com.rapidminer.tools.Tools;
    import org.apache.commons.httpclient.HttpClient;
    import org.apache.commons.httpclient.HttpException;
    import org.apache.commons.httpclient.HttpMethod;
    import org.apache.commons.httpclient.HttpStatus;
    import org.apache.commons.httpclient.methods.GetMethod;
    import org.apache.commons.httpclient.methods.PostMethod;
    import org.json.simple.JSONObject;
    import org.json.simple.parser.JSONParser;
    import org.json.simple.parser.ParseException;

    import java.io.ByteArrayOutputStream;
    import java.io.IOException;
    import java.nio.charset.StandardCharsets;
    import java.util.ArrayList;
    import java.util.Base64;
    import java.util.List;

    /**
    * Created by vc126m on 8/16/2018.
    */
    public class GetDatasetV2 extends Operator {

    public static final String PARAMETER_DATASET_ENDPOINT_URL = "DATASET_GET_URL";
    public static final String PARAMETER_DATASET_KEY = "DATASET_KEY";

    public OutputPort changesOutputPort = getOutputPorts().createPort("dataset");
    public OutputPort fileOutputPort = getOutputPorts().createPort("file");

    public GetDatasetV2(OperatorDescription description) {
    super(description);
    }

    @Override
    public void doWork() throws OperatorException {
    super.doWork();
    String dataseturl = "/datasetservice/v2/datasets";
    String datasetKey = getParameterAsString(PARAMETER_DATASET_KEY);

    try {
    UtilV2.APIResponse createapiResponse =
    UtilV2.executeGet(getLogger(), getParameters(), UtilV2.CMLP_REST_HEADER_JSON_CONTENT_TYPE_VALUE,
    false, dataseturl+ "/" + datasetKey);
    if (createapiResponse.getStatus() == HttpStatus.SC_OK) {

    getLogger().info("response: " + createapiResponse);
    JSONParser parser = new JSONParser();
    JSONObject jResponseObject = (JSONObject) parser.parse(createapiResponse.getResponse());
    getLogger().info("jResponseObject: " + jResponseObject);
    String DataSOURCEKey = null;
    if (jResponseObject.size() > 0) {

    getLogger().info("response object size: " + jResponseObject.size());
    if (jResponseObject.get("datasourceKey") != null)
    DataSOURCEKey = (jResponseObject.get("datasourceKey").toString());
    }
    try {
    UtilV2.APIResponse createapiResponse2 =
    UtilV2.executeGet(getLogger(), getParameters(), UtilV2.CMLP_REST_HEADER_JSON_CONTENT_TYPE_VALUE,
    false, "/datasourceservice/v2/datasources/"+DataSOURCEKey+"/contents");

    if (createapiResponse2.getStatus() == HttpStatus.SC_OK) {
    getLogger().info("response: " + createapiResponse2);
    String[] dataArray = null;
    String[] jResponse = createapiResponse2.getResponse().split("\r\n");
    String[] s = jResponse[0].toString().split(",");
    Attribute[] outputattributes = new Attribute[s.length];
    for (int i = 0; i < s.length; i++)
    outputattributes[i] = AttributeFactory.createAttribute(s[i], Ontology.STRING);
    final MemoryExampleTable exampleTable = new MemoryExampleTable(outputattributes);
    if (jResponse.length > 0) {
    for (int i = 1; i < jResponse.length; i++) {

    DataRowFactory factory = new DataRowFactory(DataRowFactory.TYPE_DOUBLE_ARRAY, ',');
    getLogger().info("factory: " + factory);
    dataArray = jResponse[i].replaceAll("\"","\\\"").toString().split(",(?=([^\"]*\"[^\"]*\")*[^\"]*$)");
    DataRow row = factory.create(dataArray, outputattributes);
    getLogger().info(i+ "row>>>>>" + row);
    exampleTable.addDataRow(row);
    }
    changesOutputPort.deliver(exampleTable.createExampleSet());
    }

    //File output port
    String resp = createapiResponse2.getResponse();
    ByteArrayOutputStream buffer = new ByteArrayOutputStream();
    for(int i=0;i<resp.length();i++)
    buffer.write(resp.charAt(i));
    BufferedFileObject result1 = new BufferedFileObject(
    buffer.toByteArray());
    fileOutputPort.deliver(result1);

    } else {
    UtilV2.outputError(createapiResponse2,changesOutputPort);
    }
    } catch (HttpException e) {
    e.printStackTrace();
    } catch (IOException e) {
    e.printStackTrace();
    }

    } else {
    UtilV2.outputError(createapiResponse,changesOutputPort);
    }
    } catch (HttpException e) {
    e.printStackTrace();
    } catch (IOException e) {
    e.printStackTrace();
    } catch (ParseException e) {
    e.printStackTrace();
    } catch (Exception e) {
    e.printStackTrace();
    }

    }

    @Override
    public List<ParameterType> getParameterTypes() {
    List<ParameterType> parameterTypes = super.getParameterTypes();
    UtilV2.addDefaultProperties(parameterTypes,false);

    // ParameterTypeString endpointurl = new ParameterTypeString(PARAMETER_DATASET_ENDPOINT_URL, "dataset register url", true);
    // endpointurl.setDefaultValue("/datasetservice/v2/datasets");
    // parameterTypes.add(endpointurl);
    ParameterTypeString datasetkey = new ParameterTypeString(PARAMETER_DATASET_KEY, "dataset key", true);
    // datasetkey.setDefaultValue("m09286_1516907114910_6157390775499062124");
    parameterTypes.add(datasetkey);



    return parameterTypes;
    }
  • Pavithra_RaoPavithra_Rao Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 123 RM Data Scientist

    Hi @vc126m,

     

    Would it be possible to share the java project file or the .jar file of this extension to test it and provide feedback or help here?

     

    Cheers,

  • tftemmetftemme Administrator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 164 RM Research

    Hi @vc126m,

     

    When you want to write JUnitTests for your own operator, you have to initialize RapidMiner itself in the Test Classes. You can do this by calling:

     

     

    RapidMiner.setExecutionMode(ExecutionMode.COMMAND_LINE);
    RapidMiner.init();

    Then you can use the already mentioned OperatorService to create an instance of your operator.

     

     

    GetDatasetV2 getDatasetV2operator = OperatorService.createOperator(GetDatasetV2.class);

    With this instance you can implement any JUnit Test you can think of. If you are not so familiar with JUnit Tests you can find plenty of tutorials in the web (for example I was using this).

     

    What you also can do is installing the Process Testing Extension from the Marketplace to be used in RapidMiner. You can then create UnitTest processes (which you for example can also store in your version control repository), and 'run and store expected results'. After that you can run 'test process' to check if the results of the processes changed.

     

    Hopes this helps

    Fabian

     

Sign In or Register to comment.