Enrich Data by Web Service - how to parse the response

oleksandr_dykleoleksandr_dykle Member Posts: 3 Contributor I
edited November 2018 in Help

Hi,

 

I'm trying to perform following task in RapidMiner - request the data from a web-service, and then parse the response as a data set.

the response from the server is returned as XML and i can do it successfully.

This i was able to successfully do with Enrich Data by Webservice operation.

As a result, i have a table of form

Row No.    InputParameter   Response

1               1111111111          <here is my response in XML>

 

How can i parse now the response from this column and convert it to a data set?

I tried different operations, e.g. Process Documents from Data but it compains about "The example set must contain at least one text attribute".

So, how and where do i set this text attribute?

I tried to Google for some minimal example, but did not find.

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    does your Xpath statement contain /text() ?  that will extrac the text and put it into an attribute column.  You can check out my example with Open Street Map data here: http://www.neuralmarkettrends.com/Extracting-OpenStreetMap-Data-In-RapidMiner/

  • oleksandr_dykleoleksandr_dykle Member Posts: 3 Contributor I

    Hi Thomas,

     

    thank you for replying me! But, my problem is a bit different. In your example, you call web service for each input row, and then just append returned value, queried by XPath.

     

    In my case, i have to make one call, retrieve XML which contains collection of objects, about 20. Now, i need to apply XPath to get these 20 rows, and i cannot do it. It just selects one element from collection and that's it.

    My XML looks like:

    <?xml version='1.0' encoding='UTF-8'?>
    <ArrayOfCreditBureauData xmlns="http://microsoft.com/webservices/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <CreditBureauData>
    <id>1</id>
    <INCOME>70000</INCOME>
    <ts>2013-02-22T07:37:05</ts>
    </CreditBureauData>
    <CreditBureauData>
    <id>2</id>
    <INCOME>50000</INCOME>
    <ts>2014-02-22T07:37:05</ts>
    </CreditBureauData>
    <CreditBureauData>
    <id>3</id>
    <INCOME>60000</INCOME>
    <ts>2015-02-22T07:37:05</ts>
    </CreditBureauData>
    </ArrayOfCreditBureauData>

    So, i need to turn all the entries for CreditBureauData into data rows.

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Ok,

     

    In that case try an Open File operator, set to the URL you want and connect it to a Read XML operator. You'll ping the XML file and then download it all and parse it with the Read XML opreator into an exampleset.

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="open_file" compatibility="7.3.001" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
    <parameter key="resource_type" value="URL"/>
    <parameter key="url" value="http://myURL.com"/>
    </operator>
    <operator activated="true" class="advanced_file_connectors:read_xml" compatibility="7.3.001" expanded="true" height="68" name="Read XML" width="90" x="246" y="34">
    <parameter key="file" value="C:\Users\ThomasOtt\Desktop\Credit.xml"/>
    <parameter key="xpath_for_examples" value="//default:ArrayOfCreditBureauData/default:CreditBureauData"/>
    <enumeration key="xpaths_for_attributes">
    <parameter key="xpath_for_attribute" value="default:id[1]/text()"/>
    <parameter key="xpath_for_attribute" value="default:INCOME[1]/text()"/>
    <parameter key="xpath_for_attribute" value="default:ts[1]/text()"/>
    </enumeration>
    <list key="namespaces">
    <parameter key="default" value="http://microsoft.com/webservices/"/>
    <parameter key="xsd" value="http://www.w3.org/2001/XMLSchema"/>
    <parameter key="xsi" value="http://www.w3.org/2001/XMLSchema-instance"/>
    </list>
    <parameter key="default_namespace" value="http://microsoft.com/webservices/"/>
    <list key="annotations"/>
    <list key="data_set_meta_data_information"/>
    </operator>
    <connect from_op="Open File" from_port="file" to_op="Read XML" to_port="file"/>
    <connect from_op="Read XML" from_port="output" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>
  • oleksandr_dykleoleksandr_dykle Member Posts: 3 Contributor I

    But, i need to POST some parameters in a body to that URL in order to get the data.

    This XML is a response from a web-service.

    Open File does not support POSTing, does it?

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Get Page will work, it has a a POST parameter.

Sign In or Register to comment.