ANNOUNCEMENT: RAPIDMINER 9.1 BETA HAS BEEN RELEASED TODAY!   PLEASE DOWNLOAD AND GIVE FEEDBACK. ENJOY AND HAPPY RAPIDMINING!   -- @sgenzer – Community Manager

Real-Time Financial Data via Alpha Venture API (alternative to Yahoo Finance API)

sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager
edited November 8 in Knowledge Base

 So I believe I have finally found a decent alternative to the old Yahoo Finance API so you can bring financial market data directly into RapidMiner for data analysis.  A company called Alpha Vantage has developed an API that appears to perform the same functions. 

You can get a free API key on their website, and you can use the process below directly into RapidMiner 7.6:

<?xml version="1.0" encoding="UTF-8"?><process version="7.6.000">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.6.000" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="set_macros" compatibility="7.6.000" expanded="true" height="68" name="Set Macros" width="90" x="45" y="238">
<list key="macros">
<parameter key="apiKey" value="your-KEY-here"/>
<parameter key="tickerSymbol" value="AAPL"/>
<parameter key="function" value="TIME_SERIES_DAILY"/>
<parameter key="outputSize" value="compact"/>
</list>
</operator>
<operator activated="true" class="generate_data_user_specification" compatibility="7.6.000" expanded="true" height="68" name="Generate Data by User Specification" width="90" x="179" y="238">
<list key="attribute_values"/>
<list key="set_additional_roles"/>
</operator>
<operator activated="true" class="web:enrich_data_by_webservice" compatibility="7.3.000" expanded="true" height="68" name="Enrich Data by Webservice" width="90" x="313" y="238">
<parameter key="query_type" value="Regular Expression"/>
<list key="string_machting_queries"/>
<list key="regular_expression_queries">
<parameter key="foo" value=".*"/>
</list>
<list key="regular_region_queries"/>
<list key="xpath_queries"/>
<list key="namespaces"/>
<list key="index_queries"/>
<list key="jsonpath_queries"/>
<parameter key="url" value="https://www.alphavantage.co/query?function=%{function}&amp;outputsize=%{outputSize}&amp;symbol=%{tickerSymbol}&amp;apikey=%{apiKey}"/>
<list key="request_properties"/>
</operator>
<operator activated="true" class="subprocess" compatibility="7.6.000" expanded="true" height="82" name="Subprocess" width="90" x="447" y="238">
<process expanded="true">
<operator activated="true" class="text:data_to_documents" compatibility="7.5.000" expanded="true" height="68" name="Data to Documents" width="90" x="45" y="34">
<parameter key="select_attributes_and_weights" value="true"/>
<list key="specify_weights">
<parameter key="foo" value="1.0"/>
</list>
</operator>
<operator activated="true" class="text:combine_documents" compatibility="7.5.000" expanded="true" height="82" name="Combine Documents" width="90" x="179" y="34"/>
<operator activated="true" class="text:json_to_data" compatibility="7.5.000" expanded="true" height="82" name="JSON To Data" width="90" x="313" y="34"/>
<operator activated="true" class="numerical_to_real" compatibility="7.6.000" expanded="true" height="82" name="Numerical to Real" width="90" x="447" y="136">
<parameter key="attribute_filter_type" value="regular_expression"/>
<parameter key="regular_expression" value="Time.*"/>
</operator>
<operator activated="true" class="de_pivot" compatibility="7.6.000" expanded="true" height="82" name="De-Pivot" width="90" x="514" y="34">
<list key="attribute_name">
<parameter key="value" value="Time.*"/>
</list>
<parameter key="index_attribute" value="date"/>
<parameter key="create_nominal_index" value="true"/>
</operator>
<operator activated="true" class="split" compatibility="7.6.000" expanded="true" height="82" name="Split" width="90" x="648" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="date"/>
<parameter key="split_pattern" value="[.]\s"/>
</operator>
<operator activated="true" class="replace" compatibility="7.6.000" expanded="true" height="82" name="Replace" width="90" x="782" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="date_1"/>
<parameter key="replace_what" value="Time Series \(Daily\)[.]"/>
</operator>
<operator activated="true" class="replace" compatibility="7.6.000" expanded="true" height="82" name="Replace (2)" width="90" x="916" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="date_1"/>
<parameter key="replace_what" value="[.][0-9]"/>
</operator>
<operator activated="true" class="pivot" compatibility="7.6.000" expanded="true" height="82" name="Pivot" width="90" x="1050" y="34">
<parameter key="group_attribute" value="date_1"/>
<parameter key="index_attribute" value="date_2"/>
<parameter key="consider_weights" value="false"/>
</operator>
<operator activated="true" class="rename_by_replacing" compatibility="7.6.000" expanded="true" height="82" name="Rename by Replacing" width="90" x="1184" y="34">
<parameter key="replace_what" value="value[_]"/>
</operator>
<operator activated="true" class="rename_by_replacing" compatibility="7.6.000" expanded="true" height="82" name="Rename by Replacing (2)" width="90" x="1318" y="34">
<parameter key="replace_what" value="Meta Data[.][0-9][.]\s"/>
</operator>
<operator activated="true" class="rename" compatibility="7.6.000" expanded="true" height="82" name="Rename" width="90" x="1452" y="34">
<parameter key="old_name" value="date_1"/>
<parameter key="new_name" value="Date"/>
<list key="rename_additional_attributes"/>
</operator>
<operator activated="true" class="nominal_to_date" compatibility="7.6.000" expanded="true" height="82" name="Nominal to Date" width="90" x="1586" y="34">
<parameter key="attribute_name" value="Date"/>
<parameter key="date_format" value="yyyy-MM-dd"/>
</operator>
<operator activated="true" class="set_role" compatibility="7.6.000" expanded="true" height="82" name="Set Role" width="90" x="1720" y="34">
<parameter key="attribute_name" value="Date"/>
<parameter key="target_role" value="id"/>
<list key="set_additional_roles"/>
</operator>
<connect from_port="in 1" to_op="Data to Documents" to_port="example set"/>
<connect from_op="Data to Documents" from_port="documents" to_op="Combine Documents" to_port="documents 1"/>
<connect from_op="Combine Documents" from_port="document" to_op="JSON To Data" to_port="documents 1"/>
<connect from_op="JSON To Data" from_port="example set" to_op="Numerical to Real" to_port="example set input"/>
<connect from_op="Numerical to Real" from_port="example set output" to_op="De-Pivot" to_port="example set input"/>
<connect from_op="De-Pivot" from_port="example set output" to_op="Split" to_port="example set input"/>
<connect from_op="Split" from_port="example set output" to_op="Replace" to_port="example set input"/>
<connect from_op="Replace" from_port="example set output" to_op="Replace (2)" to_port="example set input"/>
<connect from_op="Replace (2)" from_port="example set output" to_op="Pivot" to_port="example set input"/>
<connect from_op="Pivot" from_port="example set output" to_op="Rename by Replacing" to_port="example set input"/>
<connect from_op="Rename by Replacing" from_port="example set output" to_op="Rename by Replacing (2)" to_port="example set input"/>
<connect from_op="Rename by Replacing (2)" from_port="example set output" to_op="Rename" to_port="example set input"/>
<connect from_op="Rename" from_port="example set output" to_op="Nominal to Date" to_port="example set input"/>
<connect from_op="Nominal to Date" from_port="example set output" to_op="Set Role" to_port="example set input"/>
<connect from_op="Set Role" from_port="example set output" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">clean up</description>
</operator>
<connect from_op="Generate Data by User Specification" from_port="output" to_op="Enrich Data by Webservice" to_port="Example Set"/>
<connect from_op="Enrich Data by Webservice" from_port="ExampleSet" to_op="Subprocess" to_port="in 1"/>
<connect from_op="Subprocess" from_port="out 1" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
<description align="center" color="yellow" colored="false" height="201" resized="false" width="640" x="35" y="18">Alpha Vantage API(alphavantage.co)&lt;br&gt;Author: Scott Genzer&lt;br&gt;Published: &amp;#8206;&amp;#8206;8-25-2017&lt;br&gt;Link: http://community.rapidminer.com/t5/RapidMiner-Studio-Knowledge-Base/Real-Time-Financial-Data-via-Alpha-Venture-API-alternative-to/ta-p/41119&lt;br&gt;&lt;br&gt;Note: For each process below, enter your Alpha Vantage API key, ticker symbol and other request elements in the Set Macros operator before running (see https://www.alphavantage.co/documentation/)&lt;br&gt;&lt;br&gt;</description>
</process>
</operator>
</process>

 Feedback welcome.  Enjoy!

 

Scott

Telcontar120CraigBostonUSAhughesfleming68luc_bartkowskisg0

Comments

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 812   Unicorn

    Wonderful.  Works like a charm!

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • luc_bartkowskiluc_bartkowski Member Posts: 44  Guru

    Hi Scott,

     

    I implemented your XML into a subprocess and building block. But I don't understand the result sets of this process.

    See the following pictures. Each example set from the Alpha Vantage process contains different attributes. Can you explain why?

     

    TSLA3.jpegFor any subsequent operator the Alpha Vantage process delivers an example set with just 3 attributes

    TSLA2.jpegAlpha Vantage result example set written to CSV delivers a completely other example set.

    TSLA1.jpegAlpha Vantage example set connected to a process output port.

    I don't understand. What is happening here?

    Why does the result example set of the Alpha Vantage process shows 3 types of formatting dependant of the subsequent operator?

     

    Best regards,

    Luc

    sg0
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    Hello Luc,

     

    I think what is happening here is that the metadata is not being pushed through.  This is typical for Enrich Data via Webservice as RapidMiner has no idea what attributes are coming its way.  It's only picking up the ones that I rename manually afterwards: Date, date_3 and value.  Don't worry - all your attributes are there and you can "select" as many/few as you want.  It's just that you may need to add them manually rather than the nice arrow in Select Attributes.

     

    Screen Shot 2017-10-08 at 9.41.11 PM.png

     

    Scott

  • luc_bartkowskiluc_bartkowski Member Posts: 44  Guru

    Thanks Scott ?

  • luc_bartkowskiluc_bartkowski Member Posts: 44  Guru

    To other users of this building block:

    To get TIME_SERIES_INTRADAY example sets, don't forget to adjust the url REGEX in the "Enrich Data by Webservice (3)" operator that compiles the URL string. For TIME_SERIES_INTRADAY examples sets the required "interval" parameter and related value are neccesary in the url. To accomplish this add an extra Macro "requiredInterval" like this:

     

    requiredInterval.jpegAdd requiredInterval Macro

    And adjust the url REGEX in the "Enrich Data by Webservice (3)" operator like this:

    https://www.alphavantage.co/query?function=%{function}&outputsize=%{outputSize}&symbol=%{tickerSymbol}&apikey=%{apiKey}&interval=%{requiredInterval}

    sg0
  • websiteguywebsiteguy Member Posts: 19 Contributor I

    @luc_bartkowski

    Hi Luke , Im a novice, could you tell me please or post a sample how to ajust the date time elements of this process when applying the INTRADAY

    Ive made the changes as specified the process fails. Any help advice appreciated. Regards Lee

    fail.pngfail

  • luc_bartkowskiluc_bartkowski Member Posts: 44  Guru

    Hi Lee,
    Sorry for my late response.
    I'm sorry to say but I didn't continue to use the process of @sgenzer.

     

    I created a database using MySQL for daily and intraday stock prices.

    I wrote a python program that continuously updates this database.

    Accessing stock price data from RapidMiner is then no more than accessing tables/views from this database.

  • websiteguywebsiteguy Member Posts: 19 Contributor I

    @luc_bartkowski

     

    Hi Luc sorry for my even later reply :)

     

    Do you have an intraday model working I could borrow :/...

     

    I got it working on the daily, but can seem to get it to run on the intraday, prices, mabey its the formatting of the date idk.

     

    regards lee

     

     

  • luc_bartkowskiluc_bartkowski Member Posts: 44  Guru

    Hi Lee,

     

    The thing with the Alpha vantage API is that Intraday returns an extra attribute, the Interval e.g. 5min, 10min, 15min etc. @sgenzer's RM process has to be extended to support this extra parameter.

    I haven't a RM intraday model. I discovered that AV doesn't always return the correct answer, sometimes I receive an API error instead of a resultset. In my python script I included a retry function if such response is received. Secondly: I want to store the data in a database. That requires CRUD functionality that I also included in my python script. Sorry I can't help you in response to your request.

     

  • 1032956310329563 Member Posts: 1 Contributor I

    Hi,

    Is it possible to get step by step tutorial on this?

     

    Kind regards,

    Mareks

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    @10329563 - so it's even easier now than it was when I wrote that article. :) Just do the following:

     

    1. Go to the Alpha Vantage website and get an API key.

    2. Go to the Alpha Vantage Process which is already in the Community Repository by clicking on this link.

    3. Follow these steps:

     

    Screen Shot 2018-10-22 at 10.58.46 AM.png

     

    Screen Shot 2018-10-22 at 10.59.56 AM.png

     

    Click "Apply" and run the process. Done.

     

    Screen Shot 2018-10-22 at 11.03.59 AM.png

     

    Happy RapidMining!

     

    Scott

     

    sg0
  • cleisnercleisner Member Posts: 4 Contributor I

    Hi, has anyone here ever accessed Accurint (https://secure.accurint.com/app/bps/main) via RapidMiner? This is the best fee-based site for people searches and many other public records. Any tips at all would be greatly appreciated, as I'm just a beginner at this. 

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    hi @cleisner - nope not to my knowledge. Happy to collaborate on making it connect if you are a subscriber.

     

    Scott

     

  • cleisnercleisner Member Posts: 4 Contributor I

    Hi, @sgenzer. It looks like I won't be using Accurint after all. It turns out that this is a highly regulated data source and I can't use it as planned.  

  • bjoernbalubjoernbalu Member Posts: 3 Contributor I

    Hello Scott,

    thanks for uploading the process. I still have the problem to get the attributes the useable.

    For example I want to calculate the moving average of the close data. I´m not able to select the data.

    Thanks for your help

    Björn

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    hello @bjoernbalu can you post your XML so I can see what you're doing?

  • bjoernbalubjoernbalu Member Posts: 3 Contributor I

    Hello @sgenzer

    here is the XML. The API is removed.

    <?xml version="1.0" encoding="UTF-8"?><process version="9.0.003">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="9.0.003" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="set_macros" compatibility="9.0.003" expanded="true" height="68" name="Set Macros" origin="GENERATED_COMMUNITY" width="90" x="112" y="34">
    <list key="macros">
    <parameter key="apiKey" value="API"/>
    <parameter key="tickerSymbol" value="AAPL"/>
    <parameter key="function" value="TIME_SERIES_DAILY_ADJUSTED"/>
    <parameter key="outputSize" value="compact"/>
    <parameter key="datatype" value="csv"/>
    </list>
    <description align="center" color="purple" colored="true" width="126">YOU WILL NEED TO PUT YOUR ALPHA VANTAGE API KEY IN THE FIRST MACRO BEFORE RUNNING!</description>
    </operator>
    <operator activated="true" class="generate_data_user_specification" compatibility="9.0.003" expanded="true" height="68" name="Generate Data by User Specification" origin="GENERATED_COMMUNITY" width="90" x="246" y="34">
    <list key="attribute_values"/>
    <list key="set_additional_roles"/>
    <description align="center" color="transparent" colored="false" width="126">This is blank - ignore me</description>
    </operator>
    <operator activated="true" class="web:enrich_data_by_webservice" compatibility="7.3.000" expanded="true" height="68" name="Enrich Data by Webservice" origin="GENERATED_COMMUNITY" width="90" x="447" y="34">
    <parameter key="query_type" value="Regular Expression"/>
    <list key="string_machting_queries"/>
    <list key="regular_expression_queries">
    <parameter key="foo" value=".*"/>
    </list>
    <list key="regular_region_queries"/>
    <list key="xpath_queries"/>
    <list key="namespaces"/>
    <list key="index_queries"/>
    <list key="jsonpath_queries"/>
    <parameter key="url" value="https://www.alphavantage.co/query?function=%{function}&amp;outputsize=%{outputSize}&amp;symbol=%{tickerSymbol}&amp;apikey=%{apiKey}"/>
    <list key="request_properties"/>
    <description align="center" color="transparent" colored="false" width="126">Alpha Vantage API GET request</description>
    </operator>
    <operator activated="true" class="subprocess" compatibility="9.0.003" expanded="true" height="103" name="Subprocess" origin="GENERATED_COMMUNITY" width="90" x="648" y="34">
    <process expanded="true">
    <operator activated="true" class="text:data_to_documents" compatibility="8.1.000" expanded="true" height="68" name="Data to Documents" origin="GENERATED_COMMUNITY" width="90" x="45" y="34">
    <parameter key="select_attributes_and_weights" value="true"/>
    <list key="specify_weights">
    <parameter key="foo" value="1.0"/>
    </list>
    </operator>
    <operator activated="true" class="text:combine_documents" compatibility="8.1.000" expanded="true" height="82" name="Combine Documents" origin="GENERATED_COMMUNITY" width="90" x="179" y="34"/>
    <operator activated="true" class="text:json_to_data" compatibility="8.1.000" expanded="true" height="82" name="JSON To Data" origin="GENERATED_COMMUNITY" width="90" x="313" y="34"/>
    <operator activated="true" class="numerical_to_real" compatibility="9.0.003" expanded="true" height="82" name="Numerical to Real" origin="GENERATED_COMMUNITY" width="90" x="447" y="34">
    <parameter key="attribute_filter_type" value="regular_expression"/>
    <parameter key="regular_expression" value="Time.*"/>
    </operator>
    <operator activated="true" class="de_pivot" compatibility="9.0.003" expanded="true" height="82" name="De-Pivot" origin="GENERATED_COMMUNITY" width="90" x="581" y="34">
    <list key="attribute_name">
    <parameter key="value" value="Time.*"/>
    </list>
    <parameter key="index_attribute" value="date"/>
    <parameter key="create_nominal_index" value="true"/>
    </operator>
    <operator activated="true" class="split" compatibility="9.0.003" expanded="true" height="82" name="Split" origin="GENERATED_COMMUNITY" width="90" x="715" y="34">
    <parameter key="attribute_filter_type" value="single"/>
    <parameter key="attribute" value="date"/>
    <parameter key="split_pattern" value="[.]\s"/>
    </operator>
    <operator activated="true" class="replace" compatibility="9.0.003" expanded="true" height="82" name="Replace" origin="GENERATED_COMMUNITY" width="90" x="849" y="34">
    <parameter key="attribute_filter_type" value="single"/>
    <parameter key="attribute" value="date_1"/>
    <parameter key="replace_what" value="Time Series \(Daily\)[.]"/>
    </operator>
    <operator activated="true" class="replace" compatibility="9.0.003" expanded="true" height="82" name="Replace (2)" origin="GENERATED_COMMUNITY" width="90" x="983" y="34">
    <parameter key="attribute_filter_type" value="single"/>
    <parameter key="attribute" value="date_1"/>
    <parameter key="replace_what" value="[.][0-9]"/>
    </operator>
    <operator activated="true" class="pivot" compatibility="9.0.003" expanded="true" height="82" name="Pivot" origin="GENERATED_COMMUNITY" width="90" x="1117" y="34">
    <parameter key="group_attribute" value="date_1"/>
    <parameter key="index_attribute" value="date_2"/>
    <parameter key="consider_weights" value="false"/>
    </operator>
    <operator activated="true" class="rename_by_replacing" compatibility="9.0.003" expanded="true" height="82" name="Rename by Replacing" origin="GENERATED_COMMUNITY" width="90" x="1251" y="34">
    <parameter key="replace_what" value="value[_]"/>
    </operator>
    <operator activated="true" class="rename_by_replacing" compatibility="9.0.003" expanded="true" height="82" name="Rename by Replacing (2)" origin="GENERATED_COMMUNITY" width="90" x="1385" y="34">
    <parameter key="replace_what" value="Meta Data[.][0-9][.]\s"/>
    </operator>
    <operator activated="true" class="rename" compatibility="9.0.003" expanded="true" height="82" name="Rename" origin="GENERATED_COMMUNITY" width="90" x="1519" y="34">
    <parameter key="old_name" value="date_1"/>
    <parameter key="new_name" value="Date"/>
    <list key="rename_additional_attributes"/>
    </operator>
    <operator activated="true" class="nominal_to_date" compatibility="9.0.003" expanded="true" height="82" name="Nominal to Date" origin="GENERATED_COMMUNITY" width="90" x="1653" y="34">
    <parameter key="attribute_name" value="Date"/>
    <parameter key="date_format" value="yyyy-MM-dd"/>
    </operator>
    <operator activated="true" class="set_role" compatibility="9.0.003" expanded="true" height="82" name="Set Role" origin="GENERATED_COMMUNITY" width="90" x="1787" y="34">
    <parameter key="attribute_name" value="Date"/>
    <parameter key="target_role" value="id"/>
    <list key="set_additional_roles"/>
    </operator>
    <operator activated="true" class="select_attributes" compatibility="9.0.003" expanded="true" height="82" name="Select Attributes" origin="GENERATED_COMMUNITY" width="90" x="1921" y="34">
    <parameter key="attributes" value="|open|close|low|high"/>
    </operator>
    <operator activated="true" class="multiply" compatibility="9.0.003" expanded="true" height="103" name="Multiply (2)" width="90" x="2055" y="34"/>
    <operator activated="true" class="series:moving_average" compatibility="7.4.000" expanded="true" height="82" name="Moving Average" width="90" x="2189" y="85">
    <parameter key="attribute_name" value="close"/>
    <parameter key="window_width" value="100"/>
    </operator>
    <connect from_port="in 1" to_op="Data to Documents" to_port="example set"/>
    <connect from_op="Data to Documents" from_port="documents" to_op="Combine Documents" to_port="documents 1"/>
    <connect from_op="Combine Documents" from_port="document" to_op="JSON To Data" to_port="documents 1"/>
    <connect from_op="JSON To Data" from_port="example set" to_op="Numerical to Real" to_port="example set input"/>
    <connect from_op="Numerical to Real" from_port="example set output" to_op="De-Pivot" to_port="example set input"/>
    <connect from_op="De-Pivot" from_port="example set output" to_op="Split" to_port="example set input"/>
    <connect from_op="Split" from_port="example set output" to_op="Replace" to_port="example set input"/>
    <connect from_op="Replace" from_port="example set output" to_op="Replace (2)" to_port="example set input"/>
    <connect from_op="Replace (2)" from_port="example set output" to_op="Pivot" to_port="example set input"/>
    <connect from_op="Pivot" from_port="example set output" to_op="Rename by Replacing" to_port="example set input"/>
    <connect from_op="Rename by Replacing" from_port="example set output" to_op="Rename by Replacing (2)" to_port="example set input"/>
    <connect from_op="Rename by Replacing (2)" from_port="example set output" to_op="Rename" to_port="example set input"/>
    <connect from_op="Rename" from_port="example set output" to_op="Nominal to Date" to_port="example set input"/>
    <connect from_op="Nominal to Date" from_port="example set output" to_op="Set Role" to_port="example set input"/>
    <connect from_op="Set Role" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
    <connect from_op="Select Attributes" from_port="example set output" to_op="Multiply (2)" to_port="input"/>
    <connect from_op="Multiply (2)" from_port="output 1" to_port="out 1"/>
    <connect from_op="Multiply (2)" from_port="output 2" to_op="Moving Average" to_port="example set input"/>
    <connect from_op="Moving Average" from_port="example set output" to_port="out 2"/>
    <portSpacing port="source_in 1" spacing="0"/>
    <portSpacing port="source_in 2" spacing="0"/>
    <portSpacing port="sink_out 1" spacing="0"/>
    <portSpacing port="sink_out 2" spacing="0"/>
    <portSpacing port="sink_out 3" spacing="0"/>
    </process>
    <description align="center" color="transparent" colored="false" width="126">JSON parsing and more post-processing</description>
    </operator>
    <connect from_op="Generate Data by User Specification" from_port="output" to_op="Enrich Data by Webservice" to_port="Example Set"/>
    <connect from_op="Enrich Data by Webservice" from_port="ExampleSet" to_op="Subprocess" to_port="in 1"/>
    <connect from_op="Subprocess" from_port="out 1" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>
    sg0
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    hmm not sure...can you show what the ExampleSet looks like right at that Multiply operator (put a breakpoint).

  • bjoernbalubjoernbalu Member Posts: 3 Contributor I

    Hi,

    there are still 13 regular attributes, but in the list are only 3 attributes (value, date_3, Date) available to use.

    The interesting thing is that everything is working correctly when I storing the data in the repository. After retrieving the attributes are available in the moving average operator. I don´t understand that.

    Thanks for your help and your tutorials.

     

    Björn

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    yes so it is quite common for a RapidMiner process to lose its metadata along the way for various reasons. You can always type the name of the attribute directly in the parameters box even if it does not show up in the pull-down menu:

     

    Screen Shot 2018-10-29 at 4.40.46 PM.png

  • PhilipDPhilipD Member Posts: 2 Contributor I

    Hi, thanks seems to be highly useful..

     

    1) the Alpha Vantage API makes use of other Web Extensions; Web mining and Text processing, be prepared to install those. The Alpha Vantage API appeared in red, it was starightforward enough to click the warning message and follow the steps.

     

    2) When I run it now, only the Date column appears. No price or instrument info. Can't see anything obvious, will try with breakpoints to inspect the data..

     

    Thanks again

    Philip

  • PhilipDPhilipD Member Posts: 2 Contributor I

    Hi, so it looks like the Attributes are not available for selection. Might be a result of the pivot? Don't know enough about this yet to be able to tell. The data is definitely there, right up until Set Role (connect Set Role exa to Out and the result shows exactly what I expect)

     

    I guess I am wondering whether I need the Select Attributes at all.

     

    Will have to see when I try to analyse the data..

     

    Thank you.

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,832  Community Manager

    Thanks @PhilipD. Keep me posted and let me know if I need to make changes to the template.


    Scott

     

Sign In or Register to comment.