Yahoo Historical Stock Price Operator is not returing data

pradeeppradeep Member Posts: 8 Contributor I
edited November 2018 in Help

Yahoo Historical Data is returning empty values in rapidminer, is there any bug ?

I was able to download rawdat from yahoo finance website, but this operator is unable to do it.

Tagged:

Best Answers

Answers

  • keith_drakekeith_drake Member Posts: 11 Contributor II

    Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

    Anyone else?

  • keith_drakekeith_drake Member Posts: 11 Contributor II

    Yes, I noticed the same thing. And another app I use to download data from Yahoo (YLoader) won't work either -- it throws a "HTTP response error 301"

    Anyone else?

  • VayaoVayao Member Posts: 1 Contributor I

    I'm suffering this malfunction also. Please can someone help?

    Thanks a lot

     

  • keith_drakekeith_drake Member Posts: 11 Contributor II
    I've confirmed with another software vendor the problem is with Yahoo changing their protocol: "The main fix in version 4.10 is for error 301 when downloading Yahoo data, which was due to a change in the Yahoo protocol."

    The Yahoo Historical Stock Data operator is in the Finance and Economic Data extension provided by BroadReachAnalytics. Visiting their URL provided in the RM documentation (http://www.broadreachanalytics.com/) it appears they are no longer active.

    RapidMiner: Is there any opportunity to update this operator? Thanks.
  • gglsbmtgglsbmt Member Posts: 1 Contributor I

    Hello my friend thankyou so much

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Yes, I can confirm that this is no longer working and BroadReachAnalytics appears to be gone.

     

    With RapidMiner, you can always find a work around and here it is. 

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
    <parameter key="resource_type" value="URL"/>
    <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:GOOGL&amp;amp;ei=u5z8WJGtC8ejeeqWgbgJ&amp;amp;output=csv"/>
    </operator>
    <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
    <parameter key="column_separators" value=","/>
    <list key="annotations"/>
    <list key="data_set_meta_data_information"/>
    </operator>
    <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
    <connect from_op="Read CSV" from_port="output" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Hi...I have moved over to "openexchangerates.com" which has a nice API. I built a simple app for an accounting client using the API on RM server. Send me a PM if you want to see it.

    Scott
  • qwertz2qwertz2 Member Posts: 49 Guru

     

    Hi Thomas,

     

    Thanks for that work around. I can confirm that it works pretty well with a single symbol. However, is there also a way to load multiple symbols at a time?

     

    From your tutorials I know that it is possible to append several single symbols but in this case I want to download about 20. Searching the internet I found a site (LINK) which describes a select query but I couldn't manage to get this working in Rapidminer... :(

     

     

    Kind regards

    Sachs

  • pradeeppradeep Member Posts: 8 Contributor I

    Hello Sachs,

     

     

    I am new for Rapidminer,

    could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, which has been provided Thomas,

    using it in Rapidminer?

     

    Thanks

  • pradeeppradeep Member Posts: 8 Contributor I

    Hello Thomas,

     

     

    Thank you for providing the work around solution,

     

     

    I am new for Rapidminer,

    could you please help me out in using 'xml code' for Yahoo Historical Stock price operator, using it in Rapidminer?

     

    Thanks

    Pradeep.

  • keith_drakekeith_drake Member Posts: 11 Contributor II
    Hello Thomas ... thanks for the solution. How do I implement the XML script? I copied it into the Execute Script operator and got an error (see attached). Thanks.
  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Hi Sachs,

     

    Piece of cake. Just make a text file with the header Ticker and then a new symbol on each line. Then use this process below. You should save them seperately in a repository by appending %{symbol}_stock

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File" width="90" x="45" y="34">
    <parameter key="filename" value="C:\Users\ThomasOtt\Desktop\Ticker File.txt"/>
    </operator>
    <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="179" y="34">
    <list key="annotations"/>
    <list key="data_set_meta_data_information"/>
    </operator>
    <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="313" y="34">
    <parameter key="macro" value="num"/>
    <list key="additional_macros"/>
    </operator>
    <operator activated="true" class="concurrency:loop" compatibility="7.4.000" expanded="true" height="82" name="Loop" width="90" x="514" y="34">
    <parameter key="number_of_iterations" value="%{num}"/>
    <process expanded="true">
    <operator activated="true" class="extract_macro" compatibility="7.4.000" expanded="true" height="68" name="Extract Macro" width="90" x="112" y="34">
    <parameter key="macro" value="symbol"/>
    <parameter key="macro_type" value="data_value"/>
    <parameter key="attribute_name" value="Ticker"/>
    <parameter key="example_index" value="%{iteration}"/>
    <list key="additional_macros"/>
    </operator>
    <operator activated="true" class="open_file" compatibility="7.4.000" expanded="true" height="68" name="Open File (2)" width="90" x="112" y="136">
    <parameter key="resource_type" value="URL"/>
    <parameter key="filename" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
    <parameter key="url" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
    </operator>
    <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV (2)" width="90" x="447" y="85">
    <parameter key="csv_file" value="http://www.google.com/finance/historical?q=NASDAQ:%{symbol}&amp;amp;ei=LRD-WJmhHIOrmgH_4KToAg&amp;amp;output=csv"/>
    <parameter key="column_separators" value=","/>
    <list key="annotations"/>
    <list key="data_set_meta_data_information"/>
    </operator>
    <connect from_port="input 1" to_op="Extract Macro" to_port="example set"/>
    <connect from_op="Open File (2)" from_port="file" to_op="Read CSV (2)" to_port="file"/>
    <connect from_op="Read CSV (2)" from_port="output" to_port="output 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    <portSpacing port="sink_output 2" spacing="0"/>
    </process>
    </operator>
    <connect from_op="Open File" from_port="file" to_op="Read CSV" to_port="file"/>
    <connect from_op="Read CSV" from_port="output" to_op="Extract Macro (2)" to_port="example set"/>
    <connect from_op="Extract Macro (2)" from_port="example set" to_op="Loop" to_port="input 1"/>
    <connect from_op="Loop" from_port="output 1" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>
  • pradeeppradeep Member Posts: 8 Contributor I

    Thank you Sachs.

  • pradeeppradeep Member Posts: 8 Contributor I

    Thank you @Thomas_Ott.

  • qwertz2qwertz2 Member Posts: 49 Guru

     

    Hi Thomas,

     

    This already looks very promising - and brings the advantage to control stock symbols in separate external file. Pretty cool!

     

    Now I am struggling with two more problems:

    1) The process provides a collection but I rather want to have each symbol in its own column in a single example set / table.

    2) I tried to access the yahoo web api with the open file operator:

    http://ichart.finance.yahoo.com/table.csv?s=AAPL&d=4&e=10&f=2016&g=d&a=11&b=12&c=2015&ignore=table.csv

    But I couldn't manage to make the yahoo links work though they would do fine a web browser.

     

    Lookling forward to getting some advice...

     

     

    Cheers

    Sachs

     

     

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Yahoo did something to bork the whole thing recently, so I would default to Google right now. 


    With respect to putting it all in a table, I'd have to think about it but it would include using a Rename by Replacing so you can do AAPL_Close and XOM_Close and then something to merge them all together, maybe with a combination of a Remember and Recall operator but I need go AFK know. 

Sign In or Register to comment.