🦉 🎤   RapidMiner Wisdom 2020 - CALL FOR SPEAKERS   🦉 🎤

We are inviting all community members to submit proposals to speak at Wisdom 2020 in Boston.


Whether it's a cool RapidMiner trick or a use case implementation, we want to see what you have.
Form link is below and deadline for submissions is November 15. See you in Boston!

CLICK HERE TO GO TO ENTRY FORM

Data for Stock Market Prediction

wesselwessel Member Posts: 537  Guru
Dear All,

Can someone suggest some data to use for stock market prediction?
You can download daily volume and opening / closing prices for just about any stock from Google finance and Yahoo finance.

Furthermore, with a little more effort you can download news headlines.

What more?
What data is typically used by professionals?

Best regards,

Wessel

Answers

  • ronmacronmac Member Posts: 11 Contributor II
    I like to start my data analysis process by collecting various financial data series and run the RM Mutual Information Operator on the series. That helps gets me in the right "stadium" with the data sets to use The Correlation Operator is useful also. Then I work up to getting on the "playing filed" using other RM operators. Lately I have been using the Kmeans Cluster Operator to help find some useful predictive patterns. Yahoo Pipes has some great tools for data collecting.http://pipes.yahoo.com/pipes/ Here is an Excel Macro I use to pull Yahoo data directly into Exce. Then I read the workbook into RM. Makes it fairly easy and fast to find useful relationships in your data. Sorry there is no short cut and I don't think anybody that believes they have discovered the "Holy Grail" is going to come forward with the secret recipe
    A very good video I just watched covers this “Markets are Efficient if and only if P = NP” http://www.youtube.com/watch?v=7iOJZZFDKpc&feature=player_embedded

    Cheers.
    Ron McEwan
    Sub GetData()

        Dim QuerySheet As Worksheet
        Dim DataSheet As Worksheet
        Dim EndDate As Date
        Dim StartDate As Date
        Dim Symbol As String
        Dim qurl As String
       
        Application.ScreenUpdating = False
        Application.DisplayAlerts = False
        Application.Calculation = xlCalculationManual
       
        Set DataSheet = ActiveSheet
     
            StartDate = DataSheet.Range("B2").Value
            EndDate = DataSheet.Range("B3").Value
            Symbol = DataSheet.Range("B4").Value
            Range("C7").CurrentRegion.ClearContents
           
    'construct the URL for the query
           
            qurl = "http://chart.yahoo.com/table.csv?s=" & Symbol
            qurl = qurl & "&a=" & Month(StartDate) - 1 & "&b=" & Day(StartDate) & _
                "&c=" & Year(StartDate) & "&d=" & Month(EndDate) - 1 & "&e=" & _
                Day(EndDate) & "&f=" & Year(EndDate) & "&g=" & Range("C3") & "&q=q&y=0&z=" & _
                Symbol & "&x=.csv"
            Range("c5") = qurl
                     
    QueryQuote:
                With ActiveSheet.QueryTables.Add(Connection:="URL;" & qurl, Destination:=DataSheet.Range("C7"))
                    .BackgroundQuery = True
                    .TablesOnlyFromHTML = False
                    .Refresh BackgroundQuery:=False
                    .SaveData = True
                End With
               
                Range("C7").CurrentRegion.TextToColumns Destination:=Range("C7"), DataType:=xlDelimited, _
                    TextQualifier:=xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=True, _
                    Semicolon:=False, Comma:=True, Space:=False, other:=False
               
                Range(Range("C7"), Range("C7").End(xlDown)).NumberFormat = "mmm d/yy"
                Range(Range("D7"), Range("G7").End(xlDown)).NumberFormat = "0.00"
                Range(Range("H7"), Range("H7").End(xlDown)).NumberFormat = "0,000"
    UpdateScale
    RemoveNames


    'turn calculation back on
        Application.Calculation = xlCalculationAutomatic
        Application.DisplayAlerts = True
        Range("C7:H2000").Select
        Selection.Sort Key1:=Range("C8"), Order1:=xlAscending, Header:=xlGuess, _
            OrderCustom:=1, MatchCase:=False, Orientation:=xlTopToBottom
        Range("B4").Select
    Paste

    End Sub
  • wesselwessel Member Posts: 537  Guru
    Hey,

    I'm not looking for some holy grail.

    When searching ScienceDirect there are lots of papers that analyze stock market data,
    using many more variables then just past prices, like:
    Dividends for the index, bid-ask spread of T-bills, T-bill holding period, nominal stock returns of the index, excess returns of the index, various duration treasury rates, various duration CD rates, corporate bond yields, the producer price index, the consumer price index, the industrial production index, M1 money supply, various treasury term spreads, and various default spreads between BAA bonds and various treasuries.

    I want to do similar research, but I have no idea where to download such data.
    Looked at yahoo pipes, although it is cool, I don't think it helps with this problem.

    Best regards,

    Wessel
  • ronmacronmac Member Posts: 11 Contributor II
    Some of the data you are looking for can be found at the St Louis Federal Reserve site http://research.stlouisfed.org/fred2/. They have a free Excel tool to simplify data retrieval http://research.stlouisfed.org/fred-addin/
Sign In or Register to comment.