stock normalization or prediction (help)

student_computestudent_compute Member Posts: 73 Contributor II
I have the following data

I want to predict the bottom price and the percentage of one share tomorrow
I don't know if this data needs normalization?
And how do I determine for each class?
Someone has the experience to help me
I tried a lot but I can't predict tomorrow's price at RapidMiner


  • Options
    student_computestudent_compute Member Posts: 73 Contributor II
    Hi again
    I created a process as follows

    For pre-processing and clearing stock data
    Is my process and idea right?
    What to do after that?
    Does anyone know how to help me?
    Please ... :'(

    I have the main chart or chart of stock changes. Can we predict the continuation of the chart in the RapidMiner program by entering the stock chart? How????

  • Options
    hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    edited August 2019
    Hello Student Compute, I don't see any missing values in your data and I don't see a need to look for outliers at this point. Isolate your open, high, low and close values and build a standard time series forecasting process. Stock prices usually rise on low volume and fall on high volume so those attributes don't have much predictive value so stick to price. So...if you just have price, should you normalise? Take a look for yourself and see if after normalisation your open, high, low and close values maintain the correct structure.This should answer your question.

    The theory works like this. Each stock has its own volatility which can be measured by taking an average of its daily range. Once you know this then you can build some targets that will give you price levels that the stock may reach on the next trading day. Anything beyond these levels could be considered an outlier so you don't need any special outlier detection. In most cases, your time series is going to look like a random walk but that is not necessarily the case. You need to think of it in terms of signal and noise. Sometimes you will have enough signal to get a good prediction and other times you won't.

    Any prediction better than random might be considered a good result. Think of it as tossing a weighted a coin over and over again where the distribution of values is not even. This is really the best that you can achieve with financial time series prediction.

    Please go back and review how to do basic time series forecasting. This question has been answered by many different people since the beginning of the year so please don't ask again how to do this.

    Kind regards,


    Edit...here is an screenshot that will help you understand the idea. No machine learning is really necessary. You would be surprised how often a simple technique like this can capture the range of a financial time series. In many cases, this approach will often get better results than trying to do a regression on the low or high values. 

  • Options
    student_computestudent_compute Member Posts: 73 Contributor II
    Yes you are right
    I searched a lot. But it's really a training file that can accurately predict a stock's performance. I couldn't find it and couldn't even draw a small sample.
    I did a study on stock market analysis, for example candle patterns. But I want to know what can be predicted by machine learning and what can I do in the data mining domain?
    I have asked for help many times in the community. But it was just you and one of the friends in the community didn't share their experience and didn't help.
    Maybe my expectation was too high.
    But I have repeatedly asked the experienced RapidMiner friends to provide an example of a time series in stock price forecasting so that I understand the process. But...
    Thank you very much
  • Options
    sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    edited August 2019
    @student_compute - writing better questions will likely yield more helpful answers - please read https://community.rapidminer.com/discussion/55505/.

    Thank you.
  • Options
    student_computestudent_compute Member Posts: 73 Contributor II
    edited August 2019
    Thank you Mr.@hughesfleming68
    I'm based on this link
     In Excel using regression I wanted to predict
    But the result is .. !!!!!!!!
    An example of training data


    Their regression

    y = -17.021x + 3196.5
    R² = 0.8638

    Test data


    And the result predict


    I don't know why the results are negative ...!
    I did a lot of searching in the forum, Mr. @sgenzer
    But I did not find an example that fit my question
    I also did a lot of research on YouTube on how regression can predict future stock prices. But I didn't find ....
    Anyway, thank you Mr.@hughesfleming68 for his great help

  • Options
    hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    @student_compute .Take a look at his diagram and duplicate it. This is a good opportunity to understand windowing.

  • Options
    hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    edited August 2019
    The diagram above is the most basic example. Start with univariate forecasting. Adjust your window sizes and experiment with different ML operators. At each step, look to see what happens to your data. Once you understand the basics, then move on to sliding window validation.

    As far as using linear regression for stock price prediction..... to make a long story short....that isn't going to work.

    Regression problems can be difficult to interpret because depending on your learner, you will have different levels of undershoot or overshoot. You will have to figure out yourself what is acceptable error. Turning the problem into a binary classification problem has many benefits..... the most obvious one being that you are either right or your are wrong.

    Edit.. switch off regularization in the GLM. That is a subject for another time.
  • Options
    hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    Anyone serious about forecasting needs to look at the results and solutions of the M4 forecasting competitions. https://www.mcompetitions.unic.ac.cy . Worth discussing.

    The winner Slawek Smyl from Uber Tech and his solution -

     https://github.com/M4Competition/M4-methods/blob/master/118 - slaweks17/ES_RNN_SlawekSmyl.pdf
  • Options
    student_computestudent_compute Member Posts: 73 Contributor II
    Hi again Mr. @ hughesfleming68
    Thank you so much for taking the time
    You are always very helpful in the community.
    May I request to send me the xml code of the processed photo?
    Thank you for your additional file and tips.
    So should I work with a window operator and a neural network?
    I try to do as you say. Professor. Wait for the result. Maybe I need your help again.
    good day
    thank you

  • Options
    hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
    I would avoid neural nets and work with simpler models. You can get very good results with certain kinds of neural nets but start with models that you can understand. I will upload something for you tomorrow.
  • Options
    student_computestudent_compute Member Posts: 73 Contributor II
Sign In or Register to comment.