Gradient booster versus neural net

Barclaeys

hi,

I'm concluding an exercise around time series. So far I have explored different ways varying from naive models, STL decomposition, Holt-Winters, Arima which are time-series models. I would like to explore real machine learning models. I have seen a RapidMiner tutorial in relation to windowing which is applying a gradient booster.
1- How does this work precisely? 
2- What is the difference with e.g. a neural net model?
3- How are such ML models different than the regular  time-series models?

I copy @mschmitz as reference following our recent discussion.

Thank you beforehand,
Bart

jacobcybulski

The idea of using machine learning models with Sliding Window and Sliding Window Validation is that the latter acts as a form of Cross Validation. Inside you can use any model you like. However, remember that "predictors" are parts of your sliding window over the time series. This means that typical machine learning models have no understanding of the time-based relationships between these predictors. Regression is commonly used in this context as it fits well data with any linear dependencies. If you use a Decision Tree it has no idea of such dependencies and the fact that one window and the following are simply shifted by some offset or lag. However, if you use a Gradient Boosted Trees, which is an ensemble of many trees trying to improve each other performance, chances are that some trees will learn the time series patterns with a specific lag and others with a differ lag, so together they will do a much better job than a single tree. You mentioned neural networks. There are some neural nets which are very good dealing with time series. One of them are Convolutional Neural Nets, which can scan each window and its preceding and following neighbours in search of patterns. Another network which works very well with time series are LSTM neural networks, which work on different principles, they simply scan the series and have the ability to pick up patterns in data points which are very close to each other and patterns that may be far apart. This behaviour is very useful in detection of short-term effects such as rising or dropping in time series values, but also long-term effects such as seasonal changes. I hope it helps.

wufutura

3 Questions, sir - that occur to me off of your excellent post:

1. re the CNN approach is there an operator that I can look at and experiment with for this?   what is it called?
2. per the LSTM approach same question exactly. 
   
3. can these approaches be combined somehow and, if so, can you show a newbie like myself a simple diagram about how that might look?
4. lastly, do we internally have the options to work with either RL or GRU approaches?

thank you & good morning!! - Richard

jacobcybulski

@wufutura RapidMiner has an excellent Deep Learning extension which supports some of the above models, such as CNN, LSTM and more. However, if you wanted to gain access to a wider range of deep learning / AI facilities, which include RL and GRU, at this point in time I'd recommend doing so via Python with Tensorflow or PyTorch. RapidMiner has a Python scripting extension, so that you could do parts of computation in RapidMiner and parts in Tensorflow in Python (e.g. with Anaconda).

wufutura

oh boy, thanks so much with being so giving of your knowledge and your time!   I am not knowledgeable about Python yet, but I am trying to learn.   Will endeavor to understand as much of your post as I can in the short run.  do you think anyone would charge a minuscule amount to contract out for this - - -i.e. for putting in a GRU, LSTM and CNN? Thanks again,   Richard

jacobcybulski

@wufutura I think you'd need to go to one of the Python / Tensorflow communities, where you'll find lots of enthusiasts to do the job for a small fee.

wufutura

i'll check 'em out today.