Online learning with RapidMiner

tonio09tonio09 Member Posts: 12 Contributor II
edited November 2018 in Help
Hi, I'm developing an automated trading bot that receives streaming data from various sources including social media streams and market trade streams, runs machine learning algorithms on it and executes trades.

I'm using RapidMiner to prototype the machine learning system.

Is it possible to integrate RapidMiner into a Java program running on a server, do online learning with streaming data, adjust the model continuously and produce signals and predictions that can be fed back into the Java program?

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi,

    RapidMiner does not yet support stream mining or online learning, but we just laid the foundation of a stream mining framework. It will need some time though until it is ready for release.
    Concerning your question about the server: RapidAnalytics can do that for you. It runs on a central server, you edit your processes with RapidMiner and store them on the server in a so-called remote repository, which integrates seemlessly into RapidMiner and looks just like the standard local repository to the user.
    After storing the processes on the server, you can schedule them to run on a regular basis and create e.g. a model. You can store the model either in the repository for later use, or you can feed new data to the server via a webservice, and RapidAnalytics applies the model and returns the classification result (or anything else you configured it to return).

    Via the execute script operators you can of course also produce signals to external programs.

    Best, Marius
  • adrecordadrecord Member Posts: 1 Contributor I
    Has the online learning support been added to RapidMiner yet?
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi,

    unfortunately it has not been added yet.

    Best regards,
    Marius
  • vineet_p_svineet_p_s Member Posts: 3 Contributor I
    Hello
    Has RapidMiner come up with a stream plugin or something similar. Use case is to read live streaming data from twitter and provide analytics.
    Thanks
    Vineet
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    oh yes @vineet_p_s - very nice operator that pulls in Twitter feeds.  :)  

     

    Screen Shot 2017-09-12 at 9.06.09 PM.png

    There are also many processes here on the Community where people have many use cases.  Just search "Twitter"

     

    Scott

  • vineet_p_svineet_p_s Member Posts: 3 Contributor I
    Hello Scott
    The operators you mention do not capture stream data. May be I didn't explain my requirement clearly.
    Basically I want to know if we can use data stream in RM. something like Apache Storm or Kafka.

    Thanks
    Vineet
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager

    hello @vineet_p_s - my apologies for not understanding your question.  No RapidMiner currently does not support streams from Apache Storm or Kafka.

     

    Scott

Sign In or Register to comment.