RapidMiner 9.8 Beta is now available

Be one of the first to get your hands on the new features. More details and downloads here:

GET RAPIDMINER 9.8 BETA

Autoscaling on Amazon with RapidMiner Server

JEdwardJEdward RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 570   Unicorn
edited December 2019 in Help

Hi guys,

 

I'm wondering if anyone has a handy guide already for autoscaling RapidMiner Server (both scale up & scale out) using AWS? 

 

Scaling out using a JBOSS cluster (deployment mode) is relatively straight forward & lots of guides already exist for it (although none RM specific, might write one and share if none exists)....  but I'd like to know if it's possible to scale up & down JBOSS without needing to restart the Instance or RM Server. 

Alternatively, what are the best practice approaches for identifying a potential high memory process to trigger creation of a high memory instance & run the desired process just on that instance? 

 

 

Best Answer

  • zprekopcsakzprekopcsak RapidMiner Certified Expert, Member Posts: 47  Guru
    Solution Accepted

    Hi @JEdward,

    Even though JBoss has some scale-out support, RapidMiner Server does not support that as of today. We do support High-Availability with cold-swap failover as you can see in this guide: http://docs.rapidminer.com/server/advanced-topics/high-availability/

    In theory, you could run two Servers in HA, a small primary server and spin up a bigger secondary server as necessary. Then you can kill the primary and do the failover to the bigger secondary instance. You essentially scale up then, but keep in mind that you will lose running and waiting processes and web UI sessions. See the HA guide for more details.

    This is not a promise, but we are considering architectural changes in Server for later this year that would also add support for a scale-out solution.

     

    Best,

Answers

  • NikouyNikouy Member Posts: 22 Contributor II

    Have there been any architectural changes that enable autoscalign functions?

    Thanks
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,636  RM Data Scientist
    Hi @Nikouy,

    there have been a lot of change to this. The new JobAgent and HA infrastructure is vastly different.

    Is this a commercial inquiry? If yes, we are of course happy to explain it to you in a call. Just connect with me via PM.

    Best,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
    sgenzerIngoRM
Sign In or Register to comment.