Autoscaling on Amazon with RapidMiner Server

JEdwardJEdward RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 578 Unicorn
edited December 2019 in Help

Hi guys,

 

I'm wondering if anyone has a handy guide already for autoscaling RapidMiner Server (both scale up & scale out) using AWS? 

 

Scaling out using a JBOSS cluster (deployment mode) is relatively straight forward & lots of guides already exist for it (although none RM specific, might write one and share if none exists)....  but I'd like to know if it's possible to scale up & down JBOSS without needing to restart the Instance or RM Server. 

Alternatively, what are the best practice approaches for identifying a potential high memory process to trigger creation of a high memory instance & run the desired process just on that instance? 

 

 

Best Answer

  • zprekopcsakzprekopcsak RapidMiner Certified Expert, Member Posts: 47 Guru
    Solution Accepted

    Hi @JEdward,

    Even though JBoss has some scale-out support, RapidMiner Server does not support that as of today. We do support High-Availability with cold-swap failover as you can see in this guide: http://docs.rapidminer.com/server/advanced-topics/high-availability/

    In theory, you could run two Servers in HA, a small primary server and spin up a bigger secondary server as necessary. Then you can kill the primary and do the failover to the bigger secondary instance. You essentially scale up then, but keep in mind that you will lose running and waiting processes and web UI sessions. See the HA guide for more details.

    This is not a promise, but we are considering architectural changes in Server for later this year that would also add support for a scale-out solution.

     

    Best,

Answers

  • NikouyNikouy Member Posts: 22 Contributor II

    Have there been any architectural changes that enable autoscalign functions?

    Thanks
  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,497 RM Data Scientist
    Hi @Nikouy,

    there have been a lot of change to this. The new JobAgent and HA infrastructure is vastly different.

    Is this a commercial inquiry? If yes, we are of course happy to explain it to you in a call. Just connect with me via PM.

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.