Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

have anyone used RapidMiner for BigData?

BibbyBibby Member Posts: 1 Learner III
edited November 2018 in Help
Hi
I'm new to RapidMiner. And I was wondering has anyone used RapidMiner for BigData like tens of terabytes of data. How does it perform?  ::)

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi Bibby,

    most operators of RapidMiner need a copy of the data in main memory. For data of this size this is obviously not feasible, so in most cases RapidMiner cannot handle this amount of data at once. But usually you perform some sampling though and work just on parts of the data, and RapidMiner provides a good interface to common databases, so you can the the samples directly from your database. Additioanlly we are currently implementing some algorithms to work directly inside the database, so the data is not at all copied to the RapidMiner memory.

    Cheers,
    Marius
Sign In or Register to comment.