Data Storage in RapidMiner

xstreamxstream Member Posts: 1 Contributor I
edited November 2018 in Help
Hello,

i would like to know how data is stored in rapid miner if I import a csv file for example.
Is there an internal relational database in which the data is imported?

Do I need an external database like oracle when I work with a huge amount of data?

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,525   Unicorn
    Hi,
    data is either stored in the RAM of your computer or in the database you connected to.
    Unfortunately all common relational databases do not support more than something around 1000 columns, which is to restrictive for many data mining applications. So sometimes you have to store your data in main memory.
    If you have a realy huge amount of data, you will need an external database. This is simply the case if the data does not fit into your RAM. But be careful: The usage of most datamining algorithms is prohibitive with such huge example sets, since their runtime grows with the third power of number of examples. Use learners like NaiveBayes or Perceptron for such applications.

    Greetings,
      Sebastian
Sign In or Register to comment.