"Importing CSV files in general (and from DropBox.com)"

kkrawczykkkrawczyk Member Posts: 2 Contributor I
edited May 2019 in Help
I'm a beginner with RapidMiner.  I've been through some of the examples, and the tutorials.

Now, I'm attempting to import data.  So far, let's just say 'waiting'.

I've attempted to import a 10MB csv file, but after 3 hours I cancelled.  Then I attempted a 3MB file, and I ended up cancelling.

Is there a trick to importing data?  In the long run I need to import a few gigs of data....can it handle that?

Also, is the process of importing data only manual?, or can the import process point to dropbox.com?
Tagged:

Answers

  • awchisholmawchisholm RapidMiner Certified Expert, Member Posts: 458 Unicorn
    Hello

    I use a local folder linked to Dropbox for my RM repository. I have a 100mb dataset with about 700,000 rows stored like this originally loaded from a CSV file.

    When I load it to look at it, it pushes the limits of my laptop because it loads the whole lot into memory. If I had to process any more than this, I would store it in a database and do some sort of aggregration to visualise it and take samples for creating models.

    Regards

    Andrew
  • B_B_ Member Posts: 70 Maven
    Open the log window and check for errors.  Check your input for things that would cause an error, such as the wrong type of data assigned to an attribute.

    I've had processes go on and on because of things like a null getting into a data set.

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    Hi,
    if you find your dataset ok, not having any errors and the import of the most current RM version 5.1.001 still hangs, could you share the file with us? We would then try to find the problem.

    Greetings,
      Sebastian
Sign In or Register to comment.