Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

"Importing large local JSON file into rapid miner"

cjc2238cjc2238 Member Posts: 1 Learner III
edited June 2019 in Help

Hey all,

 

I'm trying to import a ~30gb dataset into rapidminer but it keeps crashing. I suspect I am doing this incorrectly as I have never worked with JSON files before. From my research it seems I need to connect to an online database to import the data into rapid miner but I am trying to work from a local file. 

 

If it helps at all, the file I am working wth is located here: https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/ 

 

Any help wpuld be greatly appreciated~

 

-C

 

Tagged:

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Studio will trying to load that entire 30gb set into memory. Do you have enough memory to available to handle that large file?

Sign In or Register to comment.