Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Use trained (SVM) model on different processes
Contributor I
Hi,
so I finally managed to get a sentiment analysis going, using the SVM classification algorithm.
Now my problem is, the dataset that I wish to analyze is huge (>400m tweets). The SVM model also took very long to train as the training database had already more than 100k rows.
As the main dataset is in a postgresql database I probably just will query specific days. And I wish to avoid that rapidminer builds the SVM model each time again.
How is this possible?
Thanks in advance
edit: Maybe I can add some questions here:
Which classification algorithm to you suggest? Which one is the fastest, which on the most accurate, which one the best combining?
How can I optimize performance of the whole process? As I wish to analyze 400m tweets, you can imagine that my first priority is getting that whole analysis done in a realistic timeframe, like 2-3 days processing time maximum (on a normal notebook). Do you think that is even possible?
so I finally managed to get a sentiment analysis going, using the SVM classification algorithm.
Now my problem is, the dataset that I wish to analyze is huge (>400m tweets). The SVM model also took very long to train as the training database had already more than 100k rows.
As the main dataset is in a postgresql database I probably just will query specific days. And I wish to avoid that rapidminer builds the SVM model each time again.
How is this possible?
Thanks in advance
edit: Maybe I can add some questions here:
Which classification algorithm to you suggest? Which one is the fastest, which on the most accurate, which one the best combining?
How can I optimize performance of the whole process? As I wish to analyze 400m tweets, you can imagine that my first priority is getting that whole analysis done in a realistic timeframe, like 2-3 days processing time maximum (on a normal notebook). Do you think that is even possible?
Tagged:
0
