Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

Dynamic modelling in Twitter

aileenzhouaileenzhou Member Posts: 12 Contributor II
in Help
How to capture behavior change over time in Twitter, for example, perception towards vaccination during and post pandemic.
Hello, has any one done dynamic topic modelling that can reflect topic change over a certain period? Thank you. 
Tagged:

Answers

  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou,

    That's work for NLP, and RapidMiner does that very well (you could also use Python NLTK together with RapidMiner). However, extracting data from Twitter is proven to be complicated because they put limits on how many tweets you get (max. 3200 per profile). If you can solve that, the rest is matter of: sorting information, tokenizing, getting parts of speech, lemmatization, stop words...

    If you are interested in the second part, ping us; the other one, I am pretty sure someone here did such a task but I don't remember who.

    All the best,

    Rod.
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thank you, Rod. To get Twitter sorted as suggested, ie, tokenising, lemmatisation, stop words ... is part of a standardised process. However, how to extract topic and the topic evolution over the time is where I stuck.  
  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou

    There are two operators that can help you:
    • Extract topics from documents
    • Extract topics from data
    Those are the same, the input varies. Have you tried these already?
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thanks a lot. It has been awhile since I tried the database last time. I will try again and let you know later. 
  • rfuentealbarfuentealba RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Sure! If you can share a sample process for us to see what you're doing, that would be marvelous!
Sign In or Register to comment.