Dynamic modelling in Twitter

aileenzhouaileenzhou Member Posts: 12 Contributor II
How to capture behavior change over time in Twitter, for example, perception towards vaccination during and post pandemic.
Hello, has any one done dynamic topic modelling that can reflect topic change over a certain period? Thank you. 

Answers

  • rfuentealbarfuentealba Moderator, RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou,

    That's work for NLP, and RapidMiner does that very well (you could also use Python NLTK together with RapidMiner). However, extracting data from Twitter is proven to be complicated because they put limits on how many tweets you get (max. 3200 per profile). If you can solve that, the rest is matter of: sorting information, tokenizing, getting parts of speech, lemmatization, stop words...

    If you are interested in the second part, ping us; the other one, I am pretty sure someone here did such a task but I don't remember who.

    All the best,

    Rod.
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thank you, Rod. To get Twitter sorted as suggested, ie, tokenising, lemmatisation, stop words ... is part of a standardised process. However, how to extract topic and the topic evolution over the time is where I stuck.  
  • rfuentealbarfuentealba Moderator, RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Hello @aileenzhou

    There are two operators that can help you:
    • Extract topics from documents
    • Extract topics from data
    Those are the same, the input varies. Have you tried these already?
  • aileenzhouaileenzhou Member Posts: 12 Contributor II
    Thanks a lot. It has been awhile since I tried the database last time. I will try again and let you know later. 
  • rfuentealbarfuentealba Moderator, RapidMiner Certified Analyst, Member, University Professor Posts: 568 Unicorn
    Sure! If you can share a sample process for us to see what you're doing, that would be marvelous!
Sign In or Register to comment.