text classification with dialect in Arabic language

BadrBadr Member Posts: 1 Newbie
in Help
I made text classification for Arabic and it works very well with standard Arabic language but now I will use   with dataset with different dialect in Arabic ? can I use same operators( tokenize and stem(Arabic)  and stopword 
Tagged:

Best Answer

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    Solution Accepted
    It should as long as the underlying characters are not different, even though the vocabulary, syntax and usage may vary in the dialect.  RapidMiner (or any NLP algorithm) doesn't really understand languages, it just transforms them all into numerical representations to manipulate.
    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
Sign In or Register to comment.