Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

Text Tokenization Using Regular Expression For Text Mining

onurer007onurer007 Member Posts: 1 Learner III
edited November 2019 in Help
Hello,
I have a problem and i need your help, please.
I want to tokenize a unstructured  document using regular expression. I have a text file where each rows include a sentence such as:

1. String1 String2 String3 String4 String5
2. String6      -      String7    -           -
...
n. String8    -        String9 String10   -               (assume string2 and string5 dont exist.)

What I exactly want to do is that tokenization will extract each word  and give the results in a table in Excel format such as:


    S1              S2            S3            S4             S5
1.   String1    String2    String3      String4    String5
2.   String6        -          String7          -              -
3.
..
n.   String8        -          String9      String10      -


which operators and and which regular expression structure can i use in Rapid Miner?
Thank you for your help in advance.

Tagged:

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    If your original document contains the dashes you can simply read it with Read CSV and specify all blanks (space, tab, etc.) as column separator.

    Best regards,
    Marius
Sign In or Register to comment.