Options

[SOLVED] SVM and polynomial attributes from word vector

VaasVaas Member Posts: 3 Contributor I
edited November 2018 in Help
Hi guys,

sorry for coming up with a question as my first post, but I am currently working on my bachelor's thesis and I experience major trouble applying the SVM learner on my dataset. I googled a lot and found 2 related posts already submitted to this forum:
http://rapid-i.com/rapidforum/index.php?topic=3845.0
http://rapid-i.com/rapidforum/index.php?topic=524.0
So I obviously did my job but still can't help myself on this one. I'll make it quick:

I basically pull out two values from my DB:
  • 'text' - which is just a field containing some text
  • 'pol' - which is a polarity label with the possible values 'pro' and 'con'
The text goes out of the DB, into a text preprocessor which tokenizes and filters the usual way. From there, it should be fed into the x-Validator and the SVM should try to learn polarity classification. The label is binominal, so that shouldn't cause any trouble.
What I understand is that the text gets converted into a word vecor (in my case TF-IDF) and the word vector becomes the items attribute - of course a word vector is polynomial so SVM can't handle it. Correct me if I am wrong on any of this. But heres the point: How can I make SVM handle the word vector correctly? If I convert it with the nominal2numerical (which is what most ppl advise, but I actually doubt that step to be useful in my case), SVM brings crappy results (< default, so basically impossible).

Has anyone ever had that before? Any help would be gratefully appreciated! Thanks in advance

Vaas

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.008">
 <context>
   <input/>
   <output/>
   <macros/>
 </context>
 <operator activated="true" class="process" compatibility="5.2.008" expanded="true" name="Process">
   <process expanded="true" height="670" width="547">
     <operator activated="true" class="read_database" compatibility="5.2.008" expanded="true" height="60" name="ReadDB" width="90" x="45" y="30">
       <parameter key="connection" value="myDB"/>
       <parameter key="query" value="SELECT `text`, `pol`&#38;#10;FROM `myTable`"/>
       <enumeration key="parameters"/>
     </operator>
     <operator activated="true" class="set_role" compatibility="5.2.008" expanded="true" height="76" name="Set Role" width="90" x="45" y="120">
       <parameter key="name" value="pol"/>
       <parameter key="target_role" value="label"/>
       <list key="set_additional_roles"/>
     </operator>
     <operator activated="true" class="text:process_document_from_data" compatibility="5.2.004" expanded="true" height="76" name="Process Documents from Data" width="90" x="45" y="255">
       <parameter key="prune_below_absolute" value="2"/>
       <parameter key="prune_above_absolute" value="999"/>
       <list key="specify_weights"/>
       <process expanded="true" height="499" width="675">
         <operator activated="true" class="web:extract_html_text_content" compatibility="5.2.003" expanded="true" height="60" name="Extract Content" width="90" x="45" y="30">
           <parameter key="minimum_text_block_length" value="1"/>
         </operator>
         <operator activated="true" class="text:transform_cases" compatibility="5.2.004" expanded="true" height="60" name="Transform Cases" width="90" x="45" y="120"/>
         <operator activated="true" class="text:tokenize" compatibility="5.2.004" expanded="true" height="60" name="Tokenize" width="90" x="45" y="210"/>
         <operator activated="true" class="text:filter_stopwords_english" compatibility="5.2.004" expanded="true" height="60" name="Filter Stopwords (English)" width="90" x="45" y="300"/>
         <connect from_port="document" to_op="Extract Content" to_port="document"/>
         <connect from_op="Extract Content" from_port="document" to_op="Transform Cases" to_port="document"/>
         <connect from_op="Transform Cases" from_port="document" to_op="Tokenize" to_port="document"/>
         <connect from_op="Tokenize" from_port="document" to_op="Filter Stopwords (English)" to_port="document"/>
         <connect from_op="Filter Stopwords (English)" from_port="document" to_port="document 1"/>
         <portSpacing port="source_document" spacing="0"/>
         <portSpacing port="sink_document 1" spacing="0"/>
         <portSpacing port="sink_document 2" spacing="0"/>
       </process>
     </operator>
     <operator activated="true" class="nominal_to_numerical" compatibility="5.2.008" expanded="true" height="94" name="Nominal to Numerical" width="90" x="246" y="210">
       <list key="comparison_groups"/>
     </operator>
     <operator activated="true" class="x_validation" compatibility="5.2.008" expanded="true" height="112" name="Validation" width="90" x="380" y="120">
       <process expanded="true" height="494" width="452">
         <operator activated="true" class="support_vector_machine_libsvm" compatibility="5.2.008" expanded="true" height="76" name="SVM" width="90" x="179" y="300">
           <parameter key="kernel_type" value="linear"/>
           <list key="class_weights"/>
         </operator>
         <connect from_port="training" to_op="SVM" to_port="training set"/>
         <connect from_op="SVM" from_port="model" to_port="model"/>
         <portSpacing port="source_training" spacing="0"/>
         <portSpacing port="sink_model" spacing="0"/>
         <portSpacing port="sink_through 1" spacing="0"/>
       </process>
       <process expanded="true" height="389" width="346">
         <operator activated="true" class="apply_model" compatibility="5.2.008" expanded="true" height="76" name="Apply Model" width="90" x="45" y="30">
           <list key="application_parameters"/>
         </operator>
         <operator activated="true" class="performance" compatibility="5.2.008" expanded="true" height="76" name="Performance" width="90" x="179" y="120"/>
         <connect from_port="model" to_op="Apply Model" to_port="model"/>
         <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
         <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
         <connect from_op="Performance" from_port="performance" to_port="averagable 1"/>
         <portSpacing port="source_model" spacing="0"/>
         <portSpacing port="source_test set" spacing="0"/>
         <portSpacing port="source_through 1" spacing="0"/>
         <portSpacing port="sink_averagable 1" spacing="0"/>
         <portSpacing port="sink_averagable 2" spacing="0"/>
       </process>
     </operator>
     <connect from_op="Mixed" from_port="output" to_op="Set Role" to_port="example set input"/>
     <connect from_op="Set Role" from_port="example set output" to_op="Process Documents from Data" to_port="example set"/>
     <connect from_op="Process Documents from Data" from_port="example set" to_op="Nominal to Numerical" to_port="example set input"/>
     <connect from_op="Nominal to Numerical" from_port="example set output" to_op="Validation" to_port="training"/>
     <connect from_op="Validation" from_port="model" to_port="result 1"/>
     <portSpacing port="source_input 1" spacing="0"/>
     <portSpacing port="sink_result 1" spacing="0"/>
     <portSpacing port="sink_result 2" spacing="0"/>
   </process>
 </operator>
</process>

Answers

  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi,

    as you'll note if you set a breakpoint after the Process Documents, the TF-IDF values generated by Process Documents are numerical - you get one numerical attribute for each token in your input data. So you can apply the SVM directly without further preprocessing. Btw, an SVM with a linear kernel as you have chosen is usually a good means for text classification.

    Best regards,
    Marius
  • Options
    VaasVaas Member Posts: 3 Contributor I
    Hi Marius, thanks for helping me out! I aggree with you on the numerical TF-IDF values. Thats exactly why I was confused to see the operation stop before training the SVM with the message
    The operator SVM does not have sufficient capabilities for the given data set: polynomial attributes not supported [...]
    So why does he refuse the dataset? Sould that be related to the column types I defined in my sql database? I doubt that because the word vector attribute is created by rapidminer and not fetched from the DB... I tried any SVM available with any parameter setup possible. It still refuses to learn (k-NN and naive bayes are working flawlessly btw, neural networks give the same error upon operation as SVM)

    Thanks again for the help, I appreciate your efforts!!
  • Options
    VaasVaas Member Posts: 3 Contributor I
    I found the error. For the record:

    The "Extract content"-Operator designed to strip input data from html-tags adds several attributes he tries to extract from so-called "meta-tags" (e.g. <meta name="author" content="John" />, found in almost all documents online), which are - of course - nominal.

    And that's the point where my SVM got confused. Setting the breakpoint and looking at what attributes are added was the right hint in this case. Marius, I have to take a bow: Instructions on how to help oneself are almost always the best way to go! Thanks again for everything and have a nice day

    vaas
  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Thanks for the kudos! I am glad that I could help you :)
Sign In or Register to comment.