RapidMiner

Custom polarity options in Aylien sentiment analysis

SOLVED
Contributor I m_oke
Contributor I

Custom polarity options in Aylien sentiment analysis

Is it possible to have custom polarity options in Aylien/RapidMiner's sentiment analysis, beyond the usual "positive, neutral, negative"?

 

Custom examples will be options like happy, sad, mild feeling etc...

8 REPLIES
RM Certified Expert
RM Certified Expert

Re: Custom polarity options in Aylien sentiment analysis

I doubt it, as this is a third party tool designed for use for lots of different people with different use cases.

You could either remap the existing polarities returned, or contact Aylien about developing a custom version for you.  Either that, or try building your own sentiment models in RapidMiner.

 

Brian T., Lindon Ventures - www.lindonventures.com
Analytics Consulting by Certified RapidMiner Analysts
Community Manager Community Manager
Community Manager

Re: Custom polarity options in Aylien sentiment analysis

Hi @m_oke - I would look into IBM Watson's Tone Analyzer API.  I use it all the time and will give you what you are looking for.


Scott

Scott Genzer
Senior Community Manager
RapidMiner, Inc.
RM Certified Expert
RM Certified Expert

Re: Custom polarity options in Aylien sentiment analysis

Nice, @sgenzer --do you have a sample RM process for that?  Or are you using it outside of RM?

Brian T., Lindon Ventures - www.lindonventures.com
Analytics Consulting by Certified RapidMiner Analysts
Highlighted
Community Manager Community Manager
Community Manager
Solution

Re: Custom polarity options in Aylien sentiment analysis

outside of RapidMiner?  Why would I do that?  Smiley Happy

 

Here you go:

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" breakpoints="after" class="text:read_document" compatibility="7.5.000" expanded="true" height="68" name="Read Document" width="90" x="45" y="34">
<parameter key="file" value="/Users/genzerconsulting/Desktop/Scott's comments from 2006.txt"/>
<description align="center" color="transparent" colored="false" width="126">get Scott's Comments from 2006 (Extraction)</description>
</operator>
<operator activated="true" breakpoints="after" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess" width="90" x="179" y="34">
<process expanded="true">
<operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data" width="90" x="45" y="34">
<parameter key="text_attribute" value="comment"/>
<parameter key="add_meta_information" value="false"/>
</operator>
<operator activated="true" class="split" compatibility="7.6.001" expanded="true" height="82" name="Split" width="90" x="179" y="34">
<parameter key="split_pattern" value="\n"/>
</operator>
<operator activated="true" class="transpose" compatibility="7.6.001" expanded="true" height="82" name="Transpose" width="90" x="313" y="34"/>
<operator activated="true" class="rename" compatibility="7.6.001" expanded="true" height="82" name="Rename" width="90" x="447" y="34">
<parameter key="old_name" value="att_1"/>
<parameter key="new_name" value="comment"/>
<list key="rename_additional_attributes"/>
</operator>
<operator activated="true" class="trim" compatibility="7.6.001" expanded="true" height="82" name="Trim" width="90" x="581" y="34"/>
<operator activated="true" class="filter_examples" compatibility="7.6.001" expanded="true" height="103" name="Filter Examples" width="90" x="715" y="34">
<parameter key="parameter_expression" value="matches(prefix(comment,1),&quot;[A-Z]&quot;)"/>
<list key="filters_list">
<parameter key="filters_entry_key" value="comment.is_not_missing."/>
</list>
</operator>
<operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (2)" width="90" x="849" y="34">
<process expanded="true">
<operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID" width="90" x="45" y="34">
<parameter key="create_nominal_ids" value="true"/>
</operator>
<operator activated="true" class="replace" compatibility="7.6.001" expanded="true" height="82" name="Replace" width="90" x="179" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="id"/>
<parameter key="include_special_attributes" value="true"/>
<parameter key="replace_what" value="id_"/>
</operator>
<connect from_port="in 1" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_op="Replace" to_port="example set input"/>
<connect from_op="Replace" from_port="example set output" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">create IDs</description>
</operator>
<connect from_port="in 1" to_op="Documents to Data" to_port="documents 1"/>
<connect from_op="Documents to Data" from_port="example set" to_op="Split" to_port="example set input"/>
<connect from_op="Split" from_port="example set output" to_op="Transpose" to_port="example set input"/>
<connect from_op="Transpose" from_port="example set output" to_op="Rename" to_port="example set input"/>
<connect from_op="Rename" from_port="example set output" to_op="Trim" to_port="example set input"/>
<connect from_op="Trim" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
<connect from_op="Filter Examples" from_port="example set output" to_op="Subprocess (2)" to_port="in 1"/>
<connect from_op="Subprocess (2)" from_port="out 1" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">clean them up (Transfer and Load)</description>
</operator>
<operator activated="true" class="filter_example_range" compatibility="7.6.001" expanded="true" height="82" name="Filter Example Range" width="90" x="313" y="34">
<parameter key="first_example" value="1"/>
<parameter key="last_example" value="3"/>
</operator>
<operator activated="true" breakpoints="after" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (8)" width="90" x="447" y="34">
<process expanded="true">
<operator activated="true" class="web:enrich_data_by_webservice" compatibility="7.3.000" expanded="true" height="68" name="Enrich Data by Webservice (9)" width="90" x="179" y="34">
<parameter key="query_type" value="Indexed"/>
<list key="string_machting_queries"/>
<list key="regular_expression_queries">
<parameter key="result" value=".*"/>
</list>
<list key="regular_region_queries">
<parameter key="foo" value="document_tone.[-}][-}]"/>
</list>
<list key="xpath_queries"/>
<list key="namespaces"/>
<parameter key="assume_html" value="false"/>
<list key="index_queries">
<parameter key="result" value="0.999999"/>
</list>
<list key="jsonpath_queries">
<parameter key="AngerScore" value="$.tones[?(@.tone_id=='anger')]"/>
</list>
<parameter key="request_method" value="POST"/>
<parameter key="service_method" value="foo"/>
<parameter key="body" value="body=&lt;%comment%&gt;"/>
<parameter key="url" value="https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2016-05-19"/>
<list key="request_properties">
<parameter key="username" value="username"/>
<parameter key="password" value="password"/>
<parameter key="Content-Type" value="text/plain"/>
<parameter key="sentences" value="false"/>
</list>
<description align="center" color="transparent" colored="false" width="126">tone with POST</description>
</operator>
<operator activated="true" class="generate_attributes" compatibility="7.6.001" expanded="true" height="82" name="Generate Attributes (10)" width="90" x="313" y="34">
<list key="function_descriptions">
<parameter key="result" value="concat(result,&quot;}&quot;)"/>
</list>
<description align="center" color="transparent" colored="false" width="126">add stupid } to the end of the extract</description>
</operator>
<operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (12)" width="90" x="447" y="34">
<process expanded="true">
<operator activated="true" class="multiply" compatibility="7.6.001" expanded="true" height="103" name="Multiply (2)" width="90" x="45" y="34"/>
<operator activated="true" class="loop_examples" compatibility="7.6.001" expanded="true" height="103" name="Loop Examples (4)" width="90" x="313" y="34">
<process expanded="true">
<operator activated="true" class="filter_example_range" compatibility="7.6.001" expanded="true" height="82" name="Filter Example Range (2)" width="90" x="45" y="34">
<parameter key="first_example" value="%{example}"/>
<parameter key="last_example" value="%{example}"/>
</operator>
<operator activated="true" class="text:data_to_documents" compatibility="7.5.000" expanded="true" height="68" name="Data to Documents (3)" width="90" x="380" y="34">
<parameter key="select_attributes_and_weights" value="true"/>
<list key="specify_weights">
<parameter key="result" value="1.0"/>
</list>
</operator>
<operator activated="true" class="text:combine_documents" compatibility="7.5.000" expanded="true" height="82" name="Combine Documents (3)" width="90" x="514" y="34"/>
<operator activated="true" class="text:extract_information" compatibility="7.5.000" expanded="true" height="68" name="Extract Information (4)" width="90" x="648" y="34">
<parameter key="query_type" value="JsonPath"/>
<list key="string_machting_queries"/>
<list key="regular_expression_queries"/>
<list key="regular_region_queries"/>
<list key="xpath_queries"/>
<list key="namespaces"/>
<list key="index_queries"/>
<list key="jsonpath_queries">
<parameter key="AngerScore" value="$.document_tone.tone_categories[0].tones[0].*"/>
<parameter key="DisgustScore" value="$.document_tone.tone_categories[0].tones[1].*"/>
<parameter key="FearScore" value="$.document_tone.tone_categories[0].tones[2].*"/>
<parameter key="JoyScore" value="$.document_tone.tone_categories[0].tones[3].*"/>
<parameter key="SadnessScore" value="$.document_tone.tone_categories[0].tones[4].*"/>
<parameter key="AnalyticScore" value="$.document_tone.tone_categories[1].tones[0].*"/>
<parameter key="ConfidenceScore" value="$.document_tone.tone_categories[1].tones[1].*"/>
<parameter key="TentativeScore" value="$.document_tone.tone_categories[1].tones[2].*"/>
<parameter key="OpennessScore" value="$.document_tone.tone_categories[2].tones[0].*"/>
<parameter key="ConscientiousnessScore" value="$.document_tone.tone_categories[2].tones[1].*"/>
<parameter key="ExtraversionScore" value="$.document_tone.tone_categories[2].tones[2].*"/>
<parameter key="AgreeablenessScore" value="$.document_tone.tone_categories[2].tones[3].*"/>
<parameter key="EmotionalRangeScore" value="$.document_tone.tone_categories[2].tones[4].*"/>
</list>
</operator>
<operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data (4)" width="90" x="782" y="34">
<parameter key="text_attribute" value="foo2"/>
</operator>
<operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (4)" width="90" x="916" y="34">
<parameter key="attribute_filter_type" value="single"/>
<parameter key="attribute" value="foo2"/>
<parameter key="invert_selection" value="true"/>
</operator>
<connect from_port="example set" to_op="Filter Example Range (2)" to_port="example set input"/>
<connect from_op="Filter Example Range (2)" from_port="example set output" to_op="Data to Documents (3)" to_port="example set"/>
<connect from_op="Data to Documents (3)" from_port="documents" to_op="Combine Documents (3)" to_port="documents 1"/>
<connect from_op="Combine Documents (3)" from_port="document" to_op="Extract Information (4)" to_port="document"/>
<connect from_op="Extract Information (4)" from_port="document" to_op="Documents to Data (4)" to_port="documents 1"/>
<connect from_op="Documents to Data (4)" from_port="example set" to_op="Select Attributes (4)" to_port="example set input"/>
<connect from_op="Select Attributes (4)" from_port="example set output" to_port="output 1"/>
<portSpacing port="source_example set" spacing="0"/>
<portSpacing port="sink_example set" spacing="0"/>
<portSpacing port="sink_output 1" spacing="0"/>
<portSpacing port="sink_output 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="append" compatibility="7.6.001" expanded="true" height="82" name="Append (4)" width="90" x="447" y="34"/>
<operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID (4)" width="90" x="313" y="238"/>
<operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID (5)" width="90" x="581" y="34"/>
<operator activated="true" class="join" compatibility="7.6.001" expanded="true" height="82" name="Join (3)" width="90" x="782" y="187">
<list key="key_attributes"/>
</operator>
<connect from_port="in 1" to_op="Multiply (2)" to_port="input"/>
<connect from_op="Multiply (2)" from_port="output 1" to_op="Loop Examples (4)" to_port="example set"/>
<connect from_op="Multiply (2)" from_port="output 2" to_op="Generate ID (4)" to_port="example set input"/>
<connect from_op="Loop Examples (4)" from_port="output 1" to_op="Append (4)" to_port="example set 1"/>
<connect from_op="Append (4)" from_port="merged set" to_op="Generate ID (5)" to_port="example set input"/>
<connect from_op="Generate ID (4)" from_port="example set output" to_op="Join (3)" to_port="left"/>
<connect from_op="Generate ID (5)" from_port="example set output" to_op="Join (3)" to_port="right"/>
<connect from_op="Join (3)" from_port="join" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">process Json data</description>
</operator>
<operator activated="true" class="subprocess" compatibility="7.6.001" expanded="true" height="82" name="Subprocess (13)" width="90" x="581" y="34">
<process expanded="true">
<operator activated="true" class="parse_numbers" compatibility="7.6.001" expanded="true" height="82" name="Parse Numbers (4)" width="90" x="45" y="34">
<parameter key="attribute_filter_type" value="regular_expression"/>
<parameter key="regular_expression" value=".*Score"/>
<description align="center" color="transparent" colored="false" width="126">convert Scores to numerical</description>
</operator>
<operator activated="true" class="select_attributes" compatibility="7.6.001" expanded="true" height="82" name="Select Attributes (7)" width="90" x="246" y="34">
<parameter key="attribute_filter_type" value="subset"/>
<parameter key="attribute" value="id"/>
<parameter key="attributes" value="result|id"/>
<parameter key="invert_selection" value="true"/>
<parameter key="include_special_attributes" value="true"/>
</operator>
<connect from_port="in 1" to_op="Parse Numbers (4)" to_port="example set input"/>
<connect from_op="Parse Numbers (4)" from_port="example set output" to_op="Select Attributes (7)" to_port="example set input"/>
<connect from_op="Select Attributes (7)" from_port="example set output" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">clean up</description>
</operator>
<connect from_port="in 1" to_op="Enrich Data by Webservice (9)" to_port="Example Set"/>
<connect from_op="Enrich Data by Webservice (9)" from_port="ExampleSet" to_op="Generate Attributes (10)" to_port="example set input"/>
<connect from_op="Generate Attributes (10)" from_port="example set output" to_op="Subprocess (12)" to_port="in 1"/>
<connect from_op="Subprocess (12)" from_port="out 1" to_op="Subprocess (13)" to_port="in 1"/>
<connect from_op="Subprocess (13)" from_port="out 1" to_port="out 1"/>
<portSpacing port="source_in 1" spacing="0"/>
<portSpacing port="source_in 2" spacing="0"/>
<portSpacing port="sink_out 1" spacing="0"/>
<portSpacing port="sink_out 2" spacing="0"/>
</process>
<description align="center" color="transparent" colored="false" width="126">IBM Watson - get comment &amp;quot;tone&amp;quot; scores</description>
</operator>
<operator activated="true" class="order_attributes" compatibility="7.6.001" expanded="true" height="82" name="Reorder Attributes" width="90" x="581" y="34">
<parameter key="attribute_ordering" value="comment"/>
</operator>
<connect from_op="Read Document" from_port="output" to_op="Subprocess" to_port="in 1"/>
<connect from_op="Subprocess" from_port="out 1" to_op="Filter Example Range" to_port="example set input"/>
<connect from_op="Filter Example Range" from_port="example set output" to_op="Subprocess (8)" to_port="in 1"/>
<connect from_op="Subprocess (8)" from_port="out 1" to_op="Reorder Attributes" to_port="example set input"/>
<connect from_op="Reorder Attributes" from_port="example set output" to_port="result 1"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="0"/>
</process>
</operator>
</process>
Scott Genzer
Senior Community Manager
RapidMiner, Inc.
RM Certified Expert
RM Certified Expert

Re: Custom polarity options in Aylien sentiment analysis

Nice, I will check it out!

Brian T., Lindon Ventures - www.lindonventures.com
Analytics Consulting by Certified RapidMiner Analysts
Contributor I m_oke
Contributor I

Re: Custom polarity options in Aylien sentiment analysis

I will definitely check this out and feedback, thank you @sgenzer

Contributor I m_oke
Contributor I

Re: Custom polarity options in Aylien sentiment analysis

@sgenzer I see that the document you used for the project is on your desktop.

If there are no privacy issues, do you mind making the document publicly avaliable?

Community Manager Community Manager
Community Manager

Re: Custom polarity options in Aylien sentiment analysis

@m_oke haha.  There are no privacy issues as long as you don't kill yourself laughing too much.  The doc is a anonymously-cleansed version of report card comments I gave my students back in 2006.  Smiley Happy  They are attached.

 

Scott

 

Scott Genzer
Senior Community Manager
RapidMiner, Inc.