Search for replies to a particular twitter handle

batstache611batstache611 Member Posts: 45 Guru
edited August 2019 in Help

Hello again,

Is there a way to configure any of the available twitter operators to look for replies to a twitter handle?? Thank you very much.

Tagged:

Best Answers

  • pschlunderpschlunder Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 96 RM Research
    Solution Accepted

    Hi Batstache,

    you can use the regular "Search Twitter" Operator to search for tweets containing the query @username to find tweets mentioning the given user.

    process.pngFinding tweets mentioning the user rapidminerresult.pngResult view for the 10 most recent results

    Here is a possible process. Keep in mind, that you need to configure a twitter connection over the "Connections" tab first. Here I'm using a connection I named "Twitter Standard".

    <?xml version="1.0" encoding="UTF-8"?><process version="7.5.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.5.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="social_media:search_twitter" compatibility="7.3.000" expanded="true" height="68" name="Search Twitter" width="90" x="983" y="34">
    <parameter key="connection" value="Twitter Standard"/>
    <parameter key="query" value="@rapidminer"/&gt;
    <parameter key="result_type" value="recent"/>
    <parameter key="limit" value="10"/>
    </operator>
    <connect from_op="Search Twitter" from_port="output" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

    Cheers,

    Philipp

  • pschlunderpschlunder Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 96 RM Research
    Solution Accepted

    Actually there is the option to use the "To-user-id" attribute to distinguish replies from retweets. It contains twitters internal id of the user that was mentioned. I created a small subprocess you can use to search for replies to a list of users.

    get_mentions.PNGGet Mentions for a list of Twitter Users

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.5.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.5.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="text:create_document" compatibility="7.5.000" expanded="true" height="68" name="List of Users" width="90" x="112" y="34">
    <parameter key="text" value="rapidminer"/>
    <description align="center" color="transparent" colored="false" width="126">Provide a list of usernames as a document</description>
    </operator>
    <operator activated="true" class="set_macro" compatibility="7.5.001" expanded="true" height="82" name="Number of Tweets" width="90" x="112" y="187">
    <parameter key="macro" value="tweets_to_search_through"/>
    <parameter key="value" value="100"/>
    <description align="center" color="transparent" colored="false" width="126">set amount of tweets to scan through for mentions</description>
    </operator>
    <operator activated="true" class="subprocess" compatibility="7.5.001" expanded="true" height="103" name="Get Mentions from" width="90" x="313" y="34">
    <process expanded="true">
    <operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data" width="90" x="313" y="34">
    <parameter key="text_attribute" value="username"/>
    </operator>
    <operator activated="true" class="concurrency:loop_values" compatibility="7.5.001" expanded="true" height="82" name="Loop Values" width="90" x="514" y="34">
    <parameter key="attribute" value="username"/>
    <parameter key="iteration_macro" value="username"/>
    <process expanded="true">
    <operator activated="true" class="social_media:get_twitter_user_details" compatibility="7.3.000" expanded="true" height="68" name="Get Twitter User Details" width="90" x="45" y="238">
    <parameter key="connection" value="Twitter Standard"/>
    <parameter key="user" value="%{username}"/>
    <description align="center" color="transparent" colored="false" width="126">get internal twitter id of the user for more accurate matching</description>
    </operator>
    <operator activated="true" class="extract_macro" compatibility="7.5.001" expanded="true" height="68" name="Extract Macro" width="90" x="179" y="238">
    <parameter key="macro" value="user-id"/>
    <parameter key="macro_type" value="data_value"/>
    <parameter key="attribute_name" value="Id"/>
    <parameter key="example_index" value="1"/>
    <list key="additional_macros"/>
    <description align="center" color="transparent" colored="false" width="126">make it available for future filtering</description>
    </operator>
    <operator activated="true" class="social_media:search_twitter" compatibility="7.3.000" expanded="true" height="68" name="Search Twitter" width="90" x="45" y="34">
    <parameter key="connection" value="Twitter Standard"/>
    <parameter key="query" value="@%{username}"/&gt;
    <parameter key="result_type" value="recent"/>
    <parameter key="limit" value="%{tweets_to_search_through}"/>
    <description align="center" color="transparent" colored="false" width="126">get all tweets mentioning the username</description>
    </operator>
    <operator activated="true" class="delay" compatibility="7.5.001" expanded="true" height="103" name="Delay" width="90" x="380" y="136">
    <parameter key="delay" value="none"/>
    <parameter key="delay_amount" value="1"/>
    <description align="center" color="transparent" colored="false" width="126">make sure both information are in-place at the same time</description>
    </operator>
    <operator activated="true" class="filter_examples" compatibility="7.5.001" expanded="true" height="103" name="Filter Examples" width="90" x="514" y="136">
    <list key="filters_list">
    <parameter key="filters_entry_key" value="To-User-Id.eq.%{user-id}"/>
    </list>
    <description align="center" color="transparent" colored="false" width="126">select only tweets directed at our targeted user-id</description>
    </operator>
    <connect from_op="Get Twitter User Details" from_port="output" to_op="Extract Macro" to_port="example set"/>
    <connect from_op="Extract Macro" from_port="example set" to_op="Delay" to_port="through 2"/>
    <connect from_op="Search Twitter" from_port="output" to_op="Delay" to_port="through 1"/>
    <connect from_op="Delay" from_port="through 1" to_op="Filter Examples" to_port="example set input"/>
    <connect from_op="Filter Examples" from_port="example set output" to_port="output 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    <portSpacing port="sink_output 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="in 1" to_op="Documents to Data" to_port="documents 1"/>
    <connect from_op="Documents to Data" from_port="example set" to_op="Loop Values" to_port="input 1"/>
    <connect from_op="Loop Values" from_port="output 1" to_port="out 1"/>
    <portSpacing port="source_in 1" spacing="0"/>
    <portSpacing port="source_in 2" spacing="0"/>
    <portSpacing port="source_in 3" spacing="0"/>
    <portSpacing port="sink_out 1" spacing="0"/>
    <portSpacing port="sink_out 2" spacing="0"/>
    </process>
    <description align="center" color="transparent" colored="false" width="126">Look through the given amount of tweets (via the set macro operator) to find tweets directed at the users listed in the provided document</description>
    </operator>
    <connect from_op="List of Users" from_port="output" to_op="Get Mentions from" to_port="in 1"/>
    <connect from_op="Number of Tweets" from_port="through 1" to_op="Get Mentions from" to_port="in 2"/>
    <connect from_op="Get Mentions from" from_port="out 1" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

     

    Cheers,

    Philipp

Answers

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    Thanks for the sample process! Is there any way to directly limit the return to replies, or is it necessary to do some text searching afterwards for that?
    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • batstache611batstache611 Member Posts: 45 Guru

    Thank you @pschlunder, I was using this setup already. But I specifically wanted to look for replies to tweets only and not retweets. I CAN filter out the retweets using RegEx, but I was hoping that similar to the operator that gets tweets from a particular handle, there might be one that gets all replies to those tweets. Perhaps I don't know very well how twitter works since I do not use it or Facebook for that matter. But thank you for your help.

  • pschlunderpschlunder Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 96 RM Research

    There is a column called "To-User-Id" (as well as "To-User" for the actual username). This one is set, if the tweet is regarded as a reply. Hence if you want to ignore Retweets make sure, that this given field is not empty. Furthermore it seems, that since it only contains one id, that only the person metioned first in the tweet is regarded as the person replied to. In general a reply in twitter is invoked by starting the tweet with "@username" followed by your message.

     

    Since there is no extra operator, yet. I've created a subprocess you can use for the time being.

     

    get_mentions.PNGGet Mentions for a list of Twitter Users

    <?xml version="1.0" encoding="UTF-8"?><process version="7.5.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.5.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="text:create_document" compatibility="7.5.000" expanded="true" height="68" name="List of Users" width="90" x="112" y="34">
    <parameter key="text" value="rapidminer"/>
    <description align="center" color="transparent" colored="false" width="126">Provide a list of usernames as a document</description>
    </operator>
    <operator activated="true" class="set_macro" compatibility="7.5.001" expanded="true" height="82" name="Number of Tweets" width="90" x="112" y="187">
    <parameter key="macro" value="tweets_to_search_through"/>
    <parameter key="value" value="100"/>
    <description align="center" color="transparent" colored="false" width="126">set amount of tweets to scan through for mentions</description>
    </operator>
    <operator activated="true" class="subprocess" compatibility="7.5.001" expanded="true" height="103" name="Get Mentions from" width="90" x="313" y="34">
    <process expanded="true">
    <operator activated="true" class="text:documents_to_data" compatibility="7.5.000" expanded="true" height="82" name="Documents to Data" width="90" x="313" y="34">
    <parameter key="text_attribute" value="username"/>
    </operator>
    <operator activated="true" class="concurrency:loop_values" compatibility="7.5.001" expanded="true" height="82" name="Loop Values" width="90" x="514" y="34">
    <parameter key="attribute" value="username"/>
    <parameter key="iteration_macro" value="username"/>
    <process expanded="true">
    <operator activated="true" class="social_media:get_twitter_user_details" compatibility="7.3.000" expanded="true" height="68" name="Get Twitter User Details" width="90" x="45" y="238">
    <parameter key="connection" value="Twitter Standard"/>
    <parameter key="user" value="%{username}"/>
    <description align="center" color="transparent" colored="false" width="126">get internal twitter id of the user for more accurate matching</description>
    </operator>
    <operator activated="true" class="extract_macro" compatibility="7.5.001" expanded="true" height="68" name="Extract Macro" width="90" x="179" y="238">
    <parameter key="macro" value="user-id"/>
    <parameter key="macro_type" value="data_value"/>
    <parameter key="attribute_name" value="Id"/>
    <parameter key="example_index" value="1"/>
    <list key="additional_macros"/>
    <description align="center" color="transparent" colored="false" width="126">make it available for future filtering</description>
    </operator>
    <operator activated="true" class="social_media:search_twitter" compatibility="7.3.000" expanded="true" height="68" name="Search Twitter" width="90" x="45" y="34">
    <parameter key="connection" value="Twitter Standard"/>
    <parameter key="query" value="@%{username}"/&gt;
    <parameter key="result_type" value="recent"/>
    <parameter key="limit" value="%{tweets_to_search_through}"/>
    <description align="center" color="transparent" colored="false" width="126">get all tweets mentioning the username</description>
    </operator>
    <operator activated="true" class="delay" compatibility="7.5.001" expanded="true" height="103" name="Delay" width="90" x="380" y="136">
    <parameter key="delay" value="none"/>
    <parameter key="delay_amount" value="1"/>
    <description align="center" color="transparent" colored="false" width="126">make sure both information are in-place at the same time</description>
    </operator>
    <operator activated="true" class="filter_examples" compatibility="7.5.001" expanded="true" height="103" name="Filter Examples" width="90" x="514" y="136">
    <list key="filters_list">
    <parameter key="filters_entry_key" value="To-User-Id.eq.%{user-id}"/>
    </list>
    <description align="center" color="transparent" colored="false" width="126">select only tweets directed at our targeted user-id</description>
    </operator>
    <connect from_op="Get Twitter User Details" from_port="output" to_op="Extract Macro" to_port="example set"/>
    <connect from_op="Extract Macro" from_port="example set" to_op="Delay" to_port="through 2"/>
    <connect from_op="Search Twitter" from_port="output" to_op="Delay" to_port="through 1"/>
    <connect from_op="Delay" from_port="through 1" to_op="Filter Examples" to_port="example set input"/>
    <connect from_op="Filter Examples" from_port="example set output" to_port="output 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="source_input 2" spacing="0"/>
    <portSpacing port="sink_output 1" spacing="0"/>
    <portSpacing port="sink_output 2" spacing="0"/>
    </process>
    </operator>
    <connect from_port="in 1" to_op="Documents to Data" to_port="documents 1"/>
    <connect from_op="Documents to Data" from_port="example set" to_op="Loop Values" to_port="input 1"/>
    <connect from_op="Loop Values" from_port="output 1" to_port="out 1"/>
    <portSpacing port="source_in 1" spacing="0"/>
    <portSpacing port="source_in 2" spacing="0"/>
    <portSpacing port="source_in 3" spacing="0"/>
    <portSpacing port="sink_out 1" spacing="0"/>
    <portSpacing port="sink_out 2" spacing="0"/>
    </process>
    <description align="center" color="transparent" colored="false" width="126">Look through the given amount of tweets (via the set macro operator) to find tweets directed at the users listed in the provided document</description>
    </operator>
    <connect from_op="List of Users" from_port="output" to_op="Get Mentions from" to_port="in 1"/>
    <connect from_op="Number of Tweets" from_port="through 1" to_op="Get Mentions from" to_port="in 2"/>
    <connect from_op="Get Mentions from" from_port="out 1" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

    Cheers,

    Philipp

Sign In or Register to comment.