"[Solved] Assigning IDs to tokens in word vector."

gamgambambam

Hi,

I have this word vector:



http://postimg.org/image/zaahfep4v/

What i want to do is copy all the tokens from the top x-axis and align them next to their appropriate IDs.

For example, the token 'actual' (on the top x-axis) has the value of 0.366 and id of 3006. What I want is for the token 'actual' to be aligned next to its id. So in this case row 45 would go: id (3006), token (actual).



http://postimg.org/image/erk3d16h5/

Hope all this makes sense.

Thanks in advance for your help,
Alex

MariusHelf

Hi Alex,

if I understand you correctly you want to De-Pivot the data (search Pivot on the internet to understand the operation). For an example see the process below. The regular expression in the attribute_name parameter of De-Pivot selects all attributes but text and id.

Best regards,
Marius

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="6.0.002">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="6.0.002" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="text:create_document" compatibility="5.3.002" expanded="true" height="60" name="Create Document" width="90" x="45" y="30">
        <parameter key="text" value="This is a text."/>
      </operator>
      <operator activated="true" class="text:create_document" compatibility="5.3.002" expanded="true" height="60" name="Create Document (2)" width="90" x="45" y="120">
        <parameter key="text" value="And here this is also a small text document."/>
      </operator>
      <operator activated="true" class="text:process_documents" compatibility="5.3.002" expanded="true" height="112" name="Process Documents" width="90" x="179" y="30">
        <parameter key="keep_text" value="true"/>
        <process expanded="true">
          <operator activated="true" class="text:tokenize" compatibility="5.3.002" expanded="true" height="60" name="Tokenize" width="90" x="179" y="30"/>
          <connect from_port="document" to_op="Tokenize" to_port="document"/>
          <connect from_op="Tokenize" from_port="document" to_port="document 1"/>
          <portSpacing port="source_document" spacing="0"/>
          <portSpacing port="sink_document 1" spacing="0"/>
          <portSpacing port="sink_document 2" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="generate_id" compatibility="6.0.002" expanded="true" height="76" name="Generate ID" width="90" x="313" y="30"/>
      <operator activated="true" class="de_pivot" compatibility="6.0.002" expanded="true" height="76" name="De-Pivot" width="90" x="447" y="30">
        <list key="attribute_name">
          <parameter key="value" value="(?!^(text|id)$).*"/>
        </list>
        <parameter key="index_attribute" value="token"/>
        <parameter key="create_nominal_index" value="true"/>
      </operator>
      <connect from_op="Create Document" from_port="output" to_op="Process Documents" to_port="documents 1"/>
      <connect from_op="Create Document (2)" from_port="output" to_op="Process Documents" to_port="documents 2"/>
      <connect from_op="Process Documents" from_port="example set" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="De-Pivot" to_port="example set input"/>
      <connect from_op="De-Pivot" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

gamgambambam

Thanks Marius! Worked perfectly.

You are a champion!

Alex