RapidMiner

Markov Chain (transition matrix and graph)

SOLVED
Highlighted
Contributor II

Markov Chain (transition matrix and graph)

So I'm trying to visualize the output of my clustering with a markov chain, and I'm stuck. It looks like Rapidminer has all the tools for the job, but I don't know how to make this work...

- I have my original dataset as an input, with an additional attribute describing the clusters found by x-means

- I feed this into a transition matrix operator, which shows me the probabilities to transition from one cluster to other clusters

- The second item in the transition matrix is a pairwise table, which is exactly what the transition graph operator needs to display a graph with transition probabilities (i.e., a markov chain)

- but I don't know how to connect those two elements... there doesn't seem to be a way to select the pairwise table from the result of the transition matrix and feed that into the transition graph Smiley Sad

Screenshot 2017-04-09 19.26.19.png

 

Any idea? Thanks in advance!

 

Here's my xml:

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="45" y="34">
        <parameter key="csv_file" value="/Users/schneibe/Dropbox/Windows/Kinect/Group/Default82-0-Kinect-Log-2017-03-27 08-23-20.csv"/>
        <parameter key="column_separators" value=","/>
        <parameter key="first_row_as_names" value="false"/>
        <list key="annotations">
          <parameter key="0" value="Name"/>
        </list>
        <parameter key="encoding" value="UTF-8"/>
        <list key="data_set_meta_data_information">
          <parameter key="0" value="Timestamp.true.polynominal.attribute"/>
          <parameter key="1" value="Session.true.polynominal.attribute"/>
          <parameter key="2" value="Index.true.integer.attribute"/>
          <parameter key="3" value="BodyID.true.integer.attribute"/>
          <parameter key="4" value="SpineBase_X.true.real.attribute"/>
          <parameter key="5" value="SpineBase_Y.true.real.attribute"/>
          <parameter key="6" value="SpineBase_Z.true.real.attribute"/>
          <parameter key="7" value="SpineBase_inferred.true.integer.attribute"/>
          <parameter key="8" value="SpineMid_X.true.real.attribute"/>
          <parameter key="9" value="SpineMid_Y.true.real.attribute"/>
          <parameter key="10" value="SpineMid_Z.true.real.attribute"/>
          <parameter key="11" value="SpineMid_inferred.true.integer.attribute"/>
          <parameter key="12" value="Neck_X.true.real.attribute"/>
          <parameter key="13" value="Neck_Y.true.real.attribute"/>
          <parameter key="14" value="Neck_Z.true.real.attribute"/>
          <parameter key="15" value="Neck_inferred.true.integer.attribute"/>
          <parameter key="16" value="Head_X.true.real.attribute"/>
          <parameter key="17" value="Head_Y.true.real.attribute"/>
          <parameter key="18" value="Head_Z.true.real.attribute"/>
          <parameter key="19" value="Head_inferred.true.integer.attribute"/>
          <parameter key="20" value="ShoulderLeft_X.true.real.attribute"/>
          <parameter key="21" value="ShoulderLeft_Y.true.real.attribute"/>
          <parameter key="22" value="ShoulderLeft_Z.true.real.attribute"/>
          <parameter key="23" value="ShoulderLeft_inferred.true.integer.attribute"/>
          <parameter key="24" value="ElbowLeft_X.true.real.attribute"/>
          <parameter key="25" value="ElbowLeft_Y.true.real.attribute"/>
          <parameter key="26" value="ElbowLeft_Z.true.real.attribute"/>
          <parameter key="27" value="ElbowLeft_inferred.true.integer.attribute"/>
          <parameter key="28" value="WristLeft_X.true.real.attribute"/>
          <parameter key="29" value="WristLeft_Y.true.real.attribute"/>
          <parameter key="30" value="WristLeft_Z.true.real.attribute"/>
          <parameter key="31" value="WristLeft_inferred.true.integer.attribute"/>
          <parameter key="32" value="HandLeft_X.true.real.attribute"/>
          <parameter key="33" value="HandLeft_Y.true.real.attribute"/>
          <parameter key="34" value="HandLeft_Z.true.real.attribute"/>
          <parameter key="35" value="HandLeft_inferred.true.integer.attribute"/>
          <parameter key="36" value="ShoulderRight_X.true.real.attribute"/>
          <parameter key="37" value="ShoulderRight_Y.true.real.attribute"/>
          <parameter key="38" value="ShoulderRight_Z.true.real.attribute"/>
          <parameter key="39" value="ShoulderRight_inferred.true.integer.attribute"/>
          <parameter key="40" value="ElbowRight_X.true.real.attribute"/>
          <parameter key="41" value="ElbowRight_Y.true.real.attribute"/>
          <parameter key="42" value="ElbowRight_Z.true.real.attribute"/>
          <parameter key="43" value="ElbowRight_inferred.true.integer.attribute"/>
          <parameter key="44" value="WristRight_X.true.real.attribute"/>
          <parameter key="45" value="WristRight_Y.true.real.attribute"/>
          <parameter key="46" value="WristRight_Z.true.real.attribute"/>
          <parameter key="47" value="WristRight_inferred.true.integer.attribute"/>
          <parameter key="48" value="HandRight_X.true.real.attribute"/>
          <parameter key="49" value="HandRight_Y.true.real.attribute"/>
          <parameter key="50" value="HandRight_Z.true.real.attribute"/>
          <parameter key="51" value="HandRight_inferred.true.integer.attribute"/>
          <parameter key="52" value="HipLeft_X.true.real.attribute"/>
          <parameter key="53" value="HipLeft_Y.true.real.attribute"/>
          <parameter key="54" value="HipLeft_Z.true.real.attribute"/>
          <parameter key="55" value="HipLeft_inferred.true.integer.attribute"/>
          <parameter key="56" value="HipRight_X.true.real.attribute"/>
          <parameter key="57" value="HipRight_Y.true.real.attribute"/>
          <parameter key="58" value="HipRight_Z.true.real.attribute"/>
          <parameter key="59" value="HipRight_inferred.true.integer.attribute"/>
          <parameter key="60" value="SpineShoulder_X.true.real.attribute"/>
          <parameter key="61" value="SpineShoulder_Y.true.real.attribute"/>
          <parameter key="62" value="SpineShoulder_Z.true.real.attribute"/>
          <parameter key="63" value="SpineShoulder_inferred.true.integer.attribute"/>
          <parameter key="64" value="HandTipLeft_X.true.real.attribute"/>
          <parameter key="65" value="HandTipLeft_Y.true.real.attribute"/>
          <parameter key="66" value="HandTipLeft_Z.true.real.attribute"/>
          <parameter key="67" value="HandTipLeft_inferred.true.integer.attribute"/>
          <parameter key="68" value="ThumbLeft_X.true.real.attribute"/>
          <parameter key="69" value="ThumbLeft_Y.true.real.attribute"/>
          <parameter key="70" value="ThumbLeft_Z.true.real.attribute"/>
          <parameter key="71" value="ThumbLeft_inferred.true.integer.attribute"/>
          <parameter key="72" value="HandTipRight_X.true.real.attribute"/>
          <parameter key="73" value="HandTipRight_Y.true.real.attribute"/>
          <parameter key="74" value="HandTipRight_Z.true.real.attribute"/>
          <parameter key="75" value="HandTipRight_inferred.true.integer.attribute"/>
          <parameter key="76" value="ThumbRight_X.true.real.attribute"/>
          <parameter key="77" value="ThumbRight_Y.true.real.attribute"/>
          <parameter key="78" value="ThumbRight_Z.true.real.attribute"/>
          <parameter key="79" value="ThumbRight_inferred.true.integer.attribute"/>
          <parameter key="80" value="KneeLeft_X.true.real.attribute"/>
          <parameter key="81" value="KneeLeft_Y.true.real.attribute"/>
          <parameter key="82" value="KneeLeft_Z.true.real.attribute"/>
          <parameter key="83" value="KneeLeft_inferred.true.integer.attribute"/>
          <parameter key="84" value="AnkleLeft_X.true.real.attribute"/>
          <parameter key="85" value="AnkleLeft_Y.true.real.attribute"/>
          <parameter key="86" value="AnkleLeft_Z.true.real.attribute"/>
          <parameter key="87" value="AnkleLeft_inferred.true.integer.attribute"/>
          <parameter key="88" value="FootLeft_X.true.real.attribute"/>
          <parameter key="89" value="FootLeft_Y.true.real.attribute"/>
          <parameter key="90" value="FootLeft_Z.true.real.attribute"/>
          <parameter key="91" value="FootLeft_inferred.true.integer.attribute"/>
          <parameter key="92" value="KneeRight_X.true.real.attribute"/>
          <parameter key="93" value="KneeRight_Y.true.real.attribute"/>
          <parameter key="94" value="KneeRight_Z.true.real.attribute"/>
          <parameter key="95" value="KneeRight_inferred.true.integer.attribute"/>
          <parameter key="96" value="AnkleRight_X.true.real.attribute"/>
          <parameter key="97" value="AnkleRight_Y.true.real.attribute"/>
          <parameter key="98" value="AnkleRight_Z.true.real.attribute"/>
          <parameter key="99" value="AnkleRight_inferred.true.integer.attribute"/>
          <parameter key="100" value="FootRight_X.true.real.attribute"/>
          <parameter key="101" value="FootRight_Y.true.real.attribute"/>
          <parameter key="102" value="FootRight_Z.true.real.attribute"/>
          <parameter key="103" value="FootRight_inferred.true.integer.attribute"/>
          <parameter key="104" value="HandLeftState.true.polynominal.attribute"/>
          <parameter key="105" value="HandLeftStateConfidence.true.polynominal.attribute"/>
          <parameter key="106" value="HandRightState.true.polynominal.attribute"/>
          <parameter key="107" value="HandRightStateConfidence.true.polynominal.attribute"/>
          <parameter key="108" value="Lean_X.true.real.attribute"/>
          <parameter key="109" value="Lean_Y.true.real.attribute"/>
          <parameter key="110" value="Lean_TrackingState.true.polynominal.attribute"/>
          <parameter key="111" value="SpineBase_movement.true.real.attribute"/>
          <parameter key="112" value="SpineShoulder_movement.true.attribute_value.attribute"/>
          <parameter key="113" value="SpineMid_movement.true.real.attribute"/>
          <parameter key="114" value="Neck_movement.true.real.attribute"/>
          <parameter key="115" value="Head_movement.true.real.attribute"/>
          <parameter key="116" value="ShoulderLeft_movement.true.real.attribute"/>
          <parameter key="117" value="ElbowLeft_movement.true.real.attribute"/>
          <parameter key="118" value="WristLeft_movement.true.real.attribute"/>
          <parameter key="119" value="HandLeft_movement.true.real.attribute"/>
          <parameter key="120" value="ShoulderRight_movement.true.real.attribute"/>
          <parameter key="121" value="ElbowRight_movement.true.real.attribute"/>
          <parameter key="122" value="WristRight_movement.true.real.attribute"/>
          <parameter key="123" value="HandRight_movement.true.real.attribute"/>
          <parameter key="124" value="HipLeft_movement.true.real.attribute"/>
          <parameter key="125" value="HipRight_movement.true.attribute_value.attribute"/>
          <parameter key="126" value="HandTipLeft_movement.true.real.attribute"/>
          <parameter key="127" value="ThumbLeft_movement.true.real.attribute"/>
          <parameter key="128" value="HandTipRight_movement.true.attribute_value.attribute"/>
          <parameter key="129" value="ThumbRight_movement.true.real.attribute"/>
          <parameter key="130" value="KneeLeft_movement.true.real.attribute"/>
          <parameter key="131" value="AnkleLeft_movement.true.real.attribute"/>
          <parameter key="132" value="FootLeft_movement.true.real.attribute"/>
          <parameter key="133" value="KneeRight_movement.true.real.attribute"/>
          <parameter key="134" value="AnkleRight_movement.true.real.attribute"/>
          <parameter key="135" value="FootRight_movement.true.real.attribute"/>
          <parameter key="136" value="Neck_angle.true.real.attribute"/>
          <parameter key="137" value="Spine_angle.true.real.attribute"/>
          <parameter key="138" value="Hip_angle.true.real.attribute"/>
          <parameter key="139" value="ShoulderL_angle.true.real.attribute"/>
          <parameter key="140" value="ShoulderR_angle.true.real.attribute"/>
          <parameter key="141" value="ElbowL_angle.true.real.attribute"/>
          <parameter key="142" value="ElbowR_angle.true.real.attribute"/>
          <parameter key="143" value="WristL_angle.true.real.attribute"/>
          <parameter key="144" value="WristR_angle.true.real.attribute"/>
          <parameter key="145" value="HandL_angle.true.real.attribute"/>
          <parameter key="146" value="HandR_angle.true.real.attribute"/>
          <parameter key="147" value="Happy.true.attribute_value.attribute"/>
          <parameter key="148" value="Engaged.true.attribute_value.attribute"/>
          <parameter key="149" value="WearingGlasses.true.attribute_value.attribute"/>
          <parameter key="150" value="LeftEyeClosed.true.attribute_value.attribute"/>
          <parameter key="151" value="RightEyeClosed.true.attribute_value.attribute"/>
          <parameter key="152" value="MouthOpen.true.attribute_value.attribute"/>
          <parameter key="153" value="MouthMoved.true.attribute_value.attribute"/>
          <parameter key="154" value="LookingAway.true.attribute_value.attribute"/>
          <parameter key="155" value="FaceYaw.true.attribute_value.attribute"/>
          <parameter key="156" value="FacePitch.true.attribute_value.attribute"/>
          <parameter key="157" value="FacenRoll.true.attribute_value.attribute"/>
          <parameter key="158" value="Talking.true.integer.attribute"/>
          <parameter key="159" value="Posture.true.attribute_value.attribute"/>
        </list>
      </operator>
      <operator activated="true" class="generate_id" compatibility="7.4.000" expanded="true" height="82" name="Generate ID" width="90" x="112" y="136"/>
      <operator activated="true" class="select_attributes" compatibility="7.4.000" expanded="true" height="82" name="Select Attributes" width="90" x="179" y="289">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attributes" value="WristR_angle|WristL_angle|Spine_angle|ShoulderR_angle|ShoulderL_angle|Neck_angle|Hip_angle|HandR_angle|HandL_angle|ElbowR_angle|ElbowL_angle"/>
      </operator>
      <operator activated="true" class="x_means" compatibility="7.4.000" expanded="true" height="82" name="X-Means" width="90" x="313" y="34"/>
      <operator activated="true" class="join" compatibility="7.4.000" expanded="true" height="82" name="Join" width="90" x="447" y="187">
        <list key="key_attributes"/>
      </operator>
      <operator activated="true" class="transition_matrix" compatibility="7.4.000" expanded="true" height="82" name="Transition Matrix" width="90" x="514" y="34">
        <parameter key="attribute" value="cluster"/>
      </operator>
      <operator activated="false" class="transition_graph" compatibility="7.4.000" expanded="true" height="82" name="Transition Graph" width="90" x="782" y="187">
        <parameter key="source_attribute" value="cluster"/>
        <parameter key="target_attribute" value="cluster"/>
      </operator>
      <connect from_op="Read CSV" from_port="output" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_op="X-Means" to_port="example set"/>
      <connect from_op="Select Attributes" from_port="original" to_op="Join" to_port="right"/>
      <connect from_op="X-Means" from_port="clustered set" to_op="Join" to_port="left"/>
      <connect from_op="Join" from_port="join" to_op="Transition Matrix" to_port="example set"/>
      <connect from_op="Transition Matrix" from_port="example set" to_port="result 1"/>
      <connect from_op="Transition Matrix" from_port="transition matrix" to_port="result 2"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>
5 REPLIES
Community Manager

Re: Markov Chain (transition matrix and graph)

There's a new extension called the Coverter extension that let's you output matrices as examplesets but I'm not sure if they'll do pairwise tables. Maybe @ey or @mschmitz can shed some light on this?

Regards,
T-Bone
Twitter: @neuralmarket
RMStaff

Re: Markov Chain (transition matrix and graph)

I think you can. Otherwise simply move for depivot afterwards Smiley Happy

 

@DArnu is the expert

--------------------------------------------------------------------------
Head of Data Science Services at RapidMiner
Contributor II

Re: Markov Chain (transition matrix and graph)

Great, the Converters extension did the trick. Thanks for your help!!

 

Here's the xml:

 

<?xml version="1.0" encoding="UTF-8"?><process version="7.4.000">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="7.4.000" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="read_csv" compatibility="7.4.000" expanded="true" height="68" name="Read CSV" width="90" x="45" y="34">
        <parameter key="csv_file" value="/Users/schneibe/Dropbox/Windows/Kinect/Group/Default82-0-Kinect-Log-2017-03-27 08-23-20.csv"/>
        <parameter key="column_separators" value=","/>
        <parameter key="first_row_as_names" value="false"/>
        <list key="annotations">
          <parameter key="0" value="Name"/>
        </list>
        <parameter key="encoding" value="UTF-8"/>
        <list key="data_set_meta_data_information">
          <parameter key="0" value="Timestamp.true.polynominal.attribute"/>
          <parameter key="1" value="Session.true.polynominal.attribute"/>
          <parameter key="2" value="Index.true.integer.attribute"/>
          <parameter key="3" value="BodyID.true.integer.attribute"/>
          <parameter key="4" value="SpineBase_X.true.real.attribute"/>
          <parameter key="5" value="SpineBase_Y.true.real.attribute"/>
          <parameter key="6" value="SpineBase_Z.true.real.attribute"/>
          <parameter key="7" value="SpineBase_inferred.true.integer.attribute"/>
          <parameter key="8" value="SpineMid_X.true.real.attribute"/>
          <parameter key="9" value="SpineMid_Y.true.real.attribute"/>
          <parameter key="10" value="SpineMid_Z.true.real.attribute"/>
          <parameter key="11" value="SpineMid_inferred.true.integer.attribute"/>
          <parameter key="12" value="Neck_X.true.real.attribute"/>
          <parameter key="13" value="Neck_Y.true.real.attribute"/>
          <parameter key="14" value="Neck_Z.true.real.attribute"/>
          <parameter key="15" value="Neck_inferred.true.integer.attribute"/>
          <parameter key="16" value="Head_X.true.real.attribute"/>
          <parameter key="17" value="Head_Y.true.real.attribute"/>
          <parameter key="18" value="Head_Z.true.real.attribute"/>
          <parameter key="19" value="Head_inferred.true.integer.attribute"/>
          <parameter key="20" value="ShoulderLeft_X.true.real.attribute"/>
          <parameter key="21" value="ShoulderLeft_Y.true.real.attribute"/>
          <parameter key="22" value="ShoulderLeft_Z.true.real.attribute"/>
          <parameter key="23" value="ShoulderLeft_inferred.true.integer.attribute"/>
          <parameter key="24" value="ElbowLeft_X.true.real.attribute"/>
          <parameter key="25" value="ElbowLeft_Y.true.real.attribute"/>
          <parameter key="26" value="ElbowLeft_Z.true.real.attribute"/>
          <parameter key="27" value="ElbowLeft_inferred.true.integer.attribute"/>
          <parameter key="28" value="WristLeft_X.true.real.attribute"/>
          <parameter key="29" value="WristLeft_Y.true.real.attribute"/>
          <parameter key="30" value="WristLeft_Z.true.real.attribute"/>
          <parameter key="31" value="WristLeft_inferred.true.integer.attribute"/>
          <parameter key="32" value="HandLeft_X.true.real.attribute"/>
          <parameter key="33" value="HandLeft_Y.true.real.attribute"/>
          <parameter key="34" value="HandLeft_Z.true.real.attribute"/>
          <parameter key="35" value="HandLeft_inferred.true.integer.attribute"/>
          <parameter key="36" value="ShoulderRight_X.true.real.attribute"/>
          <parameter key="37" value="ShoulderRight_Y.true.real.attribute"/>
          <parameter key="38" value="ShoulderRight_Z.true.real.attribute"/>
          <parameter key="39" value="ShoulderRight_inferred.true.integer.attribute"/>
          <parameter key="40" value="ElbowRight_X.true.real.attribute"/>
          <parameter key="41" value="ElbowRight_Y.true.real.attribute"/>
          <parameter key="42" value="ElbowRight_Z.true.real.attribute"/>
          <parameter key="43" value="ElbowRight_inferred.true.integer.attribute"/>
          <parameter key="44" value="WristRight_X.true.real.attribute"/>
          <parameter key="45" value="WristRight_Y.true.real.attribute"/>
          <parameter key="46" value="WristRight_Z.true.real.attribute"/>
          <parameter key="47" value="WristRight_inferred.true.integer.attribute"/>
          <parameter key="48" value="HandRight_X.true.real.attribute"/>
          <parameter key="49" value="HandRight_Y.true.real.attribute"/>
          <parameter key="50" value="HandRight_Z.true.real.attribute"/>
          <parameter key="51" value="HandRight_inferred.true.integer.attribute"/>
          <parameter key="52" value="HipLeft_X.true.real.attribute"/>
          <parameter key="53" value="HipLeft_Y.true.real.attribute"/>
          <parameter key="54" value="HipLeft_Z.true.real.attribute"/>
          <parameter key="55" value="HipLeft_inferred.true.integer.attribute"/>
          <parameter key="56" value="HipRight_X.true.real.attribute"/>
          <parameter key="57" value="HipRight_Y.true.real.attribute"/>
          <parameter key="58" value="HipRight_Z.true.real.attribute"/>
          <parameter key="59" value="HipRight_inferred.true.integer.attribute"/>
          <parameter key="60" value="SpineShoulder_X.true.real.attribute"/>
          <parameter key="61" value="SpineShoulder_Y.true.real.attribute"/>
          <parameter key="62" value="SpineShoulder_Z.true.real.attribute"/>
          <parameter key="63" value="SpineShoulder_inferred.true.integer.attribute"/>
          <parameter key="64" value="HandTipLeft_X.true.real.attribute"/>
          <parameter key="65" value="HandTipLeft_Y.true.real.attribute"/>
          <parameter key="66" value="HandTipLeft_Z.true.real.attribute"/>
          <parameter key="67" value="HandTipLeft_inferred.true.integer.attribute"/>
          <parameter key="68" value="ThumbLeft_X.true.real.attribute"/>
          <parameter key="69" value="ThumbLeft_Y.true.real.attribute"/>
          <parameter key="70" value="ThumbLeft_Z.true.real.attribute"/>
          <parameter key="71" value="ThumbLeft_inferred.true.integer.attribute"/>
          <parameter key="72" value="HandTipRight_X.true.real.attribute"/>
          <parameter key="73" value="HandTipRight_Y.true.real.attribute"/>
          <parameter key="74" value="HandTipRight_Z.true.real.attribute"/>
          <parameter key="75" value="HandTipRight_inferred.true.integer.attribute"/>
          <parameter key="76" value="ThumbRight_X.true.real.attribute"/>
          <parameter key="77" value="ThumbRight_Y.true.real.attribute"/>
          <parameter key="78" value="ThumbRight_Z.true.real.attribute"/>
          <parameter key="79" value="ThumbRight_inferred.true.integer.attribute"/>
          <parameter key="80" value="KneeLeft_X.true.real.attribute"/>
          <parameter key="81" value="KneeLeft_Y.true.real.attribute"/>
          <parameter key="82" value="KneeLeft_Z.true.real.attribute"/>
          <parameter key="83" value="KneeLeft_inferred.true.integer.attribute"/>
          <parameter key="84" value="AnkleLeft_X.true.real.attribute"/>
          <parameter key="85" value="AnkleLeft_Y.true.real.attribute"/>
          <parameter key="86" value="AnkleLeft_Z.true.real.attribute"/>
          <parameter key="87" value="AnkleLeft_inferred.true.integer.attribute"/>
          <parameter key="88" value="FootLeft_X.true.real.attribute"/>
          <parameter key="89" value="FootLeft_Y.true.real.attribute"/>
          <parameter key="90" value="FootLeft_Z.true.real.attribute"/>
          <parameter key="91" value="FootLeft_inferred.true.integer.attribute"/>
          <parameter key="92" value="KneeRight_X.true.real.attribute"/>
          <parameter key="93" value="KneeRight_Y.true.real.attribute"/>
          <parameter key="94" value="KneeRight_Z.true.real.attribute"/>
          <parameter key="95" value="KneeRight_inferred.true.integer.attribute"/>
          <parameter key="96" value="AnkleRight_X.true.real.attribute"/>
          <parameter key="97" value="AnkleRight_Y.true.real.attribute"/>
          <parameter key="98" value="AnkleRight_Z.true.real.attribute"/>
          <parameter key="99" value="AnkleRight_inferred.true.integer.attribute"/>
          <parameter key="100" value="FootRight_X.true.real.attribute"/>
          <parameter key="101" value="FootRight_Y.true.real.attribute"/>
          <parameter key="102" value="FootRight_Z.true.real.attribute"/>
          <parameter key="103" value="FootRight_inferred.true.integer.attribute"/>
          <parameter key="104" value="HandLeftState.true.polynominal.attribute"/>
          <parameter key="105" value="HandLeftStateConfidence.true.polynominal.attribute"/>
          <parameter key="106" value="HandRightState.true.polynominal.attribute"/>
          <parameter key="107" value="HandRightStateConfidence.true.polynominal.attribute"/>
          <parameter key="108" value="Lean_X.true.real.attribute"/>
          <parameter key="109" value="Lean_Y.true.real.attribute"/>
          <parameter key="110" value="Lean_TrackingState.true.polynominal.attribute"/>
          <parameter key="111" value="SpineBase_movement.true.real.attribute"/>
          <parameter key="112" value="SpineShoulder_movement.true.attribute_value.attribute"/>
          <parameter key="113" value="SpineMid_movement.true.real.attribute"/>
          <parameter key="114" value="Neck_movement.true.real.attribute"/>
          <parameter key="115" value="Head_movement.true.real.attribute"/>
          <parameter key="116" value="ShoulderLeft_movement.true.real.attribute"/>
          <parameter key="117" value="ElbowLeft_movement.true.real.attribute"/>
          <parameter key="118" value="WristLeft_movement.true.real.attribute"/>
          <parameter key="119" value="HandLeft_movement.true.real.attribute"/>
          <parameter key="120" value="ShoulderRight_movement.true.real.attribute"/>
          <parameter key="121" value="ElbowRight_movement.true.real.attribute"/>
          <parameter key="122" value="WristRight_movement.true.real.attribute"/>
          <parameter key="123" value="HandRight_movement.true.real.attribute"/>
          <parameter key="124" value="HipLeft_movement.true.real.attribute"/>
          <parameter key="125" value="HipRight_movement.true.attribute_value.attribute"/>
          <parameter key="126" value="HandTipLeft_movement.true.real.attribute"/>
          <parameter key="127" value="ThumbLeft_movement.true.real.attribute"/>
          <parameter key="128" value="HandTipRight_movement.true.attribute_value.attribute"/>
          <parameter key="129" value="ThumbRight_movement.true.real.attribute"/>
          <parameter key="130" value="KneeLeft_movement.true.real.attribute"/>
          <parameter key="131" value="AnkleLeft_movement.true.real.attribute"/>
          <parameter key="132" value="FootLeft_movement.true.real.attribute"/>
          <parameter key="133" value="KneeRight_movement.true.real.attribute"/>
          <parameter key="134" value="AnkleRight_movement.true.real.attribute"/>
          <parameter key="135" value="FootRight_movement.true.real.attribute"/>
          <parameter key="136" value="Neck_angle.true.real.attribute"/>
          <parameter key="137" value="Spine_angle.true.real.attribute"/>
          <parameter key="138" value="Hip_angle.true.real.attribute"/>
          <parameter key="139" value="ShoulderL_angle.true.real.attribute"/>
          <parameter key="140" value="ShoulderR_angle.true.real.attribute"/>
          <parameter key="141" value="ElbowL_angle.true.real.attribute"/>
          <parameter key="142" value="ElbowR_angle.true.real.attribute"/>
          <parameter key="143" value="WristL_angle.true.real.attribute"/>
          <parameter key="144" value="WristR_angle.true.real.attribute"/>
          <parameter key="145" value="HandL_angle.true.real.attribute"/>
          <parameter key="146" value="HandR_angle.true.real.attribute"/>
          <parameter key="147" value="Happy.true.attribute_value.attribute"/>
          <parameter key="148" value="Engaged.true.attribute_value.attribute"/>
          <parameter key="149" value="WearingGlasses.true.attribute_value.attribute"/>
          <parameter key="150" value="LeftEyeClosed.true.attribute_value.attribute"/>
          <parameter key="151" value="RightEyeClosed.true.attribute_value.attribute"/>
          <parameter key="152" value="MouthOpen.true.attribute_value.attribute"/>
          <parameter key="153" value="MouthMoved.true.attribute_value.attribute"/>
          <parameter key="154" value="LookingAway.true.attribute_value.attribute"/>
          <parameter key="155" value="FaceYaw.true.attribute_value.attribute"/>
          <parameter key="156" value="FacePitch.true.attribute_value.attribute"/>
          <parameter key="157" value="FacenRoll.true.attribute_value.attribute"/>
          <parameter key="158" value="Talking.true.integer.attribute"/>
          <parameter key="159" value="Posture.true.attribute_value.attribute"/>
        </list>
      </operator>
      <operator activated="true" class="generate_id" compatibility="7.4.000" expanded="true" height="82" name="Generate ID" width="90" x="45" y="136"/>
      <operator activated="true" class="replace_missing_values" compatibility="7.4.000" expanded="true" height="103" name="Replace Missing Values" width="90" x="45" y="289">
        <list key="columns"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="7.4.000" expanded="true" height="82" name="Select Attributes" width="90" x="179" y="289">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attributes" value="ElbowL_angle|ElbowR_angle|Hip_angle|Neck_angle|ShoulderL_angle|ShoulderR_angle|Spine_angle|WristR_angle|WristL_angle|HandR_angle|HandL_angle"/>
      </operator>
      <operator activated="true" class="x_means" compatibility="7.4.000" expanded="true" height="82" name="X-Means" width="90" x="380" y="34"/>
      <operator activated="true" class="join" compatibility="7.4.000" expanded="true" height="82" name="Join" width="90" x="514" y="289">
        <list key="key_attributes"/>
      </operator>
      <operator activated="true" class="multiply" compatibility="7.4.000" expanded="true" height="103" name="Multiply" width="90" x="648" y="187"/>
      <operator activated="true" class="transition_matrix" compatibility="7.4.000" expanded="true" height="82" name="Transition Matrix" width="90" x="782" y="340">
        <parameter key="attribute" value="cluster"/>
      </operator>
      <operator activated="true" class="converters:matrix_2_example_set" compatibility="0.2.000" expanded="true" height="82" name="Matrix to ExampleSet" width="90" x="782" y="493">
        <parameter key="pairwise_list" value="true"/>
      </operator>
      <operator activated="true" class="extract_prototypes" compatibility="7.4.000" expanded="true" height="82" name="Extract Cluster Prototypes" width="90" x="581" y="34"/>
      <operator activated="true" class="python_scripting:execute_python" compatibility="7.2.000" expanded="true" height="103" name="Execute Python (2)" width="90" x="782" y="34">
        <parameter key="script" value="import pandas&#10;import os&#10;import copy&#10;import matplotlib.pyplot as plt&#10;from scipy import spatial&#10;from matplotlib.patches import Ellipse&#10;from mpl_toolkits.mplot3d import Axes3D&#10;&#10;connections = {('Head','Neck'):0.0, &#10;               ('Neck','SpineMid'):0.0,&#10;               ('SpineMid','SpineBase'):0.0,&#10;               ('Neck','ShoulderLeft'):0.3,&#10;               ('Neck','ShoulderRight'):0.3,&#10;               ('ShoulderLeft','ElbowLeft'):0.4,&#10;               ('ShoulderRight','ElbowRight'):0.4,&#10;               ('ElbowLeft','WristLeft'):0.5,&#10;               ('ElbowRight','WristRight'):0.5,&#10;               ('WristLeft','HandLeft'):0.6,&#10;               ('WristRight','HandRight'):0.6,&#10;               #('WristLeft','ThumbLeft'):0.7,&#10;               #('WristRight','ThumbRight'):0.7,&#10;               #('HandLeft','HandTipLeft'):0.8,&#10;               #('HandRight','HandTipRight'):0.8,&#10;               #('SpineBase','HipLeft'):0.0,&#10;               #('SpineBase','HipRight'):0.0&#10;              }&#10;&#10;def find_rows_kmedoids(data,centers):&#10;    &#10;    rows = []&#10;    &#10;    for center_i in range(centers.shape[0]):&#10;        row = data&#10;        for col in centers.columns:&#10;            value = centers.iloc[center_i][col]&#10;            row = row.loc[(row[col] == value)]&#10;            if row.shape[0] == 1: break&#10;        rows.append(row)&#10;    return rows&#10;&#10;def find_rows_kmeans(data,centers):&#10;    &#10;    # we remove the &quot;unnamed&quot; and &quot;cluster&quot; columns&#10;    center_cols = list(centers.columns)[1:-1]&#10;    columns = list(set(data.columns).intersection(center_cols))&#10;    tree = spatial.KDTree(data[columns])&#10;    rows = []&#10;    &#10;    for center_i in range(centers.shape[0]):&#10;        row = data&#10;        similarity,index = tree.query(centers.iloc[center_i][columns])&#10;        rows.append(row.loc[(row['id'] == index)])&#10;    return rows&#10;&#10;def save_kinect_centroid(list_rows):&#10;    &#10;    fig = plt.figure(figsize=(3, 3*len(list_rows)))&#10;    &#10;    for i,df in enumerate(list_rows):&#10;        ax = fig.add_subplot(len(list_rows),1,i+1, projection='3d')&#10;        ax.set_xticklabels([])&#10;        ax.set_yticklabels([])&#10;        ax.set_zticklabels([])&#10;        &#10;        for t,color in connections.items():&#10;            a = t[0]&#10;            b = t[1]&#10;            x = (df[a+'_X'].values[0], df[b+'_X'].values[0])&#10;            y = (df[a+'_Y'].values[0], df[b+'_Y'].values[0])&#10;            z = (df[a+'_Z'].values[0], df[b+'_Z'].values[0])&#10;            &#10;            if a == 'Head' and b == 'Neck':&#10;                ax.plot(x, y, z, color=str(color), &#10;                        solid_capstyle='round', lw=6.0)&#10;            else: ax.plot(x, y, z, color=str(color), lw=4.0)&#10;            &#10;    fig.savefig(os.path.expanduser(&quot;~/Desktop/centroid.pdf&quot;))&#10;&#10;# rm_main is a mandatory function, &#10;# the number of arguments has to be the number of input ports (can be none)&#10;def rm_main(centers, data):&#10;    &#10;    # find complete rows for the cluster centers&#10;    #rows = find_rows_kmedoids(data,centers)&#10;    rows = find_rows_kmeans(data,centers)&#10;&#10;    #your code goes here&#10;    save_kinect_centroid(rows)&#10;&#10;    # connect 2 output ports to see the results&#10;    return data, centers"/>
      </operator>
      <operator activated="true" class="transition_graph" compatibility="7.4.000" expanded="true" height="82" name="Transition Graph" width="90" x="782" y="646">
        <parameter key="source_attribute" value="First Attribute"/>
        <parameter key="target_attribute" value="Second Attribute"/>
        <parameter key="strength_attribute" value="Value"/>
      </operator>
      <connect from_op="Read CSV" from_port="output" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Replace Missing Values" to_port="example set input"/>
      <connect from_op="Replace Missing Values" from_port="example set output" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_op="X-Means" to_port="example set"/>
      <connect from_op="Select Attributes" from_port="original" to_op="Join" to_port="right"/>
      <connect from_op="X-Means" from_port="cluster model" to_op="Extract Cluster Prototypes" to_port="model"/>
      <connect from_op="X-Means" from_port="clustered set" to_op="Join" to_port="left"/>
      <connect from_op="Join" from_port="join" to_op="Multiply" to_port="input"/>
      <connect from_op="Multiply" from_port="output 1" to_op="Execute Python (2)" to_port="input 2"/>
      <connect from_op="Multiply" from_port="output 2" to_op="Transition Matrix" to_port="example set"/>
      <connect from_op="Transition Matrix" from_port="transition matrix" to_op="Matrix to ExampleSet" to_port="matrix"/>
      <connect from_op="Matrix to ExampleSet" from_port="example set" to_op="Transition Graph" to_port="example set"/>
      <connect from_op="Extract Cluster Prototypes" from_port="example set" to_op="Execute Python (2)" to_port="input 1"/>
      <connect from_op="Execute Python (2)" from_port="output 1" to_port="result 1"/>
      <connect from_op="Execute Python (2)" from_port="output 2" to_port="result 2"/>
      <connect from_op="Transition Graph" from_port="example set" to_port="result 3"/>
      <connect from_op="Transition Graph" from_port="transition graph" to_port="result 4"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
      <portSpacing port="sink_result 4" spacing="0"/>
      <portSpacing port="sink_result 5" spacing="0"/>
    </process>
  </operator>
</process>
Contributor II

Re: Markov Chain (transition matrix and graph)

One comment: I just realized that you don't get "real" markov chains with this method.

 

The entire transition matrix sums up to 1:

Screenshot 2017-04-18 09.11.28.png

whereas for a markov chain you would need each column / row to sum up to 1 Smiley Sad

 

any idea of a workaround..?

Contributor II

Re: Markov Chain (transition matrix and graph)

I ended up writing a python script. The input is a dataframe that contains a column named "cluster" and outputs the correct pairwise matrix (that you can can then connect to a transition graph):

 

import pandas as pd
import numpy as np

# rm_main is a mandatory function, 
# the number of arguments has to be the number of input ports (can be none)
def rm_main(data):

    # select the column we want to use        
    column_name = 'cluster'

    # get the data
    actions = list(data[column_name])

    # transform labels into numbers
    actions = [str(act) for act in actions]
    labels_set = sorted(list(set(actions)))
    labels = [labels_set.index(action) for action in actions]

    # get the size of the matrix
    num_clusters = len(list(set(labels)))

    # build the numpy matrix
    transitions_matrix = np.zeros((num_clusters, num_clusters))
    prev = None

    # count the transitions
    for lab in labels:
        if prev != None:
            if not True:
                if prev != lab:
                    transitions_matrix[prev,lab] += 1
            else: 
                transitions_matrix[prev,lab] += 1
        prev = lab
    
    # make it a probability matrix
    for i in range(num_clusters):
        row = transitions_matrix[i,:]
        if sum(row) > 0: 
            transitions_matrix[i,:] = row / sum(row)
        for j in range(num_clusters):
            transitions_matrix[i,j] = round(transitions_matrix[i,j],3)
    
    matrix = pd.DataFrame(transitions_matrix, 
                          index=labels_set, columns=labels_set)
    
    # building a pairwise matrix
    columns_labels = ['First Attribute', 'Second Attribute', 'Value']
    num_rows = num_clusters**2
    pairwise = pd.DataFrame(index=range(num_rows), columns=columns_labels)
    
    # fill the matrix
    index = 0
    for col in range(num_clusters):
        for row in range(num_clusters):
            pairwise.iloc[index][0] = labels_set[col]
            pairwise.iloc[index][1] = labels_set[row]
            pairwise.iloc[index][2] = matrix.iloc[col][row]
            index += 1
    
    pairwise[['Value']] = pairwise[['Value']].apply(pd.to_numeric)
    
    return pairwise