how to generate a neural net weight-matrix?

eldenosoeldenoso Member Posts: 65 Contributor I
edited December 2018 in Help

Hello altogether,

as I am currently trying to get more insight into what neural nets are doing, I wanted to know, whether there is a way of exporting the weight-matrices of a neural net so that you can visualize it (e.g. heat map). By then changing the training set a couple of times, you may see differences in the "heat-map-weight-matrix" and can conclude which attributes are the ones that matter.

I know that there are real scientists out there, searching for answers to the black-box-problem of neural nets. But my curiosity just drives me towards this way and I would be glad to try this out :-).

Thank you for your answers

 

Philipp

Best Answer

  • mschmitzmschmitz Posts: 2,127  RM Data Scientist
    Solution Accepted

    Hi Philipp,

     

    in fact you can use Groovy Script to do this. I've tried this, but it wasn't as easy as i thought. Attached is a process with the script. It always extracts the first Layer. i've not tested it in various layouts though

     

    Best,

    Martin

     

    <?xml version="1.0" encoding="UTF-8"?><process version="7.3.001">
    <context>
    <input/>
    <output/>
    <macros/>
    </context>
    <operator activated="true" class="process" compatibility="7.3.001" expanded="true" name="Process">
    <process expanded="true">
    <operator activated="true" class="retrieve" compatibility="7.3.001" expanded="true" height="68" name="Retrieve Sonar" width="90" x="112" y="34">
    <parameter key="repository_entry" value="//Samples/data/Sonar"/>
    </operator>
    <operator activated="true" class="neural_net" compatibility="7.3.001" expanded="true" height="82" name="Neural Net" width="90" x="313" y="34">
    <list key="hidden_layers">
    <parameter key="1" value="10"/>
    </list>
    </operator>
    <operator activated="true" class="execute_script" compatibility="7.3.001" expanded="true" height="82" name="Execute Script" width="90" x="581" y="34">
    <parameter key="script" value=" import java.util.logging.Level&#10; import com.rapidminer.tools.LogService;&#10; import com.rapidminer.tools.Ontology;&#10; import com.rapidminer.example.utils.ExampleSetBuilder;&#10; import com.rapidminer.example.utils.ExampleSets;&#10;// we assume only one layer&#10;model = input[0]&#10;&#10;int NumberOfNodes = model.innerNodes.size()&#10;int weightLength = model.innerNodes[10].getWeights().size()&#10;&#10;for(int k = 0; k &lt; NumberOfNodes; ++k){&#10;&#9;if(model.innerNodes[k].layerIndex == 0)&#10;&#9;&#9;weightLength = model.innerNodes[k].getWeights().size();&#10;&#10;}&#10;attributes= new Attribute[weightLength];&#10;for(int n = 0; n &lt; weightLength; n++){&#10; &#9;attributes[n] = AttributeFactory.createAttribute(&quot;Weight_&quot;+Integer.toString(n), Ontology.REAL);&#10;}&#10;ExampleSetBuilder builder = ExampleSets.from(attributes)&#10;double[] row = new double[weightLength]&#10;&#10;for(int n = 0; n &lt; NumberOfNodes; n++){&#10;&#9;innerNodes = model.innerNodes[n]&#10;&#9;&#10;&#9;weights = innerNodes.getWeights()&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weights.size()))&#10;&#9;LogService.root.log(Level.SEVERE,Integer.toString(weightLength))&#10;&#9;for(int i = 0; i &lt; weights.size(); ++i){&#10;&#9;&#9;if(innerNodes.layerIndex == 0){&#10;&#9;&#9;&#9;row[i] = weights[i]&#10;&#9;&#9;}&#10;&#9;&#9;//LogService.root.log(Level.INFO,Double.toString(weights[i]))&#10;&#9;}&#10;&#9;if(innerNodes.layerIndex==0)&#10;&#9;&#9;builder.addRow(row)&#10;}&#10;&#10;return builder.build()"/>
    </operator>
    <connect from_op="Retrieve Sonar" from_port="output" to_op="Neural Net" to_port="training set"/>
    <connect from_op="Neural Net" from_port="model" to_op="Execute Script" to_port="input 1"/>
    <connect from_op="Execute Script" from_port="output 1" to_port="result 1"/>
    <portSpacing port="source_input 1" spacing="0"/>
    <portSpacing port="sink_result 1" spacing="0"/>
    <portSpacing port="sink_result 2" spacing="0"/>
    </process>
    </operator>
    </process>

     

    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany

Answers

  • eldenosoeldenoso Member Posts: 65 Contributor I

    Perfect Martin! Thank you. 

    In your case the data means that there were ten neurons in the first layer with 60 edges (weights), right?

    Best, 

    Philipp

  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,127  RM Data Scientist

    Hi,

     

    yes. And two output nodes which are interestingly part of the innerNodes object and just identfiable by their layerId...

     

    Best,

    Martin

    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.