Keras Deep Learning extension
jpuente
Employee, Member Posts: 53 
RM Product Management
Keras is a high level neural network API, supporting popular deep learning libraries like Tensorflow, Microsoft Cognitive Toolkit, and Theano.
The RapidMiner Keras extension provides a set of operators that allow an easy visual configuration of Deep Learning network structures and layers. Calculations are pushed into the Python-based backend libraries, so you can leverage the computing power of GPUs and grid environments.
The extension makes use of an existing Keras installation. This article shows how to do a simple deployment of Keras and how to configure the Keras extension to connect to it.
Let's review several options:
Anaconda on MacOS
Warning: As of version 1.2, TensorFlow no longer provides GPU support on macOS.
- Download and install Anaconda from: https://www.continuum.io/downloads#macos
- Create a new environment by typing in command line: conda create –n keras
- Activate the created environment by typing in the command line: source activate keras
- Install pandas by typing in the command line: conda install pandas
- Install scikit-learn by typing in the command line: conda install scikit-learn
- Install keras by typing in the command line: conda install -c conda-forge keras
- Install graphviz by typing in the command line: conda install –c anaconda graphviz
- Install pydotplus by typing in the commandline conda install –c conda-forge pydotplus
- In RapidMiner Studio Keras and Python Scripting panels in preferences, specify the path to your new conda environment Python executable.
You’re good to go!
Anaconda on Windows
Warning: Due to issues with package dependencies, it is not currently possible to install graphviz and pydot in a conda environment on Windows, and consequently to visualise the model graph in the results panel.
- Download and install Anaconda from: https://www.continuum.io/downloads#windows
- Create a new environment with Python 3.5.2 by typing in command line: conda create –n Python35 python=3.5.2
- Activate the created environment by typing in the command line: activate Python35
- Install pandas by typing in the command line: conda install pandas
- Install scikit-learn by typing in the command line: conda install scikit-learn
- Install keras by typing in the command line: conda install -c jaikumarm keras=2.0.4
- In RapidMiner Studio Keras and Python Scripting panels in preferences, specify the path to your new conda environment Python executable.
You’re good to go!
Windows
- Download and install Python 3.5.2 from: https://www.python.org/downloads/release/python-352/
Only python 3.5.2 works for windows.
- Install numpy with Intel Math Kernel library.
- Download the file 13.1+mklcp35cp35mwin_amd64.whl from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
- cd to the directory where you have downloaded the file and from the command line run: pip3 install 13.1+mklcp35cp35mwin_amd64.whl
- Install pandas from the command line: pip3 install pandas
- Install graphviz from the command line: pip3 install graphviz
- Install pydot from the command line: pip3 install pydot
- Install TensorFlow.
- If you would like to install TensorFlow with GPU support, please see the instructions here: https://www.tensorflow.org/install/install_windows
- If you would like to install TensorFlow only with CPU support, from the command line run: pip3 install –upgrade tensorflow
- Install Keras from the command line: pip3 install keras
You’re good to go!
RapidMiner extension
- Install the Keras extension from the RapidMiner Marketplace
- Install RapidMiner Python Scripting extension from the marketplace if not already installed.
- Restart RapidMiner Studio.
- Inside your Studio Client go to Settings (Menu) > Preferences and navigate to “Python Scripting” tab/page on the left. Provide path to Python executable and click test to ensure it is successful.
- Inside your Studio Client go to Settings (Menu) >Preferences and navigate to “Keras” tab/page on the left. Provide path to Python executable and click test to ensure it is successful.
Try out a few sample processes from the “Keras Sample” in the repository view.
Q&A
Keras Model seems to be running on a CPU or on one GPU only, there is no way of controlling which GPU is to be used and to switch to another at any point in processing.
There is indeed no way to choose a GPU as of yet but in our test all the available gpu's were being used. are you sure you could only use one?
There is no way to utilise models saved as HDF5 checkpoints, I could not even read the HDF5 checkpoint files back into RM anyway.
The Python Scripting Extension, which the Keras extension relies on, currently doesn't support serialising to HDF5, so this will have to wait.
Also not sure if Keras Model could accept anything but a data frame, if so it is curious why the model requires the input shape (the shape for a data frame is always standard).
Indeed, the superoperator can only handle ExampleSets. however, different input layers require different input shapes. if you start with a dense layer, then the input shape could be easily deduced. on the other hand convolutional or recurrent layers require specifying an input shape different than the simple number of features. for example, when using a conv1d layer, the input_shape needs to be (batch_size, timesteps, input_dim) and the pre-processing is done automatically by rapidminer. this is shown in the sample processes
Is it possible to call your own Python code in callbacks or optimizers.
not yet, but it should be made possible in the next update of the extension
I am not sure how to pass multi-label examples.
I get messages .Graphviz not installed and .Pydot not installed
To solve that:
1. Go to Anaconda prompt
2. Activate the last created environment by typing in the command line: activate Python35
3. Install pip by typing in the command line: conda install pip
4. Install graphviz by typing in the command line: pip install graphviz
5. Install pydot by typing in the command line: pip install pydot
6. Inside your Studio Client go to Settings (Menu) >Preferences and navigate to “Keras” tab/page on the left. Provide path to Python executable (the path to your new conda environment Python executable) and click test to ensure it is successful.
More Installation Troubleshooting – courtesy @jacobcybulski
In my case installing Keras for RapidMiner on Win10 was a bit more tricky, and the main issue are pydot and graphviz. When you install them both from Anaconda, RapidMiner will see them but the necessary software is still not on the system and will later fail. Here is my advice, assuming you have the current version of Anaconda, Tensorflow and Keras.
Make sure you install Keras in the previously defined Tensorflow environment, in most cases this was Python 3.5 however recently I have tried Python 3.6 with success. Then activate the environment.
- From the command line execute: activate tensorflow
Then install a number of packages, I.e. (the last two for visualisation only)
- cuDNN (if using GPU, see above)
- conda install HDF5
- conda install h5py (the previous may be downgraded at this point in time)
- conda install graphviz (you may be lucky)
- pip install pydot (you may be lucky)
Now you should be able to finish installing Keras.
- pip install keras
You can now try running RapidMiner, install Keras plugin and configure it by pointing to the python within "tensorflow" environent. Most likely RapidMiner will be happy with the installation but may not be able to display the graphs or it will fail with an error later on.
If you were not lucky then graphviz and / or pydot need to be properly installed, try these:
- Download "graphviz-2.38.msi" from
http://www.graphviz.org/Download_windows.php - Execute the "graphviz-2.38.msi" file
- Add the graphviz bin folder to the PATH system environment variable
(Example: "C:\Graphviz2.38\bin") - You may need one more step, I.e. install a python-graphviz package:
conda install python-graphviz
You should be able to see the graphs produced by Keras within RapidMiner.
However, I have found on a few systems this was not enough and I had to do the following (I have no idea why but I have found this remedy).
- Go to Anaconda Prompt using start menu (Make sure to right click and select "Run as Administrator". We may get permission issues if Prompt as not opened as Administrator)
- Execute the command: conda install graphviz
- Execute the command: pip install git+https://github.com/nlhepler/pydot.git (You will need GIT for this)
In all cases, you may wish to check if things worked out.
- Execute the command "conda list" and make sure pydot and graphviz modules are listed.
More Installation Troubleshooting – courtesy @M_Martin
As far as setting up Keras in RapidMiner (and other related Python packages Keras needs), here's what seems to be working for me on several Dev boxes in my shop (caveat: all boxes are running Windows 7 64 bit, with SP1 - all machines have either 16 or 32 GB of RAM and I7 processors).
1. Install the Keras extension from the Rapid Miner Extensions Marketplace into RapidMiner Studio
2. Download and install Anaconda 3 (https://www.anaconda.com/) Get the appropriate version (64 or 32 bit).
3. Run the Anaconda Navigator (should now be in your Windows Program Group)
4. Create a new environment using the Anaconda Navigator. The Navigator (as of this writing) will suggest using Python 3.6, but there is also an option for Python 3.5, Tick on Python 3.5 as I understand that the RapidMiner Keras extension was developed using Python 3.5. I named my Environment py35.
5. After Anaconda creates the environment, open up the Anaconda Prompt (should be listed as a shortcut from the Start Menu, or within the Anaconda program group. Though I didn't have to, you could left click on the icon for the Anaconda prompt and select "Run as Administrator"
6. You need to activate the new environment you just created. From the prompt type: activate <environment name>. If you named your environment py35, you would type: activate py35. Then hit Enter/Return. After a few seconds, the Anaconda prompt will return, and the environment will be active. The new environment name should now be part of the Anaconda prompt.
7. To see a list of existing Python packages in your new environment, type conda list and then Enter/Return. You should see a short list of packages.
8. You now need to install a few more Python packages. Type conda install pandas and then Enter. After a few seconds, you will be asked to confirm that you want to do the installation. Type y and the downloading and installation of pandas (and other dependent packages) will begin. When the installation has finished, you'll be returned to the Anaconda prompt for your environment.
9. Type conda install scikit-learn and then Enter. Confirm that you want to do the install, and then it should start. When the install is done, you'll be returned to the prompt for your Anaconda environment.
10. You now need to install a package named Graphviz which requires some extra steps. Go to http://www.graphviz.org/ and download_windows.php and download graphviz-238.msi. Then run the msi file you have downloaded to install graphviz (which is a Windows Forms application).
11. Then open the Windows Control Panel, select the System App, and then Advanced System Settings --> Environment Variables. Add the path to the Graphviz executable to (at the least) the PATH environment variable for your user account. The value to append to your existing PATH is C:\Program Files (x86)\Graphviz2.38\bin Type in a semicolon in front of C:\Program Files (x86)\Graphviz2.38\bin in order to seperate it from the previous entry in your PATH statement. For good measure (though it may not be strictly necessary, I also added the following directories to my path statement:
C:\Users\YourUserName\Anaconda3\envs\py35;C:\Users\YourUserNamel\Anaconda3\Scripts;C:\Users\YourUserName\Anaconda3\envs\py35\Lib\site-packages (remember to type in a semicolon after C:\Program Files (x86)\Graphviz2.38\bin before typing in another entry).
Substitute your Windows User Account name for the YourUserName directly above.
11. To confirm that your PATH environment variable value has been updated, open a Command Window and type path and then enter. The value of your PATH environment variable will echo to the screen. If what you see doesn't include the entries you just added, you'll need to re-boot your system and check again.
12. Assuming your PATH has been updated, you can install graphviz (from within the Anaconda prompt for your environment - which should now also show up as a shortcut from the Start Menu or from within the anaconda Program Group) by typing conda install graphviz and then Enter.
13. Then install the pydot package by typing pip install pydot and then Enter
14. Last but not least, install Keras (recently updated to version 2.0.6) by typing conda install -c conda-forge keras and then Enter. After confirming that you want to do the install, Keras and numerous dependent packages will be installed, and you'll be back at the Anaconda prompt for your environment.
15. If you want to use Tensorboard to visualize your models install the latest version of Tensorflow and Tensorboard by typing
pip install --ignore-installed --upgrade tensorflow
and then Enter. Quite a few packages will be installed, and you'll be back at the Anaconda prompt.
For info re: Tensorflow and Tensorboard, visit https://www.tensorflow.org
If you type conda list and then Enter, you will see that your environment now contains many more packages.
The last configuration step needs to occur within RapidMiner Studio by selecting Settings -> Preferences and telling RapidMiner Studio where python.exe is within your Python environment. By default, the complete path should be:
C:\Users\YourUserName\Anaconda3\envs\YourEnvironmentName\python.exe
Click on the disk icon to the left of the Test command button and navigate to python.exe within your environment twice - once for the "Keras" option and once for the "Python Scripting" option in the Preferences dialog. Be sure to click on the "Test" button both times. If there are no errors, you'll get a message box stating that Python has been detected within Anaconda. On all my Dev boxes, there were no errors, hopefully there will be no errors on your system.
Some or all of the set up commands above may not work with Windows 10, but Windows 10 does allow you to set compatibility mode to run various programs, so perhaps experimenting with compatibility settings would help.
The installation described above is a CPU installation as opposed to a GPU installation. GPU installations will run keras models quicker, but have hardware requirements and the install is tricky. For more info on this subject visit https://www.google.ca/search?q=keras+gpu+installation&oq=keras+gpu+installation&aqs=chrome..69i57j69...
You should (hopefully) now be able to run the Keras samples provided with RapidMiner which are in the repository under the entry Keras Samples.
If you want to give Tensorboard a spin, you will need to adjust a few default settings of the Keras Model operator in the process. If you don't want to try Tensorboard, you can run the process and see the outputs in the RapidMiner Results panel.
I attached some screenshots with some example settings if you want to try Tensorboard. Start with the screenshot of the process parameters panel below:
Select a loss metric from the dropdown opposite the loss parameter (I selected mean_squared_error).
Tick on the "use metric" label below the decay parameter. Then then lick on the "Edit enumeration" button opposite the "metric" parameter. A dialog will open, and if you like, you can select an additional metric to visualize. Theoretically, you should be able to select additional metrics, but selecting more than one causes (on my systems) the process to crash. The error message states that the desired metric name (a concatenation of metrics in the enumeration) is invalid, even though the process XML would appear to prevent that type of error message. After selecting a single metric from the enumeration, (I chose mape) click on OK.
Enter .25 for the "validation split" parameter (just below the verbose parameter) - I think that setting the validation split parameter to a non zero value enables the display of histograms and distributions in Tensorboard.
Click on the "Edit enumeration" button opposite the callbacks parameter. Another dialog will open. Clicking on the dropdown menu will expose several default template callback statements. The attached screenshot shows three different callback types (RM allows multiple choices for callbacks) that I set. I don't fully understand how to construct callbacks, especially what you need to specify in order to see embeddings and checkpoints. You will see that one of the callbacks creates a checkpoint file, but it doesn't display in Tensorboard, and I'm not sure why. I would appreciate any guidance on this point. The CSV Logger outputs a csv file containing information re: the values of the metrics you selected during the training process. The screenshot below shows the three callbacks I configured:
As configured in the screenshot, you should be able to see scalar values, images, graphs, distributions, and histograms in Tensorboard. I think that setting the histogram_freq value to 1 and write_grads=True enables histograms and distributions to display in Tensorflow as long as the validation split parameter (see first screen shot above) is set to a non zero value.
The next step is to create a directory using Windows Explorer where the callbacks that Tensorboard will read will be written to disk when RapidMiner executes the process. On my systems, it's C:\TensorboardLogDir
Below is a screenshot of the s&p 500 -regression sample process with a few additions - I added a Performance operator, and I connected the "his" and "exa" ports of the Keras Model operator to process "res" ports. The his (for history) delivers the same information in the csv file generated by the CSV Logger that is referenced in a callback.
Process Design View
After configuring the process, you can run it - which will take a few moments. If you go to your Tensorboard log directory, you should see three files assuming you configured the Tensorboard callback as per the screenshot. There will be an "events out" file, a csv file, and a checkpoint (.ckpt) file. If you set the "write_images" flag to True in the Tensorboard callback, the file will be quite large - several hundred megabytes. If you set it to No, it will be around 100 megabytes (that's what the file sizes were on my systems).
Create a sub-directory (a new folder) within your Tensorboard log directory and move the three files to that new folder.
The next step is to start Tensorboard. From within your Anaconda prompt for your Python environment, type:
tensorboard --logdir="Your Tensorboard Log Directory Name" --host=127.0.0.1
and then Enter.
I needed to put quotes around the log directory name. On my systems, I typed:
tensorboard --logdir=C:\"TensorboardLogDir" --host=127.0.0.1
After a few seconds, Tensorboard will load and display a message to the effect that you should open a browser and navigate to http://127.0.0.1:6006
After entering that URL in your browser, Tensorboard should load after a few seconds. On the left hand side of the screen, near the middle, you'll see the word "runs". You should see a list that has two items, C\ and C\<subdirectory name>. On my systems, its C\1. Click on the SCALARS heading at the top of the browser window and and click on C\<Sun-directory Name> below "runs". Four visualizations of scalar values should appear on your screen. On my screen, these four values are loss, mean_absolute_percentage_error, val_loss, and _val_mean_absolute_percentage_error. I'm not sure why I get four metrics as opposed to two, and I don't understand the difference between "loss" and "val_loss" and "mean_absolute_percentage_error" and "val_mean_absolute_percentage_error". Any guidance on this would be appreciated. The values for all of the visualizations you will see in Tensorboard come from the "events out" file.
If your callback included images, click on IMAGES at the top of the browser window will show images representing your model. I think images are most useful in text mining, but perhaps I may be missing something. Clicking on GRAPHS shows a Graph realization of the neural network, and clicking on DISTRIBUTIONS and HISTOGRAMS shows metadata related to the training progress (the behaviors or layers in the network) over the 256 epochs the training ran.
There is an option called INACTIVE next to HISTOGRAMS. If you click on the INACTIVE option and then click on Projector, you'll get an error message stating that a checkpoint file has not been saved, even though I configured a callback to write a checkpoint file. I'm sure I'm missing something, any suggestions appreciated.
You can then go back to RapidMiner, change the process in some way, and then run the process again. Three more files will be written to your Tensorboard logging directory. Create a new subfolder and move the new files into that subfolder.
Go to Tensorboard in your browser and reload the page. You will then see both of your sub-folders under "runs" near the middle of the left hand side of your browser window. You click single or multiple select these subfolders and the appropriate visualisation(s) will appear. I haven't found a way to toggle between vizzes except by putting files related to each run in its own sub-directory underneath the main Tensorboard log directory.
I hope the above has been helpful. There is still quite a bit of material I don't really yet understand (especially about callbacks), but at the very least, I hope the above will help with getting one through the setup so that one can at least get a feel for what's going on. There's a very informative webinar about Keras and RapidMiner at https://rapidminer.com/resource/state-deep-learning/. It's also on YouTube. There are also many good resources on the web regarding Tensorflow and Tensorboard, one of the best being tensorflow.org.
Notes:
I described how one should modify the PATH environment variable as part of installing the Graphviz package. I also said that for good measure (though it may not be absolutely required) that one should add the following directories to the PATH environment variable:
C:\Users\YourUserName\Anaconda3\envs\py35;C:\Users\YourUserNamel\Anaconda3\Scripts;C:\Users\YourUserName\Anaconda3\envs\py35\Lib\site-packages (remember to type in a semicolon after C:\Program Files (x86)\Graphviz2.38\bin before typing in another entry).
The above should substitute your Python environment name for my environment name (py35) in the statement above.
There was one type pertaining to what histograms and distributions show in Tensorflow. As far as I can tell these elements describe activity in the various layers of the network.
Best wishes, Michael
How to set up Keras on Ubuntu 16:04 LTS - courtesy @jacobcybulski
It is important that you install the versions the documentation says to install and not anything "better".
NVIDIA CUDA Toolkit (only needed for GPU support)
- CUDA 8 web site (at this stage Tensorflow dos not work with CUDA 9):
https://developer.nvidia.com/cuda-downloads - Check that you have a CUDA compatible GPU, I.e.
See the list on https://developer.nvidia.com/cuda-gpus - Download CUDA 8 GA2 x86_64 Deb for Ubuntu 16.04 (1.9Gb) + Patch 2 (128 Mb)
https://developer.nvidia.com/cuda-80-ga2-download-archive - Follow instructions to install drivers and toolkit
If you have a newer driver, install the toolkit manually and skip the driver installation,
place it in <cudapath> (e.g. /usr/local/, change to yours) - Set CUDA variables in your ~/.profile (or ~/.bash_profile), e.g. to:
export PATH=
<cudapath>/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=
<cudapath>/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export LD_LIBRARY_PATH=
<cudapath>/cuda/extras/CUPTI/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export CUDA_HOME=<cudapath>/cuda
(you may need to log out and log in again after editing this file) - Then install cuDNN 6, from web site: https://developer.nvidia.com/cudnn
Note that I have not tested Tensorflow with cuDNN 7 but you could give it a go
You may need to register for download, place it in <cudnnpath> (change it to yours) - Copy the following files into the CUDA Toolkit directory:
$ cd <cudnnpath>
$ sudo cp -P include/cudnn.h <cudapath>/cuda/include
$ sudo cp -P lib64/libcudnn* <cudapath>/cuda/lib64
$ sudo chmod a+r <cudapath>/cuda/lib64/libcudnn*
Anaconda
- Web site:
https://www.continuum.io/ - Download for Python 3.6+, 64bit, from:
https://www.continuum.io/downloads - Install in easily accessible location, e.g.:
~/anaconda3 - Note that in the process of adding Anaconda, you will be prompted to add it to the PATH. If not add it to your ~/.profile:
PATH="<condapath>/anaconda3/bin:$HOME/bin:$HOME/.local/bin:$PATH"
(you may need to log out and log in again after editing this file) - Open a new command line and test it, I.e.
$ conda –version
Tensorflow (It now works with Python 3.6+)
- Web site: https://www.tensorflow.org/
- Follow instructions for Anaconda, I.e. from command line (recently updated):
$ conda create -n tensorflow python=3.6 (or lower, e.g. v3.5)
$ source activate tensorflow (you switch to Tensorflow environment) - Install for CPU only:
(tensorflow) $ pip install --ignore-installed --upgrade tensorflow - Install for GPU only:
(tensorflow) $ pip install --ignore-installed --upgrade tensorflow-gpu - Test your installation by writing a short program, e.g.
(tensorflow) $ python
>>> import tensorflow as tf
>>> hello = tf.constant('Hello, TensorFlow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
>>> ^D - You may also wish to optionally install in that environment (esp. if you decided to use a version of Python different from that in the root):
(tensorflow) $ conda install numpy
(tensorflow) $ conda install scipy
(tensorflow) $ conda install scikit-learn
(tensorflow) $ conda install pandas
(tensorflow) $ conda install matplotlib
(tensorflow) $ conda install jupyter
Keras
- Make sure you install Keras in the previously defined Tensorflow environment, i.e. from the command line execute:
$ source activate tensorflow - Then install a number of packages, I.e.
Already installed cuDNN (if using GPU, see above)
(tensorflow) $ conda install HDF5
(tensorflow) $ conda install h5py (the previous may be downgraded)
(tensorflow) $ conda install graphviz
(tensorflow) $ conda install python-graphviz
(tensorflow) $ pip install pydot
(tensorflow) $ conda list (make sure pydot and graphviz are listed) - If you were not lucky with graphviz and / or pydot, you can leave their installation for later as both are needed only for charting of Keras models. If I ever had any issues with the installation of Keras, these two were the main cause of issues. If all fails, try googling around. You may however try this workaround to install graphviz software separately (Anaconda only provides an interface to the software), i.e.
$ sudo apt-get update
$ sudo apt-get install graphviz - Finish installation with:
(tensorflow) $ pip install Keras - You are now ready to use Keras in Python
RapidMiner Studio
- Download and install RapidMiner Studio
- Install Keras extension from the RapidMiner Marketplace
- In Settings > Preferences > Keras tab,
set Path to Python executable to Python within the Tensorflow environment, e.g.
~/anaconda3/envs/tensorflow/bin/python - You can start modeling with Keras in RapidMiner Studio
- Follow the instructions over at Microsofts installation guide to install their Cognitive Toolkit for Python. Make sure to choose the version matching your python version and selecting the one with GPU support, if you want to execute your process on GPUs as well.
- Install Keras by running `conda install keras` if you are using Anaconda or `pip install keras` if not.
- Run Keras once. E.g. by opening up a command prompt, starting python and running `import keras`.
- Now a `keras.json` file should exists in a hidden folder called `.keras` in your users home directory. It might look like this
{ "backend": "tensorflow", "floatx": "float32", "image_data_format": "channels_last", "epsilon": 1e-07 } - Change the 'backend' value in the json file to 'cntk', save the file.
- Point the RapidMiner Keras Extension to the Python you are using for CNTK (Cognitive Toolkit).
The Missing MNIST Example in Keras for RapidMiner – courtesy @jacobcybulski
As I was reading @kakkad2 comment on convolutional neural nets in Keras, I have realised that we do not have a working example anywhere to show how to deal with CNN in Keras for RM, especially when the application is in image recognition - the very staple of CNN. So I have quickly produced a CNN RM process (see attached at the end). To make it simpler, I have reproduced the Keras Python example from the Francois Challet github, see:
The problems of replicating the MNIST logic in RM are as follows:
- We cannot read image data into RM;
- RM wants all data to be in tabular format;
- Some of the Keras operator's defaults are not the same as those for Python.
To deal with reading images into RM we could utilize the IMMI extension, however at the moment I have no access to IMMI for RM 7.6. The next best option is to get the image data using Python and export it as a Pandas data frame with extra meta info, which is a valid example set. The second issue can be easily handled by unfolding the four dimensional image data (examples X pixel rows X pixel columns X color channels) into two dimensional tables of RM examples (examples X pixel color channel) - the main problem would be to fold back this representation for convolution to happen, also we need to watch for the internal representation of images as passed into RM through Tensorflow or Theano, which could be channel first or last. Finally, we need to check what defaults are given to us by RM Keras (e.g. for optimizers) vs what defaults are defined for the same functions in Python, or better experiment with different settings.
The Python code to read in data is very simple. Note that to mirror exactly what was done in the github example, the data was read from Keras standard data set "mnist", its shape saved, data transformed into floats, and finally both training and test "x" and "y" vectors have been "unfolded" and merged into Pandas data frames with "y" column defined as the RM "label" (using rm_metadata attribute). The shape information was also returned as a data frame, so that it could later be used to set image sizes and shapes in the convolutional nets.
from __future__ import print_function
import os
import numpy as np
import pandas as pd
import keras
from keras.datasets import mnist
from keras import backend as K
### Loads and returns MNIST data set in Pandas format
def rm_main():
# input image dimensions
img_rows, img_cols, ch_no = 28, 28, 1
num_classes = 10
# the data, shuffled and split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
if K.image_data_format() == 'channels_first':
input_shape = (ch_no, img_rows, img_cols)
else:
input_shape = (img_rows, img_cols, ch_no)
x_train = x_train.reshape(x_train.shape[0], img_rows*img_cols*ch_no).astype('float32')
x_test = x_test.reshape(x_test.shape[0], img_rows*img_cols*ch_no).astype('float32')
x_train /= 255
x_test /= 255
# convert image vectors to data frames
df_train = pd.concat([pd.DataFrame(data={'y': y_train}), pd.DataFrame.from_dict(x_train)], axis=1)
setattr(df_train, "rm_metadata", {})
df_train.rm_metadata['y'] = ("nominal","label")
df_test = pd.concat([pd.DataFrame(data={'y': y_test}), pd.DataFrame.from_dict(x_test)], axis=1)
setattr(df_test, "rm_metadata", {})
df_test.rm_metadata['y'] = ("nominal","label")
# Prepare shape info
shape_data = np.array([['', 'rows', 'cols', 'ch', 'shape'],
['', img_rows, img_cols, ch_no, str(input_shape)]])
shape_result = pd.DataFrame(data=shape_data[1:,1:],
index=shape_data[1:,0],
columns=shape_data[0,1:])
setattr(shape_result, "rm_metadata", {})
shape_result.rm_metadata['rows'] = ("integer",None)
shape_result.rm_metadata['cols'] = ("integer",None)
shape_result.rm_metadata['ch'] = ("integer",None)
shape_result.rm_metadata['shape'] = ("text",None)
# Return results
return df_train, df_test, shape_result
The RM process is very simple - it reads images data in, passes the training and validation data sets to Keras sequence and then checks the model performance. The architecture of the Keras network is slightly different from that in the github example, as we need to include an extra initial step to fold the data back into its original form based on the size and shape info passed into RM. So the model looks like this:
All other network elements are identical to those in the Python code. When the process runs it gives an almost exact performance as the same code in Python, which could be plotted in RM or watched in real-time using Tensorflow, e.g.
Enjoy convolutional neural nets processing images in RapidMiner.
The whole process you can find in the following RMP.
<?xml version="1.0" encoding="UTF-8"?><process version="7.6.001">
<context>
<input/>
<output/>
<macros/>
</context>
<operator activated="true" class="process" compatibility="7.6.001" expanded="true" name="Process">
<process expanded="true">
<operator activated="true" class="python_scripting:execute_python" compatibility="7.4.000" expanded="true" height="124" name="Execute Python" width="90" x="45" y="187">
<parameter key="script" value="from __future__ import print_function import os import numpy as np import pandas as pd import keras from keras.datasets import mnist from keras import backend as K ### Loads and returns MNIST data set in Pandas format def rm_main(): # input image dimensions img_rows, img_cols, ch_no = 28, 28, 1 num_classes = 10 # the data, shuffled and split between train and test sets (x_train, y_train), (x_test, y_test) = mnist.load_data() if K.image_data_format() == 'channels_first': input_shape = (ch_no, img_rows, img_cols) else: input_shape = (img_rows, img_cols, ch_no) x_train = x_train.reshape(x_train.shape[0], img_rows*img_cols*ch_no).astype('float32') x_test = x_test.reshape(x_test.shape[0], img_rows*img_cols*ch_no).astype('float32') x_train /= 255 x_test /= 255 # convert image vectors to data frames df_train = pd.concat([pd.DataFrame(data={'y': y_train}), pd.DataFrame.from_dict(x_train)], axis=1) setattr(df_train, "rm_metadata", {}) df_train.rm_metadata['y'] = ("nominal","label") df_test = pd.concat([pd.DataFrame(data={'y': y_test}), pd.DataFrame.from_dict(x_test)], axis=1) setattr(df_test, "rm_metadata", {}) df_test.rm_metadata['y'] = ("nominal","label") # Prepare shape info shape_data = np.array([['', 'rows', 'cols', 'ch', 'shape'], ['', img_rows, img_cols, ch_no, str(input_shape)]]) shape_result = pd.DataFrame(data=shape_data[1:,1:], index=shape_data[1:,0], columns=shape_data[0,1:]) setattr(shape_result, "rm_metadata", {}) shape_result.rm_metadata['rows'] = ("integer",None) shape_result.rm_metadata['cols'] = ("integer",None) shape_result.rm_metadata['ch'] = ("integer",None) shape_result.rm_metadata['shape'] = ("text",None) # Return results return df_train, df_test, shape_result "/>
</operator>
<operator activated="true" class="extract_macro" compatibility="7.6.001" expanded="true" height="68" name="Extract Macro" width="90" x="246" y="391">
<parameter key="macro" value="img_shape"/>
<parameter key="macro_type" value="data_value"/>
<parameter key="attribute_name" value="shape"/>
<parameter key="example_index" value="1"/>
<list key="additional_macros">
<parameter key="img_rows" value="rows"/>
<parameter key="img_cols" value="cols"/>
<parameter key="img_channels" value="ch"/>
</list>
</operator>
<operator activated="true" class="generate_macro" compatibility="7.6.001" expanded="true" height="82" name="Generate Macro" width="90" x="380" y="391">
<list key="function_descriptions">
<parameter key="img_size" value="eval(%{img_rows})*eval(%{img_cols})*eval(%{img_channels})"/>
</list>
</operator>
<operator activated="true" class="multiply" compatibility="7.6.001" expanded="true" height="103" name="Multiply" width="90" x="246" y="238"/>
<operator activated="true" class="keras:sequential" compatibility="1.0.003" expanded="true" height="166" name="Keras Model" width="90" x="380" y="34">
<parameter key="input shape" value="(%{img_size},)"/>
<parameter key="loss" value="categorical_crossentropy"/>
<parameter key="optimizer" value="Adadelta"/>
<parameter key="learning rate" value="1.0"/>
<parameter key="rho" value="0.95"/>
<parameter key="use metric" value="true"/>
<enumeration key="metric">
<parameter key="metric" value="categorical_accuracy"/>
</enumeration>
<parameter key="epochs" value="12"/>
<parameter key="batch size" value="128"/>
<enumeration key="callbacks">
<parameter key="callbacks" value="TensorBoard(log_dir='/tmp/keras_logs/MNIST_RM', histogram_freq=0, write_graph=True, write_images=True, embeddings_freq=0, embeddings_layer_names=None, embeddings_metadata=None)"/>
</enumeration>
<parameter key="verbose" value="2"/>
<parameter key="shuffle" value="true"/>
<process expanded="true">
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Reshape" width="90" x="45" y="34">
<parameter key="layer_type" value="Reshape"/>
<parameter key="target_shape" value="%{img_shape}"/>
<parameter key="dims" value="1.1"/>
</operator>
<operator activated="true" class="keras:conv_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Conv2D 1" width="90" x="179" y="34">
<parameter key="layer_type" value="Conv2D"/>
<parameter key="filters" value="32"/>
<parameter key="kernel_size_2d" value="3.3"/>
<parameter key="kernel_size_3d" value="1.1.1"/>
<parameter key="strides_2d" value="1.1"/>
<parameter key="strides_3d" value="1.1.1"/>
<parameter key="padding_1d" value="1.1"/>
<parameter key="cropping_1d" value="1.1"/>
<parameter key="size_2d" value="2.2"/>
<parameter key="size_3d" value="2.2.2"/>
<parameter key="dilation_rate_2d" value="1.1"/>
<parameter key="dilation_rate_3d" value="1.1.1"/>
<parameter key="activation_function" value="'relu'"/>
</operator>
<operator activated="true" class="keras:conv_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Conv2D 2" width="90" x="313" y="34">
<parameter key="layer_type" value="Conv2D"/>
<parameter key="filters" value="64"/>
<parameter key="kernel_size_2d" value="3.3"/>
<parameter key="kernel_size_3d" value="1.1.1"/>
<parameter key="strides_2d" value="1.1"/>
<parameter key="strides_3d" value="1.1.1"/>
<parameter key="padding_1d" value="1.1"/>
<parameter key="cropping_1d" value="1.1"/>
<parameter key="size_2d" value="2.2"/>
<parameter key="size_3d" value="2.2.2"/>
<parameter key="dilation_rate_2d" value="1.1"/>
<parameter key="dilation_rate_3d" value="1.1.1"/>
<parameter key="activation_function" value="'relu'"/>
</operator>
<operator activated="true" class="keras:pooling_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Pooling Layer" width="90" x="447" y="34">
<parameter key="layer_type" value="MaxPooling2D"/>
<parameter key="pool_size_2d" value="2.2"/>
<parameter key="pool_size_3d" value="2.2.2"/>
<parameter key="strides_2d" value="2.2"/>
<parameter key="strides_3d" value="2.2.2"/>
</operator>
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Dropout 1" width="90" x="581" y="34">
<parameter key="layer_type" value="Dropout"/>
<parameter key="rate" value="0.25"/>
<parameter key="dims" value="1.1"/>
</operator>
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Flatten" width="90" x="179" y="238">
<parameter key="layer_type" value="Flatten"/>
<parameter key="dims" value="1.1"/>
</operator>
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Dense 1" width="90" x="313" y="238">
<parameter key="no_units" value="128"/>
<parameter key="activation_function" value="'relu'"/>
<parameter key="dims" value="1.1"/>
</operator>
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Dropout 2" width="90" x="447" y="238">
<parameter key="layer_type" value="Dropout"/>
<parameter key="rate" value="0.5"/>
<parameter key="dims" value="1.1"/>
</operator>
<operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Dense Softmax" width="90" x="581" y="238">
<parameter key="no_units" value="10"/>
<parameter key="activation_function" value="'softmax'"/>
<parameter key="dims" value="1.1"/>
</operator>
<connect from_op="Add Reshape" from_port="layers 1" to_op="Add Conv2D 1" to_port="layers"/>
<connect from_op="Add Conv2D 1" from_port="layers 1" to_op="Add Conv2D 2" to_port="layers"/>
<connect from_op="Add Conv2D 2" from_port="layers 1" to_op="Add Pooling Layer" to_port="layers"/>
<connect from_op="Add Pooling Layer" from_port="layers 1" to_op="Add Dropout 1" to_port="layers"/>
<connect from_op="Add Dropout 1" from_port="layers 1" to_op="Add Flatten" to_port="layers"/>
<connect from_op="Add Flatten" from_port="layers 1" to_op="Add Dense 1" to_port="layers"/>
<connect from_op="Add Dense 1" from_port="layers 1" to_op="Add Dropout 2" to_port="layers"/>
<connect from_op="Add Dropout 2" from_port="layers 1" to_op="Add Dense Softmax" to_port="layers"/>
<connect from_op="Add Dense Softmax" from_port="layers 1" to_port="layers 1"/>
<portSpacing port="sink_layers 1" spacing="0"/>
<portSpacing port="sink_layers 2" spacing="0"/>
</process>
</operator>
<operator activated="true" class="generate_id" compatibility="7.6.001" expanded="true" height="82" name="Generate ID" width="90" x="581" y="34"/>
<operator activated="true" class="keras:apply" compatibility="1.0.003" expanded="true" height="82" name="Apply Keras Model" width="90" x="581" y="238">
<parameter key="batch_size" value="8"/>
<parameter key="verbose" value="1"/>
</operator>
<operator activated="true" class="performance_classification" compatibility="7.6.001" expanded="true" height="82" name="Performance" width="90" x="715" y="136">
<parameter key="kappa" value="true"/>
<parameter key="correlation" value="true"/>
<list key="class_weights"/>
</operator>
<connect from_op="Execute Python" from_port="output 1" to_op="Keras Model" to_port="training set"/>
<connect from_op="Execute Python" from_port="output 2" to_op="Multiply" to_port="input"/>
<connect from_op="Execute Python" from_port="output 3" to_op="Extract Macro" to_port="example set"/>
<connect from_op="Extract Macro" from_port="example set" to_op="Generate Macro" to_port="through 1"/>
<connect from_op="Multiply" from_port="output 1" to_op="Keras Model" to_port="validation set"/>
<connect from_op="Multiply" from_port="output 2" to_op="Apply Keras Model" to_port="unlabelled data"/>
<connect from_op="Keras Model" from_port="model" to_op="Apply Keras Model" to_port="model"/>
<connect from_op="Keras Model" from_port="history" to_op="Generate ID" to_port="example set input"/>
<connect from_op="Generate ID" from_port="example set output" to_port="result 1"/>
<connect from_op="Apply Keras Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
<connect from_op="Apply Keras Model" from_port="model" to_port="result 4"/>
<connect from_op="Performance" from_port="performance" to_port="result 2"/>
<connect from_op="Performance" from_port="example set" to_port="result 3"/>
<portSpacing port="source_input 1" spacing="0"/>
<portSpacing port="sink_result 1" spacing="0"/>
<portSpacing port="sink_result 2" spacing="21"/>
<portSpacing port="sink_result 3" spacing="189"/>
<portSpacing port="sink_result 4" spacing="126"/>
<portSpacing port="sink_result 5" spacing="0"/>
</process>
</operator>
</process>
[ADDED LATER] I have been playing with a standard MNIST example and comparing the same model performance against those running on R/R Studio and Python/Anaconda. I have stumbled on an issue while plotting the results using Tensorboard. The common metric used in measuring the classifier is "accuracy" or "acc", which will be then translated by Keras into something more appropriate depending on the output shape, so in MNIST it will be "categorical_accuracy". The RM Keras plugin is trying to be smarter than humans working with Python and R and it eliminates "accuracy" and "acc" as an option for such models. This causes problems with Tensorboard, as then the Python and R generated logs will show as "acc" and "val_acc" panels and the RM logs will be placed in separate panels "categorical_accuracy" and "val_categorical_accuracy". We need to bring the "accuracy" back to the Keras plugin metric options! By the way, the issue can be "fixed" by finding the offending option in XML and changing it there, then the RM Keras logs will rejoin the rest of the world 
Jacob
P.S. Interestingly when running MNIST over 30 epochs, compared with R (orange and dark blue) and Python (brown and light blue), RM (purple) consistently produces best validation loss but its validation accuracy places it in the middle. The Python beats all, R is the loser.
Comments
Varun
https://www.varunmandalapu.com/
Be Safe. Follow precautions and Maintain Social Distancing
For reasons not worth getting into, this thread was one of the few that was not migrated over from the old community platform. This is very unfortunate due to the rich content that you all helped contribute to. Hence I have attempted to cut-and-paste the more salient points into Jesus' original post, giving credit to those of you below it. Please DM if something is not correct or something else should be added. You can actually find the old thread by going here (this will be deleted soon so please don't bother bookmarking it).
Thanks very much!
Scott
I created a model for CNN (CONV 1D) which is shown below. I am using cross-validation for this. But the issue is the error with dense layer. Keras Error when checking target: expected dense_1 to have shape (2,) but got array with shape (1,) But my output classes are two which is the reason I gave dense value as 2. I also attached data file in this. @pschlunder @israel_jimenez @jacobcybulski @M_Martin
<?xml version="1.0" encoding="UTF-8"?><process version="9.0.003"> <context> <input/> <output/> <macros/> </context> <operator activated="true" class="process" compatibility="9.0.003" expanded="true" name="Process"> <process expanded="true"> <operator activated="true" class="retrieve" compatibility="9.0.003" expanded="true" height="68" name="Retrieve New_Rapid" width="90" x="45" y="85"> <parameter key="repository_entry" value="../../../data/New_Rapid"/> </operator> <operator activated="true" class="multiply" compatibility="9.0.003" expanded="true" height="82" name="Multiply" width="90" x="246" y="85"/> <operator activated="true" class="select_attributes" compatibility="9.0.003" expanded="true" height="82" name="Select Attributes" width="90" x="380" y="34"> <parameter key="attributes" value="X1|X2|X3|X4|X5|X6|X7|X8|X9"/> </operator> <operator activated="true" class="series:windowing" compatibility="7.4.000" expanded="true" height="82" name="Windowing" width="90" x="514" y="34"> <parameter key="window_size" value="1"/> <parameter key="create_label" value="true"/> <parameter key="label_attribute" value="Label"/> </operator> <operator activated="true" class="concurrency:cross_validation" compatibility="9.0.003" expanded="true" height="145" name="Cross Validation" width="90" x="715" y="85"> <parameter key="number_of_folds" value="5"/> <parameter key="enable_parallel_execution" value="false"/> <process expanded="true"> <operator activated="true" class="keras:sequential" compatibility="1.0.003" expanded="true" height="166" name="Keras Model" width="90" x="313" y="34"> <parameter key="input shape" value="(1,9)"/> <parameter key="loss" value="binary_crossentropy"/> <parameter key="optimizer" value="Adam"/> <enumeration key="metric"/> <parameter key="batch size" value="1"/> <enumeration key="callbacks"/> <process expanded="true"> <operator activated="true" class="keras:conv_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Convolutional Layer" width="90" x="112" y="34"> <parameter key="filters" value="64"/> <parameter key="kernel_size_2d" value="1.1"/> <parameter key="kernel_size_3d" value="1.1.1"/> <parameter key="strides_2d" value="1.1"/> <parameter key="strides_3d" value="1.1.1"/> <parameter key="padding_1d" value="1.1"/> <parameter key="cropping_1d" value="1.1"/> <parameter key="size_2d" value="2.2"/> <parameter key="size_3d" value="2.2.2"/> <parameter key="dilation_rate_2d" value="1.1"/> <parameter key="dilation_rate_3d" value="1.1.1"/> <parameter key="activation_function" value="'relu'"/> </operator> <operator activated="true" class="keras:pooling_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Pooling Layer" width="90" x="246" y="34"> <parameter key="pool_size_1d" value="1"/> <parameter key="pool_size_2d" value="2.2"/> <parameter key="pool_size_3d" value="2.2.2"/> <parameter key="strides_2d" value="2.2"/> <parameter key="strides_3d" value="2.2.2"/> </operator> <operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Drop Out Layer" width="90" x="380" y="34"> <parameter key="layer_type" value="Dropout"/> <parameter key="dims" value="1.1"/> </operator> <operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Flatten" width="90" x="514" y="34"> <parameter key="layer_type" value="Flatten"/> <parameter key="no_units" value="2"/> <parameter key="dims" value="1.1"/> </operator> <operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Dense" width="90" x="648" y="34"> <parameter key="no_units" value="126"/> <parameter key="activation_function" value="'relu'"/> <parameter key="dims" value="1.1"/> </operator> <operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Core Layer" width="90" x="782" y="187"> <parameter key="layer_type" value="Dropout"/> <parameter key="rate" value="0.5"/> <parameter key="dims" value="1.1"/> </operator> <operator activated="true" class="keras:core_layer" compatibility="1.0.003" expanded="true" height="82" name="Add Core Layer (3)" width="90" x="782" y="34"> <parameter key="no_units" value="2"/> <parameter key="activation_function" value="'softmax'"/> <parameter key="dims" value="1.1"/> </operator> <connect from_op="Add Convolutional Layer" from_port="layers 1" to_op="Add Pooling Layer" to_port="layers"/> <connect from_op="Add Pooling Layer" from_port="layers 1" to_op="Drop Out Layer" to_port="layers"/> <connect from_op="Drop Out Layer" from_port="layers 1" to_op="Flatten" to_port="layers"/> <connect from_op="Flatten" from_port="layers 1" to_op="Dense" to_port="layers"/> <connect from_op="Dense" from_port="layers 1" to_op="Add Core Layer" to_port="layers"/> <connect from_op="Add Core Layer" from_port="layers 1" to_op="Add Core Layer (3)" to_port="layers"/> <connect from_op="Add Core Layer (3)" from_port="layers 1" to_port="layers 1"/> <portSpacing port="sink_layers 1" spacing="0"/> <portSpacing port="sink_layers 2" spacing="0"/> </process> </operator> <connect from_port="training set" to_op="Keras Model" to_port="training set"/> <connect from_op="Keras Model" from_port="model" to_port="model"/> <portSpacing port="source_training set" spacing="0"/> <portSpacing port="sink_model" spacing="0"/> <portSpacing port="sink_through 1" spacing="0"/> </process> <process expanded="true"> <operator activated="true" class="keras:apply" compatibility="1.0.003" expanded="true" height="82" name="Apply Keras Model" width="90" x="45" y="34"/> <operator activated="true" class="performance" compatibility="9.0.003" expanded="true" height="82" name="Performance" width="90" x="179" y="34"> <parameter key="use_example_weights" value="false"/> </operator> <connect from_port="model" to_op="Apply Keras Model" to_port="model"/> <connect from_port="test set" to_op="Apply Keras Model" to_port="unlabelled data"/> <connect from_op="Apply Keras Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/> <connect from_op="Performance" from_port="performance" to_port="performance 1"/> <portSpacing port="source_model" spacing="0"/> <portSpacing port="source_test set" spacing="0"/> <portSpacing port="source_through 1" spacing="0"/> <portSpacing port="sink_test set results" spacing="0"/> <portSpacing port="sink_performance 1" spacing="0"/> <portSpacing port="sink_performance 2" spacing="0"/> </process> </operator> <connect from_op="Retrieve New_Rapid" from_port="output" to_op="Multiply" to_port="input"/> <connect from_op="Multiply" from_port="output 1" to_op="Select Attributes" to_port="example set input"/> <connect from_op="Select Attributes" from_port="example set output" to_op="Windowing" to_port="example set input"/> <connect from_op="Windowing" from_port="example set output" to_op="Cross Validation" to_port="example set"/> <connect from_op="Cross Validation" from_port="performance 1" to_port="result 1"/> <portSpacing port="source_input 1" spacing="0"/> <portSpacing port="sink_result 1" spacing="0"/> <portSpacing port="sink_result 2" spacing="0"/> </process> </operator> </process>Varun
https://www.varunmandalapu.com/
Be Safe. Follow precautions and Maintain Social Distancing