Java error with Image Handling and Deep Learning extensions

yho67

Hello everyone!

I reach to you because I get an error with java when trying to to a DeepLearning process

1. Overview of the problem

I wanted to breach the topic of image handling, so I went onto the marketplace and got the three extensions Image Handling 0.2.001, Deep Learning 1.2.001, and ND4J Back End 1.2.000

However, after reading the image correctly, I get an error when reaching the Deep Learning (tensor) module. When running for the first time I get:

Exception: java.lang.UnsatisfiedLinkError
Message: no jniopencv_core in java.library.path

When re-running any time after the first time, I get :

Exception: java.lang.NoClassDefFoundError
Message: Could not initialize class org.bytedeco.javacv.OpenCVFrameConverter$ToMat

I am working with the image set MNIST (sorry can't post the link on my message, but the standard image dataset) 
I am on Windows 10, using RapidMiner9.10.

2. More details on my process and the error

Here is the XML for my very simple process

<?xml version="1.0" encoding="UTF-8"?><process version="9.10.001">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="9.10.001" expanded="true" name="Process">
    <parameter key="logverbosity" value="init"/>
    <parameter key="random_seed" value="2001"/>
    <parameter key="send_mail" value="never"/>
    <parameter key="notification_email" value=""/>
    <parameter key="process_duration_for_mail" value="30"/>
    <parameter key="encoding" value="SYSTEM"/>
    <process expanded="true">
      <operator activated="true" class="image_handling:read_image_meta_data" compatibility="0.2.001" expanded="true" height="68" name="Read Image Meta Data" width="90" x="45" y="85">
        <parameter key="directory" value="C:/Users/yho67/Desktop/data_science/samy_lesson/MNIST/trainingSet"/>
        <parameter key="use_label" value="true"/>
      </operator>
      <operator activated="true" class="image_handling:image_pre_processor" compatibility="0.2.001" expanded="true" height="103" name="Pre-Process Images" width="90" x="179" y="85">
        <parameter key="path" value="Path"/>
        <parameter key="use_label" value="false"/>
        <process expanded="true">
          <connect from_port="transform" to_port="transform"/>
          <portSpacing port="source_transform" spacing="0"/>
          <portSpacing port="sink_transform" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="deeplearning:dl4j_tensor_sequential_neural_network" compatibility="1.2.001" expanded="true" height="145" name="Deep Learning (Tensor)" width="90" x="313" y="85">
        <parameter key="epochs" value="10"/>
        <parameter key="use_miniBatch" value="false"/>
        <parameter key="batch_size" value="32"/>
        <parameter key="log_each_epoch" value="true"/>
        <parameter key="epochs_per_log" value="10"/>
        <parameter key="optimization_method" value="Stochastic Gradient Descent"/>
        <parameter key="backpropagation" value="Standard"/>
        <parameter key="backpropagation_length" value="50"/>
        <parameter key="use_early_stopping" value="false"/>
        <parameter key="condition_strategy" value="score improvement"/>
        <parameter key="patience" value="5"/>
        <parameter key="minimal_score_improvement" value="0.0"/>
        <parameter key="best_epoch_score" value="0.01"/>
        <parameter key="max_iteration_score" value="3.0"/>
        <parameter key="max_iteration_time" value="10"/>
        <parameter key="updater" value="Adam"/>
        <parameter key="learning_rate" value="0.01"/>
        <parameter key="momentum" value="0.9"/>
        <parameter key="rho" value="0.95"/>
        <parameter key="epsilon" value="1.0E-6"/>
        <parameter key="beta1" value="0.9"/>
        <parameter key="beta2" value="0.999"/>
        <parameter key="RMSdecay" value="0.95"/>
        <parameter key="weight_initialization" value="Normal"/>
        <parameter key="bias_initialization" value="0.0"/>
        <parameter key="use_regularization" value="false"/>
        <parameter key="l1_strength" value="0.1"/>
        <parameter key="l2_strength" value="0.1"/>
        <parameter key="cudnn_algo_mode" value="Prefer fastest"/>
        <parameter key="infer_input_shape" value="true"/>
        <parameter key="network_type" value="Simple Neural Network"/>
        <parameter key="use_local_random_seed" value="false"/>
        <parameter key="local_random_seed" value="1992"/>
        <process expanded="true">
          <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Fully-Connected Layer" width="90" x="112" y="34">
            <parameter key="neurons" value="100"/>
            <parameter key="activation_function" value="ReLU (Rectified Linear Unit)"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <operator activated="true" class="deeplearning:dl4j_dense_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Fully-Connected Layer (2)" width="90" x="313" y="34">
            <parameter key="neurons" value="100"/>
            <parameter key="activation_function" value="ReLU (Rectified Linear Unit)"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <operator activated="true" class="deeplearning:dl4j_output_layer" compatibility="1.2.001" expanded="true" height="68" name="Add Output Layer" width="90" x="514" y="34">
            <parameter key="output_type" value="FullyConnected"/>
            <parameter key="loss_function" value="Multiclass Cross Entropy (Classification)"/>
            <parameter key="neurons" value="1"/>
            <parameter key="activation_function" value="Softmax"/>
            <parameter key="use_dropout" value="false"/>
            <parameter key="dropout_rate" value="0.25"/>
            <parameter key="overwrite_networks_weight_initialization" value="false"/>
            <parameter key="weight_initialization" value="Normal"/>
            <parameter key="overwrite_networks_bias_initialization" value="false"/>
            <parameter key="bias_initialization" value="0.0"/>
          </operator>
          <connect from_port="in layerArchitecture" to_op="Add Fully-Connected Layer" to_port="layerArchitecture"/>
          <connect from_op="Add Fully-Connected Layer" from_port="layerArchitecture" to_op="Add Fully-Connected Layer (2)" to_port="layerArchitecture"/>
          <connect from_op="Add Fully-Connected Layer (2)" from_port="layerArchitecture" to_op="Add Output Layer" to_port="layerArchitecture"/>
          <connect from_op="Add Output Layer" from_port="layerArchitecture" to_port="out layerArchitecture"/>
          <portSpacing port="source_in layerArchitecture" spacing="0"/>
          <portSpacing port="sink_out layerArchitecture" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="image_handling:read_image_meta_data" compatibility="0.2.001" expanded="true" height="68" name="Read Image Meta Data (2)" width="90" x="45" y="289">
        <parameter key="directory" value="C:/Users/yho67/Desktop/data_science/samy_lesson/MNIST/testSet"/>
        <parameter key="use_label" value="true"/>
      </operator>
      <operator activated="true" class="image_handling:image_pre_processor" compatibility="0.2.001" expanded="true" height="103" name="Pre-Process Images (2)" width="90" x="179" y="289">
        <parameter key="path" value="Path"/>
        <parameter key="use_label" value="false"/>
        <process expanded="true">
          <connect from_port="transform" to_port="transform"/>
          <portSpacing port="source_transform" spacing="0"/>
          <portSpacing port="sink_transform" spacing="0"/>
        </process>
      </operator>
      <operator activated="true" class="nd4j:apply_model_generic" compatibility="1.2.000" expanded="true" height="82" name="Apply Model (Generic)" width="90" x="380" y="340"/>
      <connect from_op="Read Image Meta Data" from_port="output" to_op="Pre-Process Images" to_port="example set"/>
      <connect from_op="Pre-Process Images" from_port="tensor" to_op="Deep Learning (Tensor)" to_port="training set"/>
      <connect from_op="Deep Learning (Tensor)" from_port="model" to_op="Apply Model (Generic)" to_port="model"/>
      <connect from_op="Read Image Meta Data (2)" from_port="output" to_op="Pre-Process Images (2)" to_port="example set"/>
      <connect from_op="Pre-Process Images (2)" from_port="tensor" to_op="Apply Model (Generic)" to_port="unlabelled data"/>
      <connect from_op="Apply Model (Generic)" from_port="labelled data" to_port="result 2"/>
      <connect from_op="Apply Model (Generic)" from_port="model" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
      <portSpacing port="sink_result 3" spacing="0"/>
    </process>
  </operator>
</process>

As for the error code, the first time I run the process I get :

Exception: java.lang.UnsatisfiedLinkError

Message: no jniopencv_core in java.library.path

Stack trace:

  java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)

  java.lang.Runtime.loadLibrary0(Runtime.java:871)

  java.lang.System.loadLibrary(System.java:1124)

  org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1718)

  org.bytedeco.javacpp.Loader.load(Loader.java:1328)

  org.bytedeco.javacpp.Loader.load(Loader.java:1132)

  org.bytedeco.opencv.global.opencv_core.<clinit>(opencv_core.java:16)

  java.lang.Class.forName0(Native Method)

  java.lang.Class.forName(Class.java:348)

  org.bytedeco.javacpp.Loader.load(Loader.java:1200)

  org.bytedeco.javacpp.Loader.load(Loader.java:1148)

  org.bytedeco.javacv.OpenCVFrameConverter.<clinit>(OpenCVFrameConverter.java:43)

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)

  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)

  java.security.AccessController.doPrivileged(Native Method)

  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)

  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.Process.executeRoot(Process.java:1464)

  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)

  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)

  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)

  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)

  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)

  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

Cause

Exception: java.lang.UnsatisfiedLinkError

Message: C:\Users\yho67\.javacpp\cache\dl-windows-libs-1.2.1-all.jar\org\bytedeco\opencv\windows-x86_64\jniopencv_core.dll: Can't find dependent libraries

Stack trace:

  java.lang.ClassLoader$NativeLibrary.load(Native Method)

  java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)

  java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)

  java.lang.Runtime.load0(Runtime.java:810)

  java.lang.System.load(System.java:1088)

  org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1668)

  org.bytedeco.javacpp.Loader.load(Loader.java:1328)

  org.bytedeco.javacpp.Loader.load(Loader.java:1132)

  org.bytedeco.opencv.global.opencv_core.<clinit>(opencv_core.java:16)

  java.lang.Class.forName0(Native Method)

  java.lang.Class.forName(Class.java:348)

  org.bytedeco.javacpp.Loader.load(Loader.java:1200)

  org.bytedeco.javacpp.Loader.load(Loader.java:1148)

  org.bytedeco.javacv.OpenCVFrameConverter.<clinit>(OpenCVFrameConverter.java:43)

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)

  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)

  java.security.AccessController.doPrivileged(Native Method)

  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)

  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.Process.executeRoot(Process.java:1464)

  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)

  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)

  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)

  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)

  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)

  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

 
And any run after the first, I get this error message :

Exception: java.lang.NoClassDefFoundError

Message: Could not initialize class org.bytedeco.javacv.OpenCVFrameConverter$ToMat

Stack trace:

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:60)

  org.datavec.image.loader.NativeImageLoader.<init>(NativeImageLoader.java:117)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:131)

  org.datavec.image.recordreader.BaseImageRecordReader.initialize(BaseImageRecordReader.java:206)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.initialize(SerializableImageRecordReaderProxy.java:213)

  com.rapidminer.extension.image_handling.tool.SerializableImageRecordReaderProxy.<init>(SerializableImageRecordReaderProxy.java:83)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.doApply(ImagePreProcessingModel.java:147)

  com.rapidminer.extension.image_handling.ioobject.ImagePreProcessingModel.apply(ImagePreProcessingModel.java:111)

  com.rapidminer.extension.image_handling.operator.ImagePreProcessor.doWork(ImagePreProcessor.java:134)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:805)

  com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:800)

  java.security.AccessController.doPrivileged(Native Method)

  com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:800)

  com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:423)

  com.rapidminer.operator.Operator.execute(Operator.java:1023)

  com.rapidminer.Process.executeRoot(Process.java:1464)

  com.rapidminer.Process.lambda$executeRootInPool$5(Process.java:1443)

  com.rapidminer.studio.concurrency.internal.AbstractConcurrencyContext$AdaptedCallable.exec(AbstractConcurrencyContext.java:362)

  java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)

  java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)

  java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)

  java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

Sorry for the long message, I didn't know exactly what part would be useful, so I included a lot of informations in the end.
If anybody could help me, that would be amazing !

Cheers,
Victor S.

yho67

Hi David,

So, after your explanation, we tried different things, and managed to find the problem ! It was resolved by installing Visual C++.

My apologies to have bothered you while it was actually something missing from my side...  

If anyone else ever has this trouble, I hope this solution will help !

Cheers,
Victor S.

David_A

Hi Victor,

thanks for reaching out and the very detailed error description. That's always helpful.

We'll have a look at the issue.

Best,
David

yho67

Hi David,

Thanks for you help ! In the meantime, I tried reinstallingRapidMiner, but well, it didn't change anything ^^"

Cheers,
Victor S.

Mate

Hey Victor,
could you please attach your logs ?
./RapidMiner/rapidminer-studio.log

(in the future, please do use the "Attach file" feature ) 

Cheers,
Mate

David_A

Thanks for the update.

I've tried to reproduce the error, but failed so far.

Do you try to train the model using a GPU (under Settings/Preferences -> Backend). If so, does it work, if you run the process with CPU as backend.

Can you execute other DL processes? Perhaps one of the tutorial processes?

Best,

David

yho67

Hey Mate, David,

Thank you for your time !

I attached my log when trying to launch the process 3 times in a Row. Sorry, I didn't see this option on my first message, my mistake ...

Another information I've come to : I'm executing this process for a lesson on Machine Learning, and after asking around my fellow students, knowing we did the same things at the same time give or take a week (installing RapidMiner at the same time, downloading extension at approximately the same time etc.), some had no trouble and a minority had exactly the same error messages than I had. I couldn't figure out the common link though. (So yeah, not too useful, sorry ...)

@David, I didn't touch the settings, so they are still on default option for the Backend, which after checking is CPU-OpenBLAS. I actually don't have a GPU on my Laptop.

For other process:
- the Tutorial Process: "MNIST classification gives me the exact same errors".
- the Tutorial Process : "Creating a Deep Learning Model on multivariate time series data" worked fine actually (I just tried after reading your message, sorry for not giving you this information sooner!)

If you need anything else from me that could be relevant, please let me know. Once again, thanks a lot for the time you're taking for that !

Cheers,
Victor S.

David_A

Hi Victor,

thanks again for the detailed feedback.

Unfortunately it looks, like everything from our side works as intended and the error is somehow related to your (and your fellow students) Windows installation.

You might need to search how to fix the opencv error regarding the missing jniopencv_core.dll.

Best,
David