Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Running Deep Learning extension with CUDA 10.2?
jacobcybulski
Member, University Professor Posts: 391 Unicorn
I can see in the new version of the Deep Learning extension the requirement for CUDA 10.0. However the new Tensorflow, which I also use on my system, requires CUDA 10.1+ and runs with the newest one too, which is CUDA 10.2. The release notes for the extension suggest to contact RM for assistance. As it is, the preferences for the GPU/CPU switch are complaining about my CUDA. I imagine I may need to set up a multi-CUDA system on my Ubuntu 18.04? Or is there some easy tweak to run the extension with the newer version of CUDA?
0
Best Answers
-
jczogalla Employee, Member Posts: 144 RM EngineeringHi @jacobcybulski,we currently rely on CUDA 10.0, so a multi-CUDA setup might be a possibility.We are also currently working on the next version, which would rely on 10.2, but the release date is not clear yet.CheersJan5
-
pschlunder Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, RMResearcher, Member Posts: 96 RM ResearchHey @jacobcybulskifind a version build against CUDA 10.2 and cuDNN 7.6 here:(link is only valid until May 14th, if you need the extension and the link expired please point it out and we'll update).You can place the downloaded jar under your .RapidMiner/extensions folder. Once we'll release 0.9.4 it should be automatically used since it's a newer version.Another option would be to also install 10.0 and set the CUDA environment variable to the 10.0 version for the environment you're using RapidMiner in.Hope this helps,Philipp
6 -
jacobcybulski Member, University Professor Posts: 391 Unicorn@jczogalla I have got a workaround! When you export the settings for LD_LIBRARY_PATH and a PATH to /usr/local/cuda within Rapid-Miner.sh, miraculously it is then possible to switch from CPU to GPU and Deep Learning operators actually execute on a GPU!I have tried to set these environment variables in /etc/profile and /etc/environment but it did not matter. Perhaps there is some global setting for JVM?1
Answers
Dortmund, Germany
I think this might now be a problem with how your path is set up. You are correct to assume that the GPU backend is only extracted/installed when it finds the correct CUDA version. Make sure that your path contains the CUDA 10.2 location and that it is before any other CUDA references in the path. I'm not sure about other environment variables in Linux that Java might pick up about libraries...
Cheers
Jan
Thu Apr 23 16:12:41 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 415.27 Driver Version: 415.27 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 108... Off | 00000000:17:00.0 Off | N/A |
| 0% 33C P8 10W / 280W | 2MiB / 11178MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX 108... Off | 00000000:65:00.0 On | N/A |
| 0% 61C P0 66W / 280W | 248MiB / 11175MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 1 1777 G /usr/lib/xorg/Xorg 167MiB |
| 1 3121 G /usr/bin/gnome-shell 79MiB |
+-----------------------------------------------------------------------------+
jacob@goblin-galore:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
jacob@goblin-galore:~$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
#define CUDNN_MAJOR 7
#define CUDNN_MINOR 4
#define CUDNN_PATCHLEVEL 2
--
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)
#include "driver_types.h"
Regarding the error message, in the 0.9.4 snapshot version, we simply forgot to adjust the message to show 10.2 instead of 10.0...
Can you provide your Studio log file? You can share it via PM if you like. Not sure how well the logging is, but maybe we can see something there.
Other than that I am not sure why it would not work, since this version did work for other people before, but that might have been on Windows machines.
That's great to hear! I think the problem here is the special handling on Linux systems with the LD library path. There might be global JVM settings, but that might hurt other java programs. And yes, you would have to touch the RapidMiner.sh file because there is no other way to put that in there.
We'll make a note and think about a possibility to provide the cuda path as a setting, similar to what we do with Python.