How to add a custom connection type in rapidminer

KanikaAg15

Hi, I am trying to access the data that is present in a shared place over the web URL which is not listed among the default connection types available in rapidminer. Please help in suggesting how to access the same. 

Mo_Abdolrahim

Hi Kanika,
I checked the RM Studio v10 and did not see a connection for downloading a shared file (no credentials) over the Internet. Hopefully I am right, because I am new to RM.

I noticed the RM provides two providers, Execute Script and Execute Program.  You can write a program with your favorite script, that downloads a web URL file to a disk file, compile, create an executable and use it in Execute Program operator.
Or write a Java or Groovy script in the Execute Script operator.  This Operator executes the Java or Groovy script in RM process. The script downloads a web URL to a disk file.  Then you can use the Read CSV operator (i am summing its a csv file) to read the data and perform further analytics in RM.

The following Java script downloads classfeb.csv file from the local server and saves it to c:\mo\classfeb.csv file.
You should modify the values for the file_url and disk_file variables.

import java.io.FileOutputStream;

import java.io.IOException;

import java.net.URL;

import java.nio.channels.Channels;

import java.nio.channels.ReadableByteChannel;

import java.io.File;

public class Main{

   public static void main(String[] args){

     try{

        String file_url = "http://localhost/classfeb.csv";  // URL File to be downloaded to a diskfile

        String disk_file = "c:\\mo\\classfeb.csv";  // file location on disk

    URL link = new URL(file_url);

        ReadableByteChannel channel = Channels.newChannel(link.openStream());

    FileOutputStream fileoutstream = new FileOutputStream(new File(disk_file));

    fileoutstream.getChannel().transferFrom(channel, 0, Long.MAX_VALUE);

    fileoutstream.close();

    channel.close();

     } catch(IOException ex){

        ex.printStackTrace();

     }}}

 

Regards
Mo

KanikaAg15

Thanks for sharing the script. As the portal is private and has credentials to access, the file that is  downloading is getting corrupted. Is there any amendment that can be done in the existing script, also if some reference python script you can help with. 

Marco_Boeck

Hi,

You can install the "Web Mining Extension" from the Marketplace. It contains operators like "GET (REST)" which you can use to download files.

Regards,
Marco

KanikaAg15

Hi @Marco_Boeck & @Mo_Abdolrahim
 thanks for your suggestion. I have somehow managed to get the api link and it's reaching the portal but it's an encoded xlsx file I am not able to locate any decoder operator. Any suggestions will be highly appreciated.