How to add a custom connection type in rapidminer

KanikaAg15KanikaAg15 Member Posts: 2 Newbie
Hi, I am trying to access the data that is present in a shared place over the web URL which is not listed among the default connection types available in rapidminer. Please help in suggesting how to access the same. 
Tagged:

Best Answer

  • Mo_AbdolrahimMo_Abdolrahim Member Posts: 5 Contributor II
    Solution Accepted
    Hi Kanika,
    I checked the RM Studio v10 and did not see a connection for downloading a shared file (no credentials) over the Internet. Hopefully I am right, because I am new to RM.

    I noticed the RM provides two providers, Execute Script and Execute Program.  You can write a program with your favorite script, that downloads a web URL file to a disk file, compile, create an executable and use it in Execute Program operator.
    Or write a Java or Groovy script in the Execute Script operator.  This Operator executes the Java or Groovy script in RM process. The script downloads a web URL to a disk file.  Then you can use the Read CSV operator (i am summing its a csv file) to read the data and perform further analytics in RM.

    The following Java script downloads classfeb.csv file from the local server and saves it to c:\mo\classfeb.csv file.
    You should modify the values for the file_url and disk_file variables.

    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.net.URL;
    import java.nio.channels.Channels;
    import java.nio.channels.ReadableByteChannel;
    import java.io.File;

    public class Main{
       public static void main(String[] args){
         try{
            String file_url = "http://localhost/classfeb.csv";  // URL File to be downloaded to a diskfile
            String disk_file = "c:\\mo\\classfeb.csv";  // file location on disk
        URL link = new URL(file_url);
            ReadableByteChannel channel = Channels.newChannel(link.openStream());
        FileOutputStream fileoutstream = new FileOutputStream(new File(disk_file));
        fileoutstream.getChannel().transferFrom(channel, 0, Long.MAX_VALUE);
        fileoutstream.close();
        channel.close();
         } catch(IOException ex){
            ex.printStackTrace();
         }}}

     

    Regards
    Mo
    KanikaAg15

Answers

  • KanikaAg15KanikaAg15 Member Posts: 2 Newbie
    Thanks for sharing the script. As the portal is private and has credentials to access, the file that is  downloading is getting corrupted. Is there any amendment that can be done in the existing script, also if some reference python script you can help with. 
  • Marco_BoeckMarco_Boeck Team Lead Software Engineering Administrator, Moderator, Employee, Member, University Professor Posts: 1,976 RM Engineering
    Hi,

    You can install the "Web Mining Extension" from the Marketplace. It contains operators like "GET (REST)" which you can use to download files.

    Regards,
    Marco
Sign In or Register to comment.