The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.

upload directory or Multiple Files to S3

abhi97660abhi97660 Member Posts: 11 Contributor II
edited January 2020 in Help


I am using (RM Server and RM Studio) 8.2

I am developing an extension in which I am downloading multiple files (eg: 10 files) sequentially for processing and want to upload processed files to S3 

I want to upload whole directory containing these files to S3.

Do we have any operator to upload Directory to S3...? I tried Write Amazon S3 it takes individual file at once but we can  use loop files operator to do that.

If I want to save upload time, I would like to upload the file as soon as it is downloaded and processed rather than upload all at once.

I tried ouputPort.deliver((IOObject) FileObject) method inside the loop but it is waiting for whole process to get completed.

Any suggestions on how to achieve this simultaneously  in rapidminer where operator is downloading rest of the files and the already downloaded files can be uploaded...?


Best Answer

  • Options
    Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn
    Solution Accepted

    @abhi97660 I've never done this before but one idea is to connect the RM SFTP operaters to an S3 bucket. You should be able to load up the entire directory then. 

Sign In or Register to comment.