upload directory or Multiple Files to S3
Contributor IIHi,
I am using (RM Server and RM Studio) 8.2
I am developing an extension in which I am downloading multiple files (eg: 10 files) sequentially for processing and want to upload processed files to S3
I want to upload whole directory containing these files to S3.
Do we have any operator to upload Directory to S3...? I tried Write Amazon S3 it takes individual file at once but we can use loop files operator to do that.
If I want to save upload time, I would like to upload the file as soon as it is downloaded and processed rather than upload all at once.
I tried ouputPort.deliver((IOObject) FileObject) method inside the loop but it is waiting for whole process to get completed.
Any suggestions on how to achieve this simultaneously in rapidminer where operator is downloading rest of the files and the already downloaded files can be uploaded...?
Best Answer
-
Options
Thomas_Ott
RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 
Unicorn
@abhi97660 I've never done this before but one idea is to connect the RM SFTP operaters to an S3 bucket. You should be able to load up the entire directory then.
0
