Options

How can I configure my S3 connection to read? (I don't have access to whole bucket, just to the fol

GuiGui Member Posts: 10 Contributor II
in Help
 Hey Everyone,

as I tried to ask in Question space above...

I need to create a connection between RM and S3. I have done it before, but I have had full access to the bucket. 

Now, I need to read datasets from a folder inside a bucket, but I don't have full visibility/permission of it. So, How can I indicate to RM to connect to this specific folder inside bucket? Is there any way to do it?

Best Answer

  • Options
    kaymankayman Member Posts: 662 Unicorn
    Solution Accepted
    With the loop Amazon S3 operator you can simply add the full path to the folder parameter field (/mybucket/my_folder/my_subfolder) . This will do whatever you want with the files inside of the S3 bucket folder.

    But the server/machine running the process does need to have access to this folder, otherwise it will not work of course.

Answers

  • Options
    jwpfaujwpfau Employee, Member Posts: 277 RM Engineering
    Hi,

    thanks for your feedback, there is already an existing feature request for this, if you know the path you can also use it in the other S3 operators like Read or Write, just the file browser and Connection check won't work.

    Greetings,
    Jonas
  • Options
    GuiGui Member Posts: 10 Contributor II
    Hi there,

    First of all, I'd like also to thank you for helping me

    @jwpfau Jonas, I have already tried these S3 operators and I have had issues (In this specific case: without visibility of whole bucket) 

    The solution @kayman sent helped to solve my problem on local execution. Now, I am going to check/work on accesses, because I couldn't read the files successfully using "run on AIHUB".




Sign In or Register to comment.