Connecting to S3

Here’s a quick example of how to connect to AWS S3 using a JetML workflow.

Workflow Setup

Add the boto3 package to enable Python to connect to S3 and other AWS services.
Add your AWS credentials as environment variables to keep them separate than your code.
Run the workflow.
Copy over the example code below into a notebook on the running workflow.

Example Notebook

import boto3

def getS3(bucket,key):
    s3 = boto3.resource('s3')
    obj = s3.Object(bucket,key)
    return obj.get()['Body'].read().decode('utf-8')

def saveS3(bucket,key,body):
    session = boto3.Session()
    s3_client = session.client('s3')

    try:
        s3_client.put_object(Body=body,Bucket = bucket, Key=key)
    except Exception as e:
        return False

saveS3('bucket','file_name.txt','Example File contents')

getS3('bucket','file_name.txt')

Connecting to S3

Workflow Setup

Example Notebook

converted by JetML