Listening to S3 events

alt text

Lambda functions can listen to S3 events e.g. when something is added or deleted from a bucket.

You will require the following AWS permissions:

  • s3:CreateBucket
  • lambda:CreateEventSourceMapping

There are two ways to create the s3 bucket:

  • Manually.
aws s3api create-bucket --bucket my-bucket
  • Using the serverless.yml file by adding the following function to your serverless.yml:
service: s3-event-handler

provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  profile: default
  memorySize: 256 # optional, in MB, default is 1024
  stage: dev
  environment:
    BUCKET_NAME:  ${self:service}-${opt:stage, self:provider.stage}-s3bucket
  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:*
      Resource: "arn:aws:s3:::${self:provider.environment.BUCKET_NAME}*"

functions:
  fileAdded:
    handler: handler.fileAdded
    events:
      - s3: ${self:provider.environment.BUCKET_NAME}

And a corresponding export in your handler.js:

module.exports.fileAdded = (event, context, callback) => {
  console.log('EVENT', JSON.stringify(event)),
  callback(null);
}

Deploy your function.

  serverless deploy

Now if you , you can go to AWS Console S3 and upload some file to our s3bucket. Your lambda function will get the following event after the file has been uploaded:

{
  "Records": [
    {
      "eventVersion": "2.0",
      "eventSource": "aws:s3",
      "awsRegion": "us-east-1",
      "eventTime": "2017-07-21T10:39:24.388Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "AWS:XXXXXXXXXXXXXXXX"
      },
      "requestParameters": {
        "sourceIPAddress": "xxx.xxx.xxx.xxx"
      },
      "responseElements": {
        "x-amz-request-id": "55AXXXXXXXXX0E31",
        "x-amz-id-2": "PsFUX/J79BSyvY9iYCRi6mOIfQKvhwhDo9X1/s/Dn1HlC/2echsarVantmB7+o/HMNvWmUfTxSc="
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
        "bucket": {
          "name": "s3bucket",
          "ownerIdentity": {
            "principalId": "XXXXXXXXXXXX"
          },
          "arn": "arn:aws:s3:::serverless-file-bucket"
        },
        "object": {
          "key": "README.me",
          "size": 240,
          "eTag": "xxxxxxxxxxxxxxxx",
          "sequencer": "23XXX4324XX324XXXXX"
        }
      }
    }
  ]
}

Records[i].s3.object will contains original filenames and sizes of uploaded files.

You can also set a filter for the events to filter, for example filtering new JPG images in uploads folder.

functions:
  fileAdded:
    handler: handler.fileAdded
    events:
      - s3:
          bucket: serverless-file-bucket
          event: s3:ObjectCreated:*
          rules:
            - prefix: uploads/
            - suffix: .jpg