Listening to DynamoDB events

alt text

You can execute functions off of the back of data changes in a DynamoDB table.

You will require the following AWS permissions:

  • dynamodb:CreateTable
  • lambda:CreateEventSourceMapping

 

To start, let's create a Serverless music collection table ServerlessTestMusicCollection on DynamoDB There are two ways to do this:

  1. Using AWSCLI via the command line.
aws dynamodb create-table --table-name ServerlessTestMusicCollection --attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES

The AWS API should respond with ARN for this table similar to the below:

{
    "TableDescription": {
        "TableArn": "arn:aws:dynamodb:us-east-1:xxxxxxxxxxx:table/ServerlessTestMusicCollection",
        ...
    }
    ...
    "LatestStreamArn": "arn:aws:dynamodb:us-east-1:xxxxxxxxxxx:table/ServerlessTestMusicCollection/stream/2017-07-21T11:19:36.118"
}
Now you can use `LatestStreamArn` to construct an api. 

 

  1. Using the serverless.yml file by adding the following function to your serverless.yml:
service: dynamo-db-event-handler

provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  profile: default
  memorySize: 256 # optional, in MB, default is 1024
  stage: dev
  environment:
      DYNAMODB_TABLE:  ${self:service}-${opt:stage, self:provider.stage}-ServerlessTestMusicCollection
  iamRoleStatements:
    - Effect: Allow
      Action:
        - dynamodb:*
      Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"

functions:
  dynamoDBEvent:
    handler: handler.dynamoDBEvent
    events:
      - stream:
          type: dynamodb
          arn:
            Fn::GetAtt:
              - DynamoDBTable
              - StreamArn

resources:
  Resources:
    DynamoDBTable:
      Type: 'AWS::DynamoDB::Table'
      DeletionPolicy: Delete
      Properties:
        AttributeDefinitions:
          -
            AttributeName: id
            AttributeType: S
        KeySchema:
          -
            AttributeName: id
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1
        StreamSpecification:
          StreamViewType: NEW_IMAGE
        TableName: ${self:provider.environment.DYNAMODB_TABLE}

And a corresponding export in your handler.js:

module.exports.dynamoDBEvent = (event, context, callback) => {
  console.log('EVENT', JSON.stringify(event)),
  callback(null);
}

 

Deploy your function:

  serverless deploy

Once deployed, if you add anything to your DynamoDB Table, the Lambda will be notified. Add something to your table using the following command:

  aws dynamodb put-item --table-name dynamo-db-event-handler-dev-ServerlessTestMusicCollection --item '{ "id": {"S": "1" }, "Artist": { "S": "Tailor Swift"}, "SongTitle": { "S": "Shake It Off"}, "AlbumTitle": { "S": "1989"}}'

Check the logs in CloudWatch or using Dumptruck for the log statement from the lambda.

It should look similar to the below:

{
    "Records": [
        {
            "eventID": "0d2fa79c11e010c6c7d80d2b087d363a",
            "eventName": "INSERT",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "eu-west-1",
            "dynamodb": {
                "ApproximateCreationDateTime": 1528193160,
                "Keys": {
                    "id": {
                        "S": "1"
                    }
                },
                "NewImage": {
                    "Artist": {
                        "S": "Tailor Swift"
                    },
                    "SongTitle": {
                        "S": "Shake It Off"
                    },
                    "AlbumTitle": {
                        "S": "1989"
                    },
                    "id": {
                        "S": "1"
                    }
                },
                "SequenceNumber": "100000000044902851105",
                "SizeBytes": 59,
                "StreamViewType": "NEW_IMAGE"
            },
            "eventSourceARN": "arn:aws:dynamodb:eu-west-1:189075651281:table/dynamo-db-event-handler-dev-ServerlessTestMusicCollection/stream/2018-06-05T10:00:55.153"
        }
    ]
}

To delete the row:

    aws dynamodb delete-item --table-name dynamo-db-event-handler-dev-ServerlessTestMusicCollection --key '{ "id": {"S": "1" } }'

The response will look like something like this:

{
    "Records": [
        {
            "eventID": "fd107cb379f5c97898e7b03ed0d4c8f3",
            "eventName": "REMOVE",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "eu-west-1",
            "dynamodb": {
                "ApproximateCreationDateTime": 1528193460,
                "Keys": {
                    "id": {
                        "S": "1"
                    }
                },
                "SequenceNumber": "400000000044903307939",
                "SizeBytes": 3,
                "StreamViewType": "NEW_IMAGE"
            },
            "eventSourceARN": "arn:aws:dynamodb:eu-west-1:189075651281:table/dynamo-db-event-handler-dev-ServerlessTestMusicCollection/stream/2018-06-05T10:00:55.153"
        }
    ]
}

The same method described here can be used to process Kinesis events as well.