Calling other lambdas

We assume that you've already created your First Serverless project

Prerequisites

Dumptruck is here to help you monitoring what happens with your lambda functions on AWS in real time.

When would you call other lambdas?

The typical Lambda function would take some input, provide some small modular functionality and produce some output. Usually, any complex project would require multiple lambda functions, some of which might depend on each other. Whenever the output of one lambda is required as an input for another lambda function, it is necessary to have multiple options to call one lambda function from another.

Two approaches to the problem: Calling other λs

___	Through SNS Queue	Directly via AWS API (HTTP)
Cost efficient	Yes	No
Immediate response	No	Yes
Can pass limited data to other λ	Yes*	No
Longer execution time of first lambda	No	Yes
Fire and Forget	Yes	Yes

* - If you need to pass a lot of data, you can save it to S3 or any other storage and pass the link to it, although, it requires more hassle.

First and correct approach is is to use SNS queue. It follows the event-driven architecture and is more cost efficient. Once you send and event to the queue your origin λ can forget about it and successfully exit thus saving you execution time and money.

Second approach can be used if the result of the second λ is absolutely required in the first λ. In that case you can directly call second λ from first λ. This approach also allows you to send more data between λs as SNS doesn't allow you to pass much data. The disadvantage of this approach is that you essentially make first λ to wait while the second is executing, essentially doubling the execution time you are charged for. Also, if second λ also calls some other λs it might increase your costs even more. On more negative scenario is when the second λ is not warmed up, you will be charged for the warm up of the second λ in the first λ.

Note that you can also use Fire and Forget in case of the second approach. In that case the first lambda is not gonna wait for the execution of the second lambda.

Detailed instructions on how to specify SNS topic to invoke λ when something is posted there.

Serverless framework supports the creation of SNS topic for you, you just have to have the following yaml for your function:

  multiply-number-sns:
    handler: handler.multiplyNumberSNS
    description: "slspress calling other lambdas using sns example"
    events:
      - sns: 
          topicName: multiply-number-dispatch
          displayName: Multiply Number Dispatch

On first deploy, Serverless will automatically create the topic and make subscribe multiplyNumberSNS function on it. Unfortunately, at the moment of writing the documentation there was no good way of relatively referencing the topic ARN resource from other functions, therefore usually you would take a different approach. You can define the ARN as a variable and pass it all functions that need it as well as to functions that need to listen to the topic. Example yaml:

service: serverless-hello-world
custom:
  multiplyNumberDispatchTopic: 
    Fn::Join:
      - ""
      - - "arn:aws:sns:"
        - Ref: "AWS::Region"
        - ":"
        - Ref: "AWS::AccountId"
        - ":multiply-number-dispatch"
provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  profile: default
  memorySize: 256
  stage: dev
  environment:
    SERVERLESS_SERVICE_NAME: ${self:service}
    SERVERLESS_SERVICE_STAGE: ${opt:stage, self:provider.stage}
  iamRoleStatements:
    - Effect: Allow
      Action:
        - sns:Publish
        - sns:CreateTopic
      Resource: "${self:custom.multiplyNumberDispatchTopic}"

package:
  individually: true

functions:

  post-to-sns:
    handler: handler.handle
    name: "${self:service}-${self:provider.stage}-post-to-sns"
    description: "slspress calling other lambdas using sns example"
    environment:
      MULTIPLY_NUMBER_SNS_TOPIC: "${self:custom.multiplyNumberDispatchTopic}"
    events:
      - http:
          path: post-to-sns
          method: get
          cors: true

  multiply-number-sns:
    handler: handler.multiplyNumberSNS
    description: "slspress calling other lambdas using sns example"
    events:
      - sns: 
          topicName: multiply-number-dispatch
          displayName: Multiply Number Dispatch

In the code, postToSNS function will have access to the topic where to post.

const { create, jsonMiddleware } = require('slspress');
const AWS = require('aws-sdk');

const handler = create();

const sns = new AWS.SNS();

handler.on('handle')
    .middleware(jsonMiddleware)
    .get('/post-to-sns', (req, res) => {

        var params = {
            Message: JSON.stringify({ x: 345, y: 543 }),
            Subject: 'post-to-sns',
            TopicArn: process.env.MULTIPLY_NUMBER_SNS_TOPIC
          };
    
        sns.publish(params, (err, data) => {
            if (err) return res.handleError(err);
            else return res.ok(data);
        });
    });

module.exports = handler.export();

and simplest version of multiplyNumberSNS that will listen to events:

module.exports.multiplyNumberSNS = (event, context) => {
  console.log('EVENT: ', JSON.stringify(event))
  const vars = JSON.parse(event.Records[0].Sns.Message);
  console.log(vars.x * vars.y);
};

You can use our internal tool Dumptruck to monitor both postToSNS and multiplyNumberSNS logs in a fast way.

dumptruck

If you try to go to your postToSNS function API Gateway URL, you should see in terminal window logs that function failed to post to SNS because it doesn't have permissions to SNS. That's correct, λ function should be given permissions to post to SNS queue.

Login to AWS Console -> Lambda -> Functions -> serverless-hello-world-dev-postToSNS -> Configuration. Here you should press Role -> Create Custom Role Name the role lambda-post-to-sns-allow and set it to policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sns:Publish"
      ],
      "Resource": "arn:aws:sns:us-east-1:xxxxxxxxxxxx:multiplyNumberDispatch"
    }
  ]
}

where xxxxxxxxxxxx is you AWS Account ID. This will make sure this λ can do sns:Publish on our newly created SNS Topic. After saving role and λ configuration (make sure you press Save on both), you can retry invoking postToSNS API Gateway URL. Now, you'll see in dumptruck of multiplyNumberSNS that it received data from the queue.

Let's just create a simple JS handler for our both our functions in handler.js

const { create, jsonMiddleware } = require('slspress');
const AWS = require('aws-sdk');

const handler = create();

const lambda = new AWS.Lambda( { region: 'eu-west-1' } );

const fPrepend = [
    process.env.SERVERLESS_SERVICE_NAME,
    process.env.SERVERLESS_SERVICE_STAGE, ''
    ].join('-');

handler.on('handle')
    .middleware(jsonMiddleware)
    .get('/calling-other-lambdas', (req, res) => {

        const params = {
            FunctionName: fPrepend + 'multiply-number', 
            InvocationType: "RequestResponse", 
            LogType: "Tail", 
            Payload: JSON.stringify({x: 5, y: 100}),
          };

        lambda.invoke(params, (err, data) => {
            if (err) return res.handleError(err);
            else return res.ok(data);
        });
    })
    .post('/multiply-number', (req, res) => {
        const vars = req.event.body;
        const response = {
            statusCode: 200,
            headers: {
              'Access-Control-Allow-Origin': '*', // Required for CORS support to work
            },
            body: JSON.stringify({
              result: vars.x * vars.y || 0,
              input: req.event,
            })
        };
        return res.ok(response);
    });

module.exports = handler.export();

lambda.invoke function will do the call from one lambda function to another. Keep in mind that better approach of calling one lambda function from another would be to use SNS queue described previously.

Our serverless.yml looks like this:

service: calling-other-lambdas

provider:
  name: aws
  runtime: nodejs8.10
  region: eu-west-1
  profile: default
  memorySize: 256
  stage: dev
  environment:
    SERVERLESS_SERVICE_NAME: ${self:service}
    SERVERLESS_SERVICE_STAGE: ${opt:stage, self:provider.stage}
  iamRoleStatements:
    - Effect: Allow
      Action:
        - lambda:InvokeFunction
      Resource: "arn:aws:lambda:${opt:region, self:provider.region}:*:function:${self:service}-${opt:stage, self:provider.stage}-multiply-number"

package:
  individually: true

functions:
  calling-other-lambdas:
      handler: handler.handle
      name: "${self:service}-${self:provider.stage}-calling-other-lambdas"
      events:
        - http:
            path: calling-other-lambdas
            method: get
            cors: true

  multiply-number:
      handler: handler.handle
      name: "${self:service}-${self:provider.stage}-multiply-number"
      events:
        - http:
            path: multiply-number
            method: post
            cors: true

Now if you created handler.js and serverless.yml you can deploy function with:

serverless deploy

and just open the API Gateway URL of multiply-number function. You should see the result looking like this:

{"message":"//////helloWorld called multiplyNumber and second function responded with: 500","responseBody":{"result":500,"input":{"x":5,"y":100}}}

Note, that you can also use fire and forget. In that case, you just have to specify

var params = {
  FunctionName: fPrepend + 'multiplyNumber',
  InvocationType: "Event",
  LogType: "Tail",
  Payload: JSON.stringify({x: 5, y: 100}),
};

The callback will succeed when the other lambda was invoked but it'll not wait for it to be finished.