Serverless Framework: Lambdas Invoking Lambdas

This article is adapted from my online course .

There are times when you may want one Lambda in your project to invoke another Lambda. Although it is wise to keep your code modular and decoupled, thus not tying two Lambdas together as a functional unit, there are exceptions.

In the past, I've made good use of the 'lambdas-invoking-lambdas' pattern by setting up one Lambda function as a cron task that launches multiple instances of another Lambda function that does some work.

This use-case could easily be implemented as one cron Lambda that fires off a normal (i.e. non-Lambda) function multiple times, but this means the function performing the action can't be consumed by other clients. Keeping the cron Lambda separate from the action Lambda means other clients can call/invoke the action Lambda, too.

Web scraping comes to mind here.

Function #1 (Scraper) is a web scraper that scrapes data from a website, the url of which is passed to it by function #2. The results of the scrape job are then saved to a database.

Function #2 (Cron) makes a database call that results in a list of website urls to scrape, and it makes this call at a set interval (e.g. on the first of every month). Function #2 then iterates over the array of urls and invokes the second (scraper function #1) Lambda function with a url as an argument.

This pattern as two advantages:

1. If your working in an async environment (Node by default, and Python using an async library), multiple scrapers can work in parallel rather than launching them sequentially.
2. If you need an on-demand scrape job, your scraper Lambda can be invoked ad hoc.

To implement this pattern using the Serverless framework, you need to do two things:

1. Set up the proper permissions in your serverless.yml file.
2. Use the aws-sdk of your preferred language (Node and Python 3 examples are provided) to invoke the second function from the first.

Permissions (iamRoleStatements)

Invoking a Lambda from another Lambda can't be done without some configuration. In your serverless.yml file, permission must be specified in order to invoke another Lambda. This can be accomplished by adding an iamRoleStatements section under the provider property (lines 4-8 below).

provider:
	name: aws
	runtime: <runtime goes here> # e.g. python3.6 or nodejs6.10
	iamRoleStatements:
	  - Effect: Allow
	    Action:
	      - lambda:InvokeFunction
	    Resource: "*"

On StackOverflow and other tutorials, you may also see - lambda:InvokeAsync in addition to what you see above (under Action). The InvokeAsync API is deprecated (see for yourself ), so you can exclude it from your serverless.yml file.

Invocation

Once the proper permissions are set up in serverless.yml, invoking one Lambda from another requires no tricks – you use the aws-sdk as you would normally.

Regardless of your runtime, the functions section of your serverless.yml file should look something like this:

functions:
  print_strings:
    handler: handler.print_strings

  cron_launcher:
    handler: handler.cron_launcher
    events:
      - schedule: rate(1 minute)

The print_string function has no event property, because it will be invoked directly via the aws-sdk. The cron_launcher function has an event property, where the event is defined as a cron schedule.

Node

Here's what the handler looks like in a Node project.

"use strict";

const AWS = require("aws-sdk");

const lambda = new AWS.Lambda({
  region: "us-west-2"
});

// The action lambda
module.exports.print_strings = (event, context, callback) => {
  const response = {
    statusCode: 200,
    body: JSON.stringify({
      message: `${event} - from the other function`
    })
  };
  callback(null, response);
};

// The cron Lambda
module.exports.cron_launcher = (event, context, callback) => {
  const fakeDBResults = [
    "Good morning.",
    "How are you?",
    "May I pet your dog?",
    "Oh, that's nice"
  ];

  fakeDBResults.forEach(message_string => {
    const params = {
      FunctionName: "lambda-invokes-lambda-node-dev-print_strings",
      InvocationType: "RequestResponse",
      Payload: JSON.stringify(message_string)
    };

    return lambda.invoke(params, function(error, data) {
      if (error) {
        console.error(JSON.stringify(error));
        return new Error(`Error printing messages: ${JSON.stringify(error)}`);
      } else if (data) {
        console.log(data);
      }
    });
  });
};

At the top of the file, import the aws-sdk and then initialize it with the region in which your Lambdas reside (lines 3-7). The print_string function returns a callback with a status code and a JSON body with a message property.

The interesting bits are in the cron_launcher function. Here you have an array of messages (fakeDBResults), and then a .forEach() (starting on line 29 above) method that invokes a print_string function for every message in the array (starting on line 36 above).

Interlude: Public Lambda Names

The cron_launcher is straightforward except for the FunctionName property in the params constant. This name is printed in the terminal after deploying your service. If you forget what the Lambda name is, with your terminal navigate into your root project folder and enter sls info -s <stage_name>. stage_name is typically something like dev, qa, staging, or production, depending on your deployment pipeline. Here's an example output:

Service Information
service: your-service-name
stage: dev
region: us-east-1
stack: your-service-name-stage_name
api keys:
  None
endpoints:
  None
functions:
  thing: your-service-name-dev-thing

In the output above, the function name within the service is thing but the public name – the name you need to invoke it – is your-service-name-dev-thing. AWS's pattern for generating the public name ofyour function is service_name-stage-function_name.

For a detailed overview of useful terminal commands, see my other post Serverless Framework Terminal Commands.

Python

Here's the same logic expressed in Python 3.

import json
from boto3 import client as boto3_client

lambda_client = boto3_client('lambda', region_name="us-west-2",)


def print_strings(event, context):
    print(event)
    body = {
        "message": "{} - from the other function".format(event),
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }

    return response


def cron_launcher(event, context):

    fakeDBResults = [
        "Good morning.",
        "How are you?",
        "May I pet your dog?",
        "Oh, that's nice"
    ]

    for message in fakeDBResults:
        response = lambda_client.invoke(
            FunctionName="lambda-invokes-lambda-python3-dev-print_strings",
            InvocationType='RequestResponse',
            Payload=json.dumps(message)
        )

        string_response = response["Payload"].read().decode('utf-8')

        parsed_response = json.loads(string_response)

        print("Lambda invocation message:", parsed_response)

At the top of the file, import the boto3 (the Python AWS SDK) and then initialize it with the region in which your Lambdas reside (lines 1-4). The print_string function returns a reponse with a status code and a JSON body with a message property.

In the cron_launcher function there's a list of messages (fakeDBResults), and a for loop that invokes a print_string function for every message in the list of messages. The Lambda response is then decoded (line 36), parsed (line 38), and printed out.