Matillion ETL Shared Job

Run your code via AWS Lambda serverless cloud infrastructure.

This shared job enables you to run scripting code using AWS Lambda, rather than locally on your Matillion ETL instance with a Python Script or Bash Script component.

Using serverless cloud infrastructure in this way has two DevOps advantages:

To use this shared job, create your AWS Lambda source code as a .zip file and save it into an S3 bucket. You must also create an IAM role with the privileges that the Lambda function will need to run successfully.

Run Serverless via AWS Lambda


AWS RegionAWS region e.g. eu-west-1
Handler NameName of the Lambda handler function
IAM Role ARNThe ARN of the IAM role for the function execution. Pattern: arn:(aws[a-zA-Z-]*)?:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@-_/]+
Lambda function nameThe name of the Lambda function, e.g. tmp_myfunction
Memory MbAmount of memory to allocate, in Mb
Lambda RuntimeThe Lambda Runtime, e.g. python3.6
Source bucket nameName of S3 bucket containing the source code
Source file nameFilename of the source code, e.g.
Timeout secsTimeout for the function, in seconds
S3 log outputFull S3 path for the output logfile E.g. s3://mybucket/mypath/mylogfile.log


The aws command line utility must be installed on your Matillion ETL instance. If the shared job fails with an error line X: aws: command not found then please follow this guide to installing the aws command.

This shared job deletes and recreates the Lambda function every time before attempting to run it. Ensure that the EC2 instance credentials attached to your Matillion ETL instance include the privilege to manage Lambda functions, and to read from S3. For more information, refer to the “IAM in AWS” section in this article on RBAC in the Cloud.


Licensed under: Matillion Free Subscription License

Download METL-aws-1.61.6-run-serverless-aws-lambda.melt

Installation instructions

How to Install a Matillion ETL Shared Job

Author: Matillion