Matillion ETL Shared Job

Author: Matillion
Date Posted: Oct 20, 2023
Last Modified: Nov 9, 2023

Pip3 Install

Install a chosen Python library into a named location, similar to virtualenv for Matillion ETL.

Pip3 Install Shared Job

It can manage pip installations for any Python version listed under Admin > Manage Interpreters. In the example below, the display name “Python 3.8” is associated with the highlighted Python runtime /usr/bin/python3.8.

Manage Interpreters

Parameters

ParameterDescription
Library NameThe name of the library to install
Location StringChoose any string to install the library into the named location. Use the updated value of jv_script_header as the first line in any downstream Python script, to ensure that location is searched first. Alternatively (not recommended), specify “system” or “user” to install systemwide with pip’s --user parameter.
Python LocationThe executable path of one of the installed Python versions, from Admin > Manage Interpreters
Script HeaderLeave this parameter at its default value. It is an output parameter, used when a named location is chosen for installation

Named Location installation example

In the screenshot below, the paramiko library is installed for /usr/bin/python3.8 into a location named the-project.

Named Location installation

This is a named location installation, so downstream Python scripts will need to include the new path in their module search. The Shared Job’s Export tab saves this fragment of Python into a local variable.

Export Script Header

The first line in the following Python Script is ${my_py_header} which extends the module search path. After that, the script can safely import paramiko. Note that the Interpreter is set to Python 3.8, matching the Python Location used in the Shared Job.

Use Script Header


Downloads

Licensed under: Matillion Free Subscription License

  • Download Pip3-Install.melt
    • Target: Any target cloud data platform
    • Version: 1.68.3 or higher

Installation Instructions

How to Install a Matillion ETL Shared Job