Matillion ETL Shared Job

Author: Matillion
Date Posted: Nov 9, 2023
Last Modified: Nov 22, 2023

Move GCS File

Move a file from one location in Google Cloud Storage to another, optionally changing the file name.

The shared job will fail if the source or target bucket is not known, or if the source object name did not match any files, or if the instance credentials lack the necessary privilege.

Parameters

ParameterDescription
Source Bucket NameThe source GCS bucket name (do not include the gs:// prefix, do not include the object path)
Source Object NameThe source GCS object name (including path, but do not prefix with /). If you use wildcard(s) then the Target Object Name will be treated as a prefix. For more information refer to Google’s gsutil documentation
Target Bucket NameThe target GCS bucket name (do not include the gs:// prefix, do not include the object path)
Target Object NameThe target GCS object name (including path, but do not prefix with /)

Prerequisites

GCS permissions are inherited from the Matillion ETL VM instance. The gsutil mv command performs a copy from source to destination followed by removing the source. So you must ensure that the Service Account attached to your Matillion ETL VM instance has permission to read and delete objects from cloud storage. For more information, refer to the “IAM in GCP” section in this article on RBAC in the Cloud.

This shared job uses the gsutil command to move the file. It is only pre-installed by default on GCP VM instances.


Downloads

Licensed under: Matillion Free Subscription License

Installation Instructions

How to Install a Matillion ETL Shared Job