Author: Matillion
Date Posted: Nov 9, 2023
Last Modified: Nov 22, 2023
Move GCS File
Move a file from one location in Google Cloud Storage to another, optionally changing the file name.
The shared job will fail if the source or target bucket is not known, or if the source object name did not match any files, or if the instance credentials lack the necessary privilege.
Parameters
| Parameter | Description |
|---|---|
| Source Bucket Name | The source GCS bucket name (do not include the gs:// prefix, do not include the object path) |
| Source Object Name | The source GCS object name (including path, but do not prefix with /). If you use wildcard(s) then the Target Object Name will be treated as a prefix. For more information refer to Google’s gsutil documentation |
| Target Bucket Name | The target GCS bucket name (do not include the gs:// prefix, do not include the object path) |
| Target Object Name | The target GCS object name (including path, but do not prefix with /) |
Prerequisites
GCS permissions are inherited from the Matillion ETL VM instance. The gsutil mv command performs a copy from source to destination followed by removing the source. So you must ensure that the Service Account attached to your Matillion ETL VM instance has permission to read and delete objects from cloud storage. For more information, refer to the “IAM in GCP” section in this article on RBAC in the Cloud.
This shared job uses the gsutil command to move the file. It is only pre-installed by default on GCP VM instances.
Downloads
Licensed under: Matillion Free Subscription License
- Download METL-1.51.5-move-gcs-file.melt
- Target: Any target cloud data platform
- Version: 1.51.5 or higher