Matillion ETL Shared Job
Author: Matillion
Date Posted: Nov 14, 2023
Last Modified: Nov 14, 2023
Author: Matillion
Date Posted: Nov 14, 2023
Last Modified: Nov 14, 2023
Email Attachment Download
Download all attachments from a specified email, and save to cloud storage.
Typically used inside an Iterator component that loops over the Email ID values previously acquired from an Email Query component. The idea is that the Email Query component retrieves emails that match specific criteria. This shared job is used afterwards to download all the attachments from those emails.
Parameters
| Parameter | Description |
|---|---|
| IMAP Server | IMAP server name or IP address e.g. imap.gmail.com |
| IMAP Port | IMAP Server Port (typically 993) |
| Email User | Email User |
| Email Password | Password Manager entry for Email User under Projects > Manage Passwords |
| Email Folder | Email folder that the Email resides in |
| Email ID | The unique internal id of an email that has attachments to be downloaded, as returned by the Email Query component |
| Local Stage Directory | Directory path that defines where to temporarily stage the email attachment file before moving to cloud storage, e.g. /tmp |
| Delete Staged Data | A stage table and view is created for each email. Set to 1 to delete these objects upon completion |
Extra parameters for Snowflake on AWS
| Parameter | Description |
|---|---|
| Snowflake Warehouse | Snowflake Warehouse used when staging metadata |
| Snowflake Database | Snowflake Database used for staged metadata |
| Snowflake Schema | Snowflake Schema used for staged metadata |
| S3 Bucket Stage | AWS S3 Bucket for staging temporary data, e.g. yourS3Bucket or yourS3Bucket/yourFolder |
| S3 Bucket Target | AWS S3 Bucket to copy downloaded attachment files to, e.g. yourS3Bucket or yourS3Bucket/yourFolder |
Extra parameters for Redshift on AWS
| Parameter | Description |
|---|---|
| Redshift Schema | Redshift Schema used for staged metadata |
| S3 Bucket Stage | AWS S3 Bucket for staging temporary data, e.g. yourS3Bucket or yourS3Bucket/yourFolder |
| S3 Bucket Target | AWS S3 Bucket to copy downloaded attachment files to, e.g. yourS3Bucket or yourS3Bucket/yourFolder |
Extra parameters for Snowflake on Azure
| Parameter | Description |
|---|---|
| Snowflake Warehouse | Snowflake Warehouse used when staging metadata |
| Snowflake Database | Snowflake Database used for staged metadata |
| Snowflake Schema | Snowflake Schema used for staged metadata |
| Stage Azure Storage Account | Azure Storage Account used to stage metadata |
| Stage Azure Blob Container | Azure Blob Container used for staged metadata |
| Target Azure Storage Account | Azure Storage Account target for the Email Attachment |
| Target Azure Blob Container | Azure Blob Container target for the Email Attachment |
Extra parameters for Google BigQuery
| Parameter | Description |
|---|---|
| BigQuery Project | Google BigQuery Project used for staged metadata |
| BigQuery Dataset | Google BigQuery Dataset used for staged metadata |
| Stage Cloud Storage Bucket | Google Cloud Storage Bucket for staging temporary data, e.g. myGCSBucket or myGCSBucket/myFolder |
| Target GCS Bucket | Google Cloud Storage Bucket to copy downloaded attachment files to, e.g. myGCSBucket or myGCSBucket/myFolder |
Downloads
Licensed under: Matillion Free Subscription License
- Download METL-aws-rs-1.42.7-email-attachment-download.melt
- Platform: AWS
- Target: Redshift
- Version: 1.42.7 or higher
- Download METL-aws-sf-1.42.7-email-attachment-download.melt
- Platform: AWS
- Target: Snowflake
- Version: 1.42.7 or higher
- Download METL-az-sf-1.42.9-email-attachment-download.melt
- Platform: Azure
- Target: Snowflake
- Version: 1.42.9 or higher
- Download METL-gcp-bq-1.42.9-email-attachment-download.melt
- Platform: GCP
- Target: Google BigQuery
- Version: 1.42.9 or higher