CDC Load to Warehouse

Load data exported from a CDC agent into target tables on your chosen cloud data platform.

Please refer to the CDC shared jobs overview for full documentation.

These shared jobs are fully productionized and utilize a common pattern developed by the solution engineers within Matillion. The general steps to implement and manage them are detailed below.

Note that these shared jobs do not work with the Manage Change Data Capture feature in Matillion ETL. Instead, these shared jobs are for the CDC loading feature in Matillion Change Data Capture.

Parameters

Each shared job has its own reference page for information:

• Sync All Tables
• Sync Single Table
• Drop CDC Tables

For Snowflake, one additional shared job is provided:

• Create or Refresh External Table

Schema Drift

Schema Drift is handled by the shared jobs and is also covered in further detail in the CDC shared jobs overview document.

Downloads

Licensed under: Matillion Free Subscription License

• Download matillion-data-loader-cdc-bq.melt
  • Last updated: 2023-10-17
  • Platform: GCP
  • Target: Google BigQuery
  • Version: 1.63.4 or higher
• Download matillion-data-loader-cdc-dl.melt
  • Last updated: 2024-07-05
  • Target: Deltalake
  • Version: 1.72.0 or higher
• Download matillion-data-loader-cdc-rs.melt
  • Last updated: 2023-10-10
  • Platform: AWS
  • Target: Redshift
  • Version: 1.69.0 or higher
• Download matillion-data-loader-cdc-sf.melt
  • Last updated: 2024-10-14
  • Target: Snowflake
  • Version: 1.63.4 or higher

Installation Instructions