Matillion ETL Shared Job
Author: Matillion
Date Posted: Nov 21, 2023
Last Modified: Nov 21, 2023
Author: Matillion
Date Posted: Nov 21, 2023
Last Modified: Nov 21, 2023
Schema Table Profiler
Automated table level data profiling for all tables in a named schema.
This shared job allows you to record table level metadata for all tables in a named schema, from a Matillion ETL orchestration job.
Provide a database name and a schema name, and the shared job will create a new view named VW_<schema_name>_PROFILED in that schema. The view contains one record per table, with the following metadata columns taken from INFORMATION_SCHEMA.TABLES.
- TABLE_NAME
- TABLE_CATALOG
- TABLE_SCHEMA
- TABLE_OWNER
- TABLE_TYPE
- IS_TRANSIENT
- CLUSTERING_KEY
- ROW_COUNT
- BYTES
- SUM_BYTES_IN_MB
- RETENTION_TIME
- CREATED
- LAST_ALTERED
- LAST_ALTERED_RANK
- AUTO_CLUSTERING_ON
- COMMENT
- SCHEMA_TOTAL_BYTES_IN_MB (aggregated to schema level, and repeated per table)
- SCHEMA_LATEST_ALTERED_TABLE (ditto)
- SCHEMA_OLDEST_ALTERED_TABLE (ditto)
- PROCESS_CREATE_TIMESTAMP
Parameters
| Parameter | Description |
|---|---|
| Database Name | Name of the database containing the schema to profile. Set to ${environment_database} to use the Environment default database |
| Schema Name | Name of the schema to profile. Set to ${environment_default_schema} to use the Environment default schema |
Downloads
Licensed under: Matillion Free Subscription License
- Download METL-sf-1.68.3-schema-table-profiler.melt
- Target: Snowflake
- Version: 1.68.3 or higher