Matillion ETL Shared Job

Schema Table Profiler

Automated table level data profiling for all tables in a named schema.

This shared job allows you to record table level metadata for all tables in a named schema, from a Matillion ETL orchestration job.

Provide a database name and a schema name, and the shared job will create a new view named VW_<schema_name>_PROFILED in that schema. The view contains one record per table, with the following metadata columns taken from INFORMATION_SCHEMA.TABLES.

TABLE_NAME
TABLE_CATALOG
TABLE_SCHEMA
TABLE_OWNER
TABLE_TYPE
IS_TRANSIENT
CLUSTERING_KEY
ROW_COUNT
BYTES
SUM_BYTES_IN_MB
RETENTION_TIME
CREATED
LAST_ALTERED
LAST_ALTERED_RANK
AUTO_CLUSTERING_ON
COMMENT
SCHEMA_TOTAL_BYTES_IN_MB (aggregated to schema level, and repeated per table)
SCHEMA_LATEST_ALTERED_TABLE (ditto)
SCHEMA_OLDEST_ALTERED_TABLE (ditto)
PROCESS_CREATE_TIMESTAMP

Parameters

Parameter	Description
Database Name	Name of the database containing the schema to profile. Set to `${environment_database}` to use the Environment default database
Schema Name	Name of the schema to profile. Set to `${environment_default_schema}` to use the Environment default schema

Downloads

Licensed under: Matillion Free Subscription License

Download METL-sf-1.68.3-schema-table-profiler.melt

Target: Snowflake
Version: 1.68.3 or higher

Installation Instructions

How to Install a Matillion ETL Shared Job

Author: Matillion
Date Posted: Nov 21, 2023
Last Modified: Nov 21, 2023