Data Quality - Column Validation Rules

Enforce data quality rules at column level with this set of Data Productivity Cloud pipelines.

Call this pipeline with a list of the data quality rules to enforce, in the grid variable ColumnValidationRules.

Every rule contains:

• DatabaseName - Database where the table exists
• SchemaName - Schema where the table exists
• TableName - Table where column is found
• DataColumnName - Name of column to validate
• RowIdentifierColumnName - Column in same table used to identify row e.g. Primary Key
• ValidationRule - Validation rule (NOT_NULL, IS_DATE or IS_NUMBER)

Creating column validation rules using Matillion Maia

Use the following prompt:

Create a reusable solution to run data quality checks against a named database table.
The solution should accept a list of rules.
Each rule should verify one of the following things about a column in a table:
- NOT_NULL - the column always has a value
- IS_DATE - the column is a VARCHAR but all values can be successfully converted into a DATE
- IS_NUMBER - the column is a VARCHAR but all values can be successfully parsed as numeric

Matillion Maia Demonstration

Downloads

Licensed under: Matillion Free Subscription License

• Download Data-Quality-Column-Validation-Rules-Snowflake.zip

Installation Instructions