Data Productivity Cloud Pipeline

Author: Matillion
Date Posted: Jun 12, 2024
Last Modified: Jun 12, 2024

Snowflake Cortex Components

Use Snowflake Cortex generative AI capabilities in a Data Productivity Cloud transformation pipeline.

The example is from a travel company, which collects hotel stay reviews. In the screenshot below, they arrive in the Review comments component.

Image ofMatillion Snowflake Cortex Components
Matillion Snowflake Cortex Components

Install this pipeline in your own environment, and use the Sample Data tab on all the components to view the Snowflake Cortex functions in action.

More details are available in this article on Snowflake Cortex.

Snowflake Cortex TRANSLATE

The review comments arrive in French, whereas the company’s standard language is English. The Cortex Translate component uses SNOWFLAKE.CORTEX.TRANSLATE to convert the review from French to English.

Snowflake Cortex SUMMARIZE

After translation into English, a Cortex Summarize component is used to generate a summary of the hotel visitor feedback. Internally this component uses SNOWFLAKE.CORTEX.SUMMARIZE.

Snowflake Cortex SENTIMENT

Next a Cortex Sentiment component analyzes the comment and generates a sentiment number in the range between -1 (very negative) and 1 (very positive). This generates a SQL function call to SNOWFLAKE.CORTEX.SENTIMENT.

Once the sentiment has been expressed as a number, it’s easy to filter and deal with the records appropriately. The Negative Reviews filter component selects only records with a sentiment score less than -0.5. These are strongly negative reviews, and need to be handled sensitively.

Snowflake Cortex EXTRACT_ANSWER

This is another interface to Snowflake’s internal generative AI functionality. In this example the question presented is What did the guest especially dislike?. Summary insights like this will help the travel company prioritize improvements to their hotels.

Behind the component there is a call to the SNOWFLAKE.CORTEX.EXTRACT_ANSWER function.

Snowflake Cortex COMPLETE

This is a conversational prompt/response component, which offers a choice of Large Language Model. In this example the settings are:

  • mistral-large LLM
  • System Prompt: You are a hotel manager dealing with complaints
  • User Prompt: Write an apology to the guest based on their comment

Internally this component uses SNOWFLAKE.CORTEX.COMPLETE, with the prompt generated from a combination of the properties.

When you run this, you will notice the apology is automatically written in the original language of the comment. There is no need to explicitly request a response in French. A final Extract Nested Data component extracts just the text response from the LLM.


Downloads

Licensed under: Matillion Free Subscription License

Installation Instructions

How to Install a Data Productivity Cloud Pipeline