- Overview
- Architecture
- Data Warehouse Data Model
- Instructions
- Reference
- Known Issues and Solution
- Future Enhancements
This repository contains code that demonstrates the use of ChatGPT to create and query data models in Snowflake.
Refer to the diagram below for a high-level architecture.
The model below was generated using the prompt in a ChatGPT chat session.
- vendor_id (PK)
- vendor_code
- vendor_name
- date_id (PK)
- date
- year
- month
- day
- day_name
- month_name
- day_of_week
- day_of_year
- is_weekend
- is_holiday
- rate_id (PK)
- rate_code
- rate_name
- location_id (PK)
- longitude
- latitude
- payment_type_id (PK)
- payment_type_code
- payment_type_name
- trip_id (PK)
- vendor_id (FK)
- pickup_date_id (FK)
- dropoff_date_id (FK)
- pickup_location_id (FK)
- dropoff_location_id (FK)
- rate_id (FK)
- payment_type_id (FK)
- passenger_count
- trip_distance
- pickup_timestamp
- dropoff_timestamp
- fare_amount
- extra
- mta_tax
- tip_amount
- tolls_amount
- improvement_surcharge
- total_amount
Refer to this blog for more details.
- https://www.kaggle.com/datasets/elemento/nyc-yellow-taxi-trip-data?resource=download
- https://www.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf
- https://github.com/darshilparmar/uber-etl-pipeline-data-engineering-project
- https://github.com/fivethirtyeight/data
- https://pythonspeed.com/articles/activate-conda-dockerfile/
- https://github.com/mydgd/snowflake-table-catalog/tree/main
- https://quickstarts.snowflake.com/guide/frosty_llm_chatbot_on_streamlit_snowflake/
| Error | Solution |
|---|---|
| When testing on a schema with a few hundred tables and columns, the following error occurred: InvalidRequestError: This model's maximum context length is 4097 tokens. However, your messages resulted in 22564 tokens. Please reduce the length of the messages. | You must use a different model which supports more tokens, or logically reduce the tokens before making the API call. |
| SnowparkSQLException: (1304): 390114 (08001): Authentication token has expired. The user must authenticate again. |
| Feature | Status |
|---|