overview
This solution architecture shows how to use Snowflake Notebooks to build an end-to-end data engineering workflow.
- Load Parquet data to Snowflake using schema inference
- Setup access to Snowflake Marketplace weather data
- Create a Python UDF to run transformations
- Build a data engineering pipeline with Python stored procedures to incrementally process data