snowflake

Features and Architecture Snowflake Architecture Snowflake Editions Standard Edition Enterprise Edition Business Critical Edition Virtual Private Snowflake (VPS) Table Types Temporary Table Transient Table Permanent Table External Table Virtual Warehouse Data Spilling Caching Layers Query Result Cache Metadata Cache Warehouse Data Cache Clustering Cache (aka Partition Pruning Efficiency) Key Takeaways Time Travel Fail-safe Important Notes Summary Task Stored Procedure System Tables and Views Resource Monitors Account Access and Security Snowflake Account Roles Privileges Ownership Monitor Privilege Requirements for Common Use Cases Authentication Methods Multi-Factor Authentication (MFA) Federated Authentication (SSO) Network Policy Key Rotation Performance Concepts Scaling Policy Search Optimisation Service (SOS) Table Clustering Depth Materialised View Data Loading and Unloading Upload into Stage Snowflake Stage Internal Stage External Stage Temporary Stage Directory Table in Snowflake Stage COPY INTO Load data into Table Unload data from Table to Stage ON_ERROR Truncating column values PURGE=TRUE INSERT OVERWRITE Snowpipe Data Transformation VARIANT Data Type Clustering Key Automatic Clustering Flatten() Table Clone Privilege APPROX_COUNT_DISTINCT Sampling Data Protection and Data Sharing Data Sharing Row Access Policy External Functions Reference Features and Architecture Snowflake Architecture Snowflake uses a multi-cluster shared data architecture, means...

Content

General Introduction Key customers 7 Best Practices for Building Data Applications Isolation workloads with Snowflake Key features Zero-Copy Clone Time Travel AS OF feature Semi-structured Support add timestamp column Materialized Views Snowflake materialized views is different with other platforms Code in your Language snowflake connection diagnose system SnowPipe Use the Ecosystem Manage Cost Manage compute configure with Auto Suspend configure with Auto Resume configure suspend window after idle time window Monitoring resources all data app utilise resource monitoring FAQ Q: Does Snowflake a transactional database?...

SnowSQL For macOS, you can use brew cask to install SnowSQL from command-line 1 brew cask install snowflake-snowsql JDBC Drive Snowflake JDBC maven repo: https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/ 1 wget https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.9.2/snowflake-jdbc-3.9.2.jar Python SDK 1 pip install snowflake-connector-python Spark SDK Snowflake Spark Connector repo: https://github.com/snowflakedb/spark-snowflake

In this note, I am going to create my first Database and table in Snowflake, and load csv data files from AWS S3 into our table. Typical Data Ingestion Process Prepare your files Stage the data Execute COPY command Managing regular loads Loading data from AWS S3 Bucket Step 1: Database and table initialization So first of all, we need to create our database. Here we are creating a new database called OUR_FIRST_DATABASE and our table called OUR_FIRST_TABLE...