Snowflake Note IV - Data Ingestion  [draft]

Content

May 10, 2020 · 1 min · word · Eric

7 Best Practices for Snowflake

General Introduction Key customers 7 Best Practices for Building Data Applications Isolation workloads with Snowflake Key features Zero-Copy Clone Time Travel AS OF feature Semi-structured Support add timestamp column Materialized Views Snowflake materialized views is different with other platforms Code in your Language snowflake connection diagnose system SnowPipe Use the Ecosystem Manage Cost Manage compute configure with Auto Suspend configure with Auto Resume configure suspend window after idle time window Monitoring resources all data app utilise resource monitoring FAQ Q: Does Snowflake a transactional database?...

May 5, 2020 · 5 min · 991 words · Eric

Snowflake Note III - Essential Tools

SnowSQL For macOS, you can use brew cask to install SnowSQL from command-line 1 brew cask install snowflake-snowsql JDBC Drive Snowflake JDBC maven repo: https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/ 1 wget https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.9.2/snowflake-jdbc-3.9.2.jar Python SDK 1 pip install snowflake-connector-python Spark SDK Snowflake Spark Connector repo: https://github.com/snowflakedb/spark-snowflake

May 4, 2020 · 1 min · 41 words · Eric

Snowflake Note II - Data Ingestion

In this note, I am going to create my first Database and table in Snowflake, and load csv data files from AWS S3 into our table. Typical Data Ingestion Process Prepare your files Stage the data Execute COPY command Managing regular loads Loading data from AWS S3 Bucket Step 1: Database and table initialization So first of all, we need to create our database. Here we are creating a new database called OUR_FIRST_DATABASE and our table called OUR_FIRST_TABLE...

May 3, 2020 · 2 min · 244 words · Eric

Communication in Workplace

Email Make the subject relevant, don’t reply to a lunch invite mail with critical business info - one topic per thread. Keep it short and to the point - long emails should be in a document of some sort. Don’t hide the actions in the middle - use @ to make them explicit and make them clearly visible. To means you need me to read it and/or do something. CC means you don’t need me to do anything, and I can read it when it suits me....

May 3, 2020 · 2 min · 265 words · Eric