Snowflake Note II - Data Ingestion

In this note, I am going to create my first Database and table in Snowflake, and load csv data files from AWS S3 into our table. Typical Data Ingestion Process Prepare your files Stage the data Execute COPY command Managing regular loads Loading data from AWS S3 Bucket Step 1: Database and table initialization So first of all, we need to create our database. Here we are creating a new database called OUR_FIRST_DATABASE and our table called OUR_FIRST_TABLE...

May 3, 2020 · 2 min · 244 words · Eric

Communication in Workplace

Email Make the subject relevant, don’t reply to a lunch invite mail with critical business info - one topic per thread. Keep it short and to the point - long emails should be in a document of some sort. Don’t hide the actions in the middle - use @ to make them explicit and make them clearly visible. To means you need me to read it and/or do something. CC means you don’t need me to do anything, and I can read it when it suits me....

May 3, 2020 · 2 min · 265 words · Eric

Snowflake Note I - General Introduction

General Introduction Basic ideas about Snowflake Separation of storage and compute Built for Cloud environment built from scratch optimized for cloud storage & compute is decoupled Software as a Service No software, infrastructure or upgrade to manage Pay only for used compute & storage Storage & compute charged independently, only for use Scalable Virtual warehouse enable compute scaling Authentication Configuration Resource Management Data Protection Availability Optimization Unique features in Snowflake Time travel Cloning More details can be found at {% post_link snowflake-online-talk ‘Snowflake Online Talk’ %}...

April 28, 2020 · 1 min · 176 words · Eric

Be Energetic

Energy Energy Types Physical Energy Emotional Energy Mental Energy Spiritual Energy Physical Energy Symptom & Complication I have lots of habits and lots of things I want to do, I am just too tired and out of energy to do it. This energy is probably the simplest one to understand. This is about your Sleeping Eating Exercising If you are not looking after yourself physically, you cannot achieve anything in life....

March 17, 2020 · 1 min · 210 words · Eric

Git - how to move a folder from one repo to another and keep its commits history

I was encountered into this situation that I need to transfer some existing code to another separated GitHub repository. Ideally it would be perfect to keep all historic git commit histories so that we can still be able to revert back if necessary. After digging into Google and Stack Overflow, luckily I found there are others who need to achieve similar things. Here is the solution I used to resolve this problem eventually....

March 15, 2020 · 2 min · 386 words · Eric