Posts

Reference eugeneyan: 1-on-1s

Jenkins itself is not able to parse JSON response body. However, we can use Python with the built-in json library to parse JSON object within command-line. For example, if we request the URL like http://md5.jsontest.com/?text=example_text , this will return the values as 1 2 3 4 { md5: "fa4c6baa0812e5b5c80ed8885e55a8a6", original: "example_text" } With the help of Python json library, we can do something like 1 2 3 Url=http://md5.jsontest.com/?text=example_text Md5Info=`curl $Url -k | python -c "import sys, json; print json....

Book Info Name: Learning Spark Author: Holden Karau, Andy Konwinski, Patrick Wendell, Matei Zaharia Publisher: O’Reilly Media Release Date: 2015 Topic: Learning Apache Spark 1.x Preface Use one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and Storm. Apache Spark provides multiple components that can achieve lots of things: Spark SQL is the competitor of Hive for interactive queries MLlib is the competitor of Mahout for machine learning Spark Streaming is the competitor of Storm for streaming GraphX is the competitor of Neo4J for graph processing Apache Spark offers three main benefits: 1) easy to use....

Inmon Framework (Enterprise Data Warehouse) A centralized data repository that Kimball Framework (Dimensional Data Warehouse) Reference Data Warehouse Architecture Comparison: Kimball and Inmon dbt: Kimball in the Context of Morden Data Warehouse

Basic Concepts Dimensional Modelling Dimensional modeling is widely accepted as the preferred technique for presenting analytic data because it addresses two simultaneous requirements: Deliver that that’s understandable to the business users Deliver fast query performance Dimensional modelling always uses the concepts of facts and dimensions. Facts, or measures are typically (but not always) numeric values that can be aggregated; Dimensions are groups of hierarchies and descriptors that define the facts....