Managing Your Python Environment with Pipenv  [draft]

General Pipenv is a packaging tool for Python that solves some common problems associated with the typical workflow using pip, virtualenv, and requirements.txt. Pipenv is designed to resolve dependency management chaos created by requirements.txt. Introduction First, let’s install the pipenv package. Pipenv uses pip and virtualenv under the hood but simplifies their usage with a single command-line interface. 1 pip install --user pipenv Pipenv introduces two new files. Pipfile is to replace the old requirements....

December 3, 2015 · 4 min · 677 words · Eric

Type Hint with Python  [draft]

Bad Code 1 2 3 4 5 6 7 def get_authors_names(posts): authors_names = [] for post in posts: author = post["author"] author_name = author["name"] authors_names.append(author_name) return authors_names Good code 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 from typing import List, TypedDict class Author(TypedDict): name: str email: str bio: str website: str class Post(TypeDict): title: str author: Author publication_date: str content: str def get_authors_names(posts: List[Post]) -> List[str]: return [post["author"]["name"] for post in posts]

December 3, 2015 · 1 min · 81 words · Eric

Interview Preparation for Hiring Manager Round

Question List Role Team Company Culture Values What are the company’s mission and value statements? Why: This is a basic check on whether the interviewer know about company culture or not. Why: This is to check whether the interviewer acknowledge / care about big picture of the company What is the part of the company culture that you are proud of? Why: This is to check whether the interviewer is truly engaged and believe with company culture or not....

September 20, 2015 · 2 min · 274 words · Eric

Hive Query Performance Tuning

There are several parameters that we can tune in Hive to improve the overall query performance. For instance, 1 2 3 4 5 6 7 8 9 10 11 12 -- refers to http://hortonworks.com/community/forums/topic/mapjoinmemoryexhaustionexception-on-local-job/ -- before running your query to disable local in-memory joins and force the join to be done as a distributed Map-Reduce phase. -- After running your query you should set the value back to true with: set hive....

May 11, 2015 · 1 min · 108 words · Eric

Something about GeoJSON

Introduction GeoJSON is a subset of JSON. According to GeoJSON site, GeoJSON is a format for encoding a variety of geographic data structures. Currently GeoJSON supports: Point LineString Polygon MultiPoint MultiLineString MultiPolygon With in the geometric objects, we need to add an extra object called Feature. A set of features will be contained in a FeatureCollection objects. Examples Point LineString Polygon Reference GeoJSON site GeoJSON Specification Learn GeoJSON GeoJSON.io JSON Lint

May 10, 2015 · 1 min · 71 words · Eric