Infrastructure

Kafka Theory Cluster Rack Broker Every broker in Kafka is a “bootstrap server” which knows about all brokers, topics and partitions (metadata) that means Kafka client (e.g. producer, consumer etc) only need to connect to one broker in order to connect to entire cluster. At all times, only one broker should be the controller, and one broker must always be the controller in the cluster Topic Kafka takes bytes as input without even loading them into memory (that’s called zero copy) Brokers have defaults for all the topic configuration parameters Partition Topic can have one or more partition....

Recently I just faced some network issues when building up docker images on our AWS Bastion. The docker build command cannot pick up the environment variable for network proxy. Network Issue with Docker Build When I used curl -i https://registry.docker.io to check the network, everything looks OK. But when I was trying to build the docker image via docker build -t nba ./ and it raised the following Timeout error....

kubectl create is so called Imperative Management. This approach will tell Kubernetes API what you want to create, replace or delete, not how you want your Kubernetes cluster world to look like. kubectl apply is part of Declarative Management approach, where changes that you may have applied to a live object (i.e. through scale) are maintained even if you apply other changes to the object. Both approaches are valid ways to work in production....

Prune images 1 docker image prune -af or 1 docker rm $(docker ps -q -a -f "status=exited")

Jenkins itself is not able to parse JSON response body. However, we can use Python with the built-in json library to parse JSON object within command-line. For example, if we request the URL like http://md5.jsontest.com/?text=example_text , this will return the values as 1 2 3 4 { md5: "fa4c6baa0812e5b5c80ed8885e55a8a6", original: "example_text" } With the help of Python json library, we can do something like 1 2 3 Url=http://md5.jsontest.com/?text=example_text Md5Info=`curl $Url -k | python -c "import sys, json; print json....