Initial import: links posts + github knowledge

2026-08-09 05:06:40 +10:00 · 2026-04-08 11:32:53 +10:00
commit 25063b4790
2376 changed files with 107477 additions and 0 deletions
@@ -0,0 +1,54 @@
+# GitHub Knowledge
+
+Technical notes and reference materials ingested from [github.com/wahyd4/knowledge](https://github.com/wahyd4/knowledge).
+
+## Structure
+
+```
+github-knowledge/
+├── application.md
+├── apps.md
+├── database.md
+├── devops.md
+├── fintech.md
+├── go.md
+├── iot.md
+├── java.md
+├── javascript.md
+├── kotlin.md
+├── microservice.md
+├── ruby.md
+├── software-design.md
+├── team-work.md
+├── testing.md
+├── uncategorised.md
+├── web.md
+├── web-scraping.md
+├── apps/
+│   ├── elasticsearch.md
+│   ├── etcd.md
+│   ├── kafka.md
+│   └── mqtt.md
+├── devops/
+│   ├── aws.md
+│   ├── docker.md
+│   ├── gcp.md
+│   ├── incident.md
+│   ├── kubernetes.md
+│   ├── linux.md
+│   └── more-tools.md
+├── english/
+│   └── business-spoken-english.md
+└── interviews/
+    └── interview.md
+```
+
+## Format
+
+Each file includes YAML frontmatter with:
+- Title
+- Created date (from git history)
+- Updated date (from git history)
+- Type: summary
+- Tags: tech, reference
+- External link to original source on GitHub
@@ -0,0 +1,162 @@
+---
+title: Application
+created: 2019-05-13
+updated: 2019-05-13
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/application.md
+---
+
+# Search
+
+- ElasticSearch
+	- Logstash
+	- Kibana
+
+# APM
+
+## Newrelic
+
+The famous APM service provider, it support multiple languages, it requires you install agent to the application.
+## Datadog
+
+Another alternative APM service provider
+## Pinpoint
+
+The Java based open source APM.
+
+# Ping check service
+
+## Pingdom
+It's a online web services, it gives you ability to do ping to your website from different regions, web servers.
+
+# Monitoring
+
+## Bosun
+
+## Prometheus
+
+### Prometheus Metric types
+
+* Counter, A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. For example, you can use a counter to represent the number of requests served, tasks completed, or errors. Do not use a counter to expose a value that can decrease.
+* A gauge is a metric that represents a single numerical value that can arbitrarily go up and down.
+Gauges are typically used for measured values like temperatures or current memory usage, but also "counts" that can go up and down, like the number of running goroutines.
+* A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values.
+* Summary, Similar to a histogram, a summary samples observations (usually things like request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.
+
+
+# API
+
+## RPC(remote procedure call)
+	- disadvantages
+		- coupling
+		- clients needs to know the request procedure names
+	- advantages
+		- more freedoms to define any requests.
+	- examples: <https://api.slack.com/methods>
+
+## Restful /Representational State Transfer
+	- Easy to cache
+	- every url represent a resource
+	- disadvantages some special method is hard to use restful to name it. like /login, /resetpassword
+	- examples [https://developer.github.com/v3/](https://developer.github.com/v3/)
+	- methods
+		- GET
+		- POST
+		- PUT
+        - DELETE
+
+## GraphQL
+
+ It Can allows you to define fields you want to have.
+
+## gRPC
+	- protocol buffers
+
+```go
+    // The greeter service definition.
+    service Greeter {
+        // Sends a greeting
+        rpc SayHello (HelloRequest) returns (HelloReply) {}
+    }
+    // The request message containing the user's name.
+    message HelloRequest {
+        string name = 1;
+    }
+    // The response message containing the greetings
+    message HelloReply {
+        string message = 1;
+    }
+```
+	- A opensource  remote procedure call framework based on HTTP2 and protobuf
+	- advantages
+		- baed on HTTP2, better performance
+		- call remote like local method, and support multiple language
+		- protobuf serialization and deserialization is faster than JSON
+
+## JSON-RPC
+
+It's a little bit similar with gRPC one, but you define the action method in the JSON request body.
+
+POST /api
+```json
+{
+    "jsonrpc": "2.0",
+    "method": "subtract",
+    "params": [42, 23],
+    "id": 1
+}
+```
+Links:
+* JSON-RPC specification: <http://www.jsonrpc.org/specification>
+* O API - an alternative to REST APIs
+: <https://hackernoon.com/o-api-an-alternative-to-rest-apis-e9a2ed53b93c>
+
+## Open API
+
+The OpenAPI Specification (OAS) defines a standard, programming language-agnostic interface description for REST APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic. When properly defined via OpenAPI, a consumer can understand and interact with the remote service with a minimal amount of implementation logic. Similar to what interface descriptions have done for lower-level programming, the OpenAPI Specification removes guesswork in calling a service. Open API can be described in `yaml` or `json`
+
+### Use cases
+
+* Interactive documentation
+* Mock server generation
+* Client code generation
+* Automation testing
+
+### Implementation
+
+* [Swagger](http://swagger.io)
+
+### Tools
+
+[Redoc](https://github.com/Rebilly/ReDoc) OpenAPI/Swagger-generated API Reference Documentation, which gives you beautiful interface to the API spec file.
+
+`redoc-cli` is a command line tool to generate a single HTML version of API Spec file.
+
+```bash
+redoc-cli bundle swagger.json
+```
+Some sample swagger files: <http://rackerlabs.github.io/wadl2swagger/openstack.html>
+
+## Versioning
+
+### Proper define the version of your application
+
+According to [semver](https://semver.org/), the following is the general guideline.
+
+> Given a version number MAJOR.MINOR.PATCH, increment the:
+>
+> 1. MAJOR version when you make incompatible API changes,
+> 2. MINOR version when you add functionality in a backwards-compatible manner, and
+> 3. PATCH version when you make backwards-compatible bug fixes.
+> Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.
+
+# Log and Error
+
+## Sentry
+
+Within Sentry, you can view all your errors, and also do filtering and grouping things.
+
+## ELK
+
+Elasticsearch + Logstash + Kibana
@@ -0,0 +1,10 @@
+---
+title: Apps
+created: 2022-04-24
+updated: 2022-04-24
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/apps.md
+---
+
+# Some apps and tools
@@ -0,0 +1,26 @@
+---
+title: Elasticsearch
+created: 2022-04-30
+updated: 2022-04-30
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/apps/elasticsearch.md
+---
+
+# Elasticsearch
+
+Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
+
+
+## Tips
+
+## Calculate number of replicas
+
+> N >= R + 1
+
+N : number of nodes
+R: Number of replicas
+
+### How to resolve unassigned shards
+
+https://www.datadoghq.com/blog/elasticsearch-unassigned-shards/#reason-3-you-need-to-reenable-shard-allocation
@@ -0,0 +1,44 @@
+---
+title: Etcd
+created: 2022-06-10
+updated: 2022-06-10
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/apps/etcd.md
+---
+
+# etcd
+
+etcd is a strongly consistent, distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. It gracefully handles leader elections during network partitions and can tolerate machine failure, even in the leader node.
+
+## Set up a ETCD cluster
+
+https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md
+
+##  Tips
+
+### Make the k8s embedded single etcd to be a cluster
+
+Unfortunately, there is no straightfoward way to change the existing embedded etcd to a cluster.
+But we can leverage etcd `make-mirror` command to replicate the existing etcd to a new cluster.
+The steps are:
+
+1. Follow the guide to setup a new etcd cluster
+2. Run the following command to replicate k8s emmbedded etcd data to the new cluster
+
+```
+sudo ETCDCTL_API=3 etcdctl make-mirror https://new_etcd_node:2379 --endpoints=https://192.168.1.2:2379  --cacert=/k8s_etcd/ca.crt   --cert=/k8s_etcd/server.crt   --key=/k8s_etcd/server.key   --dest-cacert=/new_etcd/ca.pem   --dest-cert=/new_etcd/etcd.pem   --dest-key=/new_etcd/etcd-key.pem
+```
+3. On one of the new cluster node, to verify the data
+```
+sudo ETCDCTL_API=3 etcdctl get --prefix=true "" -w json  --endpoints=https://127.0.0.1:2379   --cacert=/new_etcd/ca.pem   --cert=/new_etcd/etcd.pem   --key=/new_etcd/etcd-key.pem
+```
+
+### Join another existing cluster
+
+1. Run the join command on an existing node, for instnace
+```
+sudo ETCDCTL_API=3 etcdctl member add server-2 --peer-urls=https://192.168.1.xxx:2380  --endpoints=https://192.168.1.xxx:2379,https://192.168.1.xxx:2379   --cacert=/etc/etcd/ca.pem   --cert=/etc/etcd/etcd.pem   --key=/etc/etcd/etcd-key.pem
+```
+2. Prepare for the configursations for the new node, and remember to set `--initial-cluster-state` to existing, add put all the existing nodes to `--initial-cluster` field
+3. Start the etcd service of the new node
@@ -0,0 +1,46 @@
+---
+title: Kafka
+created: 2022-04-24
+updated: 2022-04-24
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/apps/kafka.md
+---
+
+# Kafka
+
+# What is Kafka
+
+<iframe width="560" height="315" src="https://www.youtube.com/embed/FKgi3n-FyNU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+
+Apache Kafka is an open-source distributed event streaming platform
+
+## How to migrate self hosted Kafka to Some hosting vendor
+
+1. Setup target Kafka cluster
+2. Setup various replicators to replicate schemas and events in kafka. Before replicate schemas, make sure the schema registry in target cluster is empty and in `IMPORT_ONLY` mode.
+3. Setup users, roles for applications in the target clusters
+4. Start migrate consumers including apps and sink connectors
+    1. Reset consumer offset in the target Kafka cluster for a consumer(As the offsert will be different in most cases, unless the no events from the source cluster ever been deleted.)
+    2. Update Kafka configurations and scema registries for the consumer to the new cluster
+    3. Restarted consumer to consume events from the new cluster
+5. Stop schema replicator before migrating publisher, otherwise new publisher won't be able to publish schemas as the schema registry is in IMPORT mode. Once updating target schema registry to `READ_WRITE` mode, then schema replicator won't work anymore.
+6. Once all the consumers migerated, then migrated publichers (apps and source connectors)
+    1. Update Kafka configurations for Kafka clusters and shema registries
+    2. Restart publishers
+7. Stop replcators and source Kafka clusters
+
+## A typical Kafka workflow with schema registry
+
+![typical workflow](https://docs.confluent.io/platform/current/_images/schema-registry-and-kafka.png)
+
+Note: image from https://docs.confluent.io/platform/current/schema-registry/index.html
+
+
+## Some facts and best practices
+
+* Set proper partitions for topics, would be better to start with 3 or more depends on the numbers of events and type of events in the topic
+* Enable RBAC for topics, which means a Kafka user/service account can only access what he supposed to access. Minimal access scope
+* Use Debezium related source connnectors to achieve at least once delivery guarantee.
+* Use various sink connectors to export data via Restful API/S3 bucket and so on.
+* Set proper schema compatibility levels and tests to make sure existing events can be still consumed by new version of consumers.
@@ -0,0 +1,12 @@
+---
+title: Mqtt
+created: 2022-04-24
+updated: 2022-04-24
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/apps/mqtt.md
+---
+
+## MQTT
+
+A lightweight messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks. It's basically a machine-to-machine `protocol`
@@ -0,0 +1,186 @@
+---
+title: Database
+created: 2021-01-26
+updated: 2021-01-26
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/database.md
+---
+
+# Relation
+- Mysql
+- Postgres
+- Microsoft SQL Server
+- Oracle
+- Sqlite
+
+# NoSql
+- Monodb
+	- Replica Set
+		- Primary
+		- Secondary
+		- Hidden
+- Redis
+	- Cluster
+		- Strong recommend 3 masters and 3 slaves
+- Hbase
+- Cassendra
+
+# SQL
+
+## Index
+	- Index is unique but not mandatory
+## join
+
+## MYSQL
+
+### Tips
+		- datetime
+			- 8 bytes
+ 			    `1000-01-01 00:00:00 ~ 9999-12-31 23:59:59`
+			- store a absolute value
+		- timestamp
+			- 4 bytes `1970-01-01 08:00:01 ~ 2038-01-19 11:14:07`
+			- will impacted by time zone
+			- index by timestamp will be faster than datetime due to 4 bytes.
+
+## Postgres
+
+### Commands
+
+```bash
+\list # list databases
+\c one # switch to datebase one
+\dt list all the tables
+```
+
+### Create a table
+
+```sql
+CREATE TABLE pm25
+(
+	id SERIAL,  -- id with auto increment
+	time TIMESTAMP not NULL,
+	pm25 integer DEFAULT 0,
+	concentration integer DEFAULT 0,
+	city VARCHAR not NULL
+);
+
+```
+### Import data from CSV
+```sql
+	COPY persons(first_name,last_name,dob,email)
+	FROM 'C:\tmp\persons.csv' DELIMITER ',' CSV HEADER;
+```
+### Import data with SQL
+```bash
+psql -U postgres -W -p 5432 -d somename -h 127.0.0.1 -f  ~/somefile.sql
+```
+
+### Create user
+```sql
+CREATE DATABASE yourdbname;
+CREATE USER youruser WITH ENCRYPTED PASSWORD 'yourpass';
+GRANT ALL PRIVILEGES ON DATABASE yourdbname TO youruser;
+```
+
+### Update
+
+```sql
+UPDATE badges set type = 'onetype' where label != ''
+```
+
+### Queries
+
+#### Query with json field
+
+```sql
+select count(*), arguments ->> 'SomeProp', arguments -> 'FirstLevel' ->> 'SecondLevel'  from jobs group by arguments ->> 'SomeProp', arguments -> 'FirstLevel' ->> 'SecondLevel'
+```
+### Group by
+
+```sql
+select job_ids from (
+	select count(*) as c, string_agg(cast(job_id as varchar),',') as job_ids, args ->> 'ResourceID' args -> 'Measurement' ->> 'Property'
+	from  jobs
+	group by args ->> 'ResourceID',  args -> 'Measurement' ->> 'Property'
+) as f1 where c > 1;
+```
+#### Group by time
+
+To get data entries grouped by time (minute/hour/day/month)
+
+```sql
+select
+  date_trunc('day', created_at), -- or hour, day, week, month, year
+  count(1)
+from view_records
+group by 1
+```
+
+If you want to group time base on a specific timezone and the timestamp field you choose contains that information.
+
+```sql
+select
+  date_trunc('day', created_at AT TIME ZONE 'Australia/Melbourne'), -- or hour, day, week, month, year
+  count(1)
+from view_records
+group by 1
+```
+
+If the time field doesn't have timezone information, we will need to ask postgres to intercept the time to the timezone we would like to use
+
+```sql
+select
+  date_trunc('day', created_at AT TIME ZONE 'Australia/Melbourne') AT TIME ZONE 'China/Beijing')
+  count(1)
+from view_records
+group by 1
+```
+
+### Postgres trigger example
+
+Giving we have a table called users, and the columns are:
+```bash
+id name age
+1  tom  10
+2  Dave  17
+```
+there is a requirement which is when some user's age turn to `18` move that person to `adults` table.
+The adults table have the same fields with `users`.
+
+The potential trigger implementation can be
+
+```sql
+
+CREATE OR REPLACE FUNCTION to_adults ()
+	RETURNS TRIGGER
+	AS $$
+BEGIN
+	INSERT INTO adults VALUES (new.*);
+	DELETE FROM users WHERE age >= 18;
+	RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+
+CREATE TRIGGER to_adults_trigger
+	AFTER UPDATE ON users
+	FOR EACH ROW
+	WHEN (NEW.age >= 18)
+	EXECUTE PROCEDURE to_adults();
+```
+
+
+
+## SQL join diagram
+
+![sql join diagram](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/FC67BE77-F837-4207-B3C4-45205F7C9C40.png)
+
+# Message Queue
+
+# Rabbitmq
+
+# activeMQ
+
+# Redis
@@ -0,0 +1,353 @@
+---
+title: Devops
+created: 2020-02-17
+updated: 2020-02-17
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops.md
+---
+
+## CI
+
+### Platforms
+- Jenkins
+- Gitlab CI
+- Travis
+- Buildkite
+
+### Scaleable Jenkins architecture
+
+![declarative jobs](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/declareative-config.png)
+![scaleable jenkins](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/scaleable-jenkins.png)
+
+
+- Use external storage plugin to host artifacts
+- Store job/pipeline configurations in code repository
+- Use Docker container for each stage, which means to need to install various dependencies on agents
+- Scale up/down(On-demand) agents when needed to save costs.
+
+### A simple jenkins job configuration file
+
+```
+pipeline {
+    agent none
+    stages {
+        stage('Back-end') {
+            agent {
+                docker { image 'maven:3-alpine' }
+            }
+            steps {
+                sh 'mvn --version'
+            }
+        }
+        stage('Front-end') {
+            agent {
+                docker { image 'node:7-alpine' }
+            }
+            steps {
+                sh 'node --version'
+            }
+        }
+    }
+}
+```
+
+
+### A more comprehensive jenkins pipeline
+
+From <https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes/blob/master/sample-app/Jenkinsfile>
+
+```
+pipeline {
+
+  environment {
+    PROJECT = "REPLACE_WITH_YOUR_PROJECT_ID"
+    APP_NAME = "gceme"
+    FE_SVC_NAME = "${APP_NAME}-frontend"
+    CLUSTER = "jenkins-cd"
+    CLUSTER_ZONE = "us-east1-d"
+    IMAGE_TAG = "gcr.io/${PROJECT}/${APP_NAME}:${env.BRANCH_NAME}.${env.BUILD_NUMBER}"
+    JENKINS_CRED = "${PROJECT}"
+  }
+
+  agent {
+    kubernetes {
+      label 'sample-app'
+      defaultContainer 'jnlp'
+      yaml """
+apiVersion: v1
+kind: Pod
+metadata:
+labels:
+  component: ci
+spec:
+  # Use service account that can deploy to all namespaces
+  serviceAccountName: cd-jenkins
+  containers:
+  - name: golang
+    image: golang:1.10
+    command:
+    - cat
+    tty: true
+  - name: gcloud
+    image: gcr.io/cloud-builders/gcloud
+    command:
+    - cat
+    tty: true
+  - name: kubectl
+    image: gcr.io/cloud-builders/kubectl
+    command:
+    - cat
+    tty: true
+"""
+}
+  }
+  stages {
+    stage('Test') {
+      steps {
+        container('golang') {
+          sh """
+            ln -s `pwd` /go/src/sample-app
+            cd /go/src/sample-app
+            go test
+          """
+        }
+      }
+    }
+    stage('Build and push image with Container Builder') {
+      steps {
+        container('gcloud') {
+          sh "PYTHONUNBUFFERED=1 gcloud builds submit -t ${IMAGE_TAG} ."
+        }
+      }
+    }
+    stage('Deploy Canary') {
+      // Canary branch
+      when { branch 'canary' }
+      steps {
+        container('kubectl') {
+          // Change deployed image in canary to the one we just built
+          sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${IMAGE_TAG}#' ./k8s/canary/*.yaml")
+          step([$class: 'KubernetesEngineBuilder',namespace:'production', projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/services', credentialsId: env.JENKINS_CRED, verifyDeployments: false])
+          step([$class: 'KubernetesEngineBuilder',namespace:'production', projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/canary', credentialsId: env.JENKINS_CRED, verifyDeployments: true])
+          sh("echo http://`kubectl --namespace=production get service/${FE_SVC_NAME} -o jsonpath='{.status.loadBalancer.ingress[0].ip}'` > ${FE_SVC_NAME}")
+        }
+      }
+    }
+    stage('Deploy Production') {
+      // Production branch
+      when { branch 'master' }
+      steps{
+        container('kubectl') {
+        // Change deployed image in canary to the one we just built
+          sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${IMAGE_TAG}#' ./k8s/production/*.yaml")
+          step([$class: 'KubernetesEngineBuilder',namespace:'production', projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/services', credentialsId: env.JENKINS_CRED, verifyDeployments: false])
+          step([$class: 'KubernetesEngineBuilder',namespace:'production', projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/production', credentialsId: env.JENKINS_CRED, verifyDeployments: true])
+          sh("echo http://`kubectl --namespace=production get service/${FE_SVC_NAME} -o jsonpath='{.status.loadBalancer.ingress[0].ip}'` > ${FE_SVC_NAME}")
+        }
+      }
+    }
+    stage('Deploy Dev') {
+      // Developer Branches
+      when {
+        not { branch 'master' }
+        not { branch 'canary' }
+      }
+      steps {
+        container('kubectl') {
+          // Create namespace if it doesn't exist
+          sh("kubectl get ns ${env.BRANCH_NAME} || kubectl create ns ${env.BRANCH_NAME}")
+          // Don't use public load balancing for development branches
+          sh("sed -i.bak 's#LoadBalancer#ClusterIP#' ./k8s/services/frontend.yaml")
+          sh("sed -i.bak 's#gcr.io/cloud-solutions-images/gceme:1.0.0#${IMAGE_TAG}#' ./k8s/dev/*.yaml")
+          step([$class: 'KubernetesEngineBuilder',namespace: "${env.BRANCH_NAME}", projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/services', credentialsId: env.JENKINS_CRED, verifyDeployments: false])
+          step([$class: 'KubernetesEngineBuilder',namespace: "${env.BRANCH_NAME}", projectId: env.PROJECT, clusterName: env.CLUSTER, zone: env.CLUSTER_ZONE, manifestPattern: 'k8s/dev', credentialsId: env.JENKINS_CRED, verifyDeployments: true])
+          echo 'To access your environment run `kubectl proxy`'
+          echo "Then access your service via http://localhost:8001/api/v1/proxy/namespaces/${env.BRANCH_NAME}/services/${FE_SVC_NAME}:80/"
+        }
+      }
+    }
+  }
+}
+```
+
+## CD
+
+### Tools Comparision
+
+![OSI Models](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/iac.png)
+
+### An example CI/CD flow
+
+```mermaid
+graph TB
+
+A(("Code commit")) -->B1["CI job"]
+B1 -->C["New Docker Image"]
+C --Update Kubernetes template--> D[New Kubernetes manifest Yaml]
+D --Git commit-->E[Deployment Repo]
+E --Self trigger--> F((Deploy to K8s))
+F --Manual/Auto trigger--> G[Update manifest]
+G --> H((Deploy to Next env))
+```
+
+### Spinnaker
+
+Spinnaker is an open-source, multi-cloud continuous delivery platform that helps you release software changes with high velocity and confidence. Spinnaker provides two core sets of features:
+
+	application management
+
+	application deployment
+
+## Everything is code
+
+- Infrastructure as code
+- CI config as code
+- Dockerfile
+
+## Automation tools
+
+### Capistrano
+A ruby tool for deploy applications
+### Chef
+
+#### What is chef
+
+Chef enables you to manage and scale cloud infrastructure with no downtime or interruptions. Freely move applications and configurations from one cloud to another. Chef is integrated with all major cloud providers including Amazon EC2, VMWare, IBM Smartcloud, Rackspace, OpenStack, Windows Azure, HP Cloud, Google Compute Engine, Joyent Cloud and others.
+
+Chef running a `chef-client` on each node to be a runner to install applications on remote server. The client can use both `pull` and `push` modes to to communication.
+
+More:
+
+- Chef vs ansible: <https://www.chef.io/ansible/>
+
+### Ansible
+
+#### What is Ansible
+Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates. Ansible’s goals are foremost those of simplicity and maximum ease of use.
+
+Ansible uses `SSH` to execute commands on remote server and use `RabbitMQ` at the transport level by `push` updates
+
+### Puppet
+
+Is also a configuration management tool for servers and designed to install and manage software on existing servers.
+
+### Packer
+
+Packer is a tool to allows you build a machine image which contains all preinstalled and configged applications.
+
+### Terraform
+
+#### What is Terraform
+Terraform use a declarative language to describe the current state of infrastructure, and there is no server side agent installed.
+
+Terraform is more about declare the resources/services requirements, rather than install softwares and libraries on remote server.
+
+
+#### Basic steps
+```bash
+# go to the folder which stores the environment credentials
+terraform init # Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
+terraform get # Downloads and installs modules that defined by configuration files
+terraform plan # Kind of try run, you will what kind of changes you will have
+terraform apply # Apply changes to cluster
+```
+
+## HA(High Availability)
+- LB - haproxy
+- Keepalived -- vrrp
+	- Nginx + Keepalived
+- Heartbeat
+
+## Network
+- IP
+	- Private IP Addresses
+		- 10.0.0.0 – 10.255.255.255
+		- 172.16.0.0 – 172.31.255.255
+		- 192.168.0.0 – 192.168.255.255
+
+### OSI Model( 7 layers network)
+
+![OSI Models](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/osi-model.png)
+
+### CIDR blocks
+
+[Wikipedia](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing)
+
+|**Address**|**Difference**|**Mask**|**Addresses**|**2n**|**Relative**|**Restrictions**|**Typical use**|
+| :----:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|
+|a.b.c.d/32|+0.0.0.0|255.255.255.255|1|20|1⁄256 C| |Host route
+|a.b.c.d/31|+0.0.0.1|255.255.255.254|2|21|1⁄128 C|d = 0 ... (2n) ... 254|Point to point links (RFC 3021)
+|a.b.c.d/30|+0.0.0.3|255.255.255.252|4|22|1⁄64 C|d = 0 ... (4n) ... 252|Point to point links (glue network)
+|a.b.c.d/29|+0.0.0.7|255.255.255.248|8|23|1⁄32 C|d = 0 ... (8n) ... 248|Smallest multi-host network
+|a.b.c.d/28|+0.0.0.15|255.255.255.240|16|24|1⁄16 C|d = 0 ... (16n) ... 240|Small LAN
+|a.b.c.d/27|+0.0.0.31|255.255.255.224|32|25|⅛ C|d = 0 ... (32n) ... 224|
+|a.b.c.d/26|+0.0.0.63|255.255.255.192|64|26|¼ C|d = 0, 64, 128, 192|
+|a.b.c.d/25|+0.0.0.127|255.255.255.128|128|27|½ C|d = 0, 128|Large LAN
+|a.b.c.0/24|+0.0.0.255|255.255.255.0|256|28|1 C| |
+|a.b.c.0/23|+0.0.1.255|255.255.254.0|512|29|2 C|c = 0 ... (2n) ... 254|
+|a.b.c.0/22|+0.0.3.255|255.255.252.0|1,024|210|4 C|c = 0 ... (4n) ... 252|Small business
+|a.b.c.0/21|+0.0.7.255|255.255.248.0|2,048|211|8 C|c = 0 ... (8n) ... 248|Small ISP/ large business
+|a.b.c.0/20|+0.0.15.255|255.255.240.0|4,096|212|16 C|c = 0 ... (16n) ... 240|
+|a.b.c.0/19|+0.0.31.255|255.255.224.0|8,192|213|32 C|c = 0 ... (32n) ... 224|ISP/ large business
+|a.b.c.0/18|+0.0.63.255|255.255.192.0|16,384|214|64 C|c = 0, 64, 128, 192|
+|a.b.c.0/17|+0.0.127.255|255.255.128.0|32,768|215|128 C|c = 0, 128|
+|a.b.0.0/16|+0.0.255.255|255.255.0.0|65,536|216|256 C = B| |
+|a.b.0.0/15|+0.1.255.255|255.254.0.0|131,072|217|2 B|b = 0 ... (2n) ... 254|
+|a.b.0.0/14|+0.3.255.255|255.252.0.0|262,144|218|4 B|b = 0 ... (4n) ... 252|
+|a.b.0.0/13|+0.7.255.255|255.248.0.0|524,288|219|8 B|b = 0 ... (8n) ... 248|
+|a.b.0.0/12|+0.15.255.255|255.240.0.0|1,048,576|220|16 B|b = 0 ... (16n) ... 240|
+|a.b.0.0/11|+0.31.255.255|255.224.0.0|2,097,152|221|32 B|b = 0 ... (32n) ... 224|
+|a.b.0.0/10|+0.63.255.255|255.192.0.0|4,194,304|222|64 B|b = 0, 64, 128, 192|
+|a.b.0.0/9|+0.127.255.255|255.128.0.0|8,388,608|223|128 B|b = 0, 128|
+|a.0.0.0/8|+0.255.255.255|255.0.0.0|16,777,216|224|256 B = A| |Largest IANA block allocation
+|a.0.0.0/7|+1.255.255.255|254.0.0.0|33,554,432|225|2:00 am|a = 0 ... (2n) ... 254|
+|a.0.0.0/6|+3.255.255.255|252.0.0.0|67,108,864|226|4:00 am|a = 0 ... (4n) ... 252|
+|a.0.0.0/5|+7.255.255.255|248.0.0.0|134,217,728|227|8:00 am|a = 0 ... (8n) ... 248|
+|a.0.0.0/4|+15.255.255.255|240.0.0.0|268,435,456|228|16 A|a = 0 ... (16n) ... 240|
+|a.0.0.0/3|+31.255.255.255|224.0.0.0|536,870,912|229|32 A|a = 0 ... (32n) ... 224|
+|a.0.0.0/2|+63.255.255.255|192.0.0.0|1,073,741,824|230|64 A|a = 0, 64, 128, 192|
+|a.0.0.0/1|+127.255.255.255|128.0.0.0|2,147,483,648|231|128 A|a = 0, 128|
+|0.0.0.0/0|+255.255.255.255|0.0.0.0|4,294,967,296|232|256 A| |
+
+## tools
+
+### git
+	- cherrypick
+	- rebase
+	- commit
+	- checkout
+		- checkout -b br com
+	- push
+		- --tags
+			- push tags
+		- —all
+			- push all branchs
+		- -f
+			- force push
+	- branch -d xx
+		- delete a local branch
+	- remote
+		- show xxx
+			- show info of a remote branch
+		- add xxx
+			- add a remote repo
+	- branch
+		- list all the branchs
+	- mv Rename/move files and folder with git history records
+
+      e.g. git mv -k ./a/b ./c/d   option `k` would avoid the "can not move directory into itself" error.
+### oh-my-zsh
+
+### IDE/tools
+
+- Intellij
+- vscode
+
+
+## Security
+
+
+### Mutual TLS authentication
+
+![CTLS](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/ctls.png)
@@ -0,0 +1,21 @@
+---
+title: Aws
+created: 2020-01-06
+updated: 2020-01-06
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/aws.md
+---
+
+## EC2
+Virtual machines
+
+## ECS
+
+Running and managing containers without worry about infrastructure
+
+## Cloudfront
+
+## S3
+
+## RDS
@@ -0,0 +1,219 @@
+---
+title: Docker
+created: 2021-02-08
+updated: 2021-02-08
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/docker.md
+---
+
+# Docker
+
+Easy to use, just drag and drop things.
+
+## Benefits of using Docker
+
+- Escape the app dependency matrix
+- Solve the environment issue. it works on my machine.
+- Very low resource allocation cost when compare with Virtual Machine. So on the same machine, you can run more containers and spend much less time.
+
+## How Docker works
+
+Very useful blog talking about how docker works internally: [Understanding the Docker Internals](https://medium.com/@nagarwal/understanding-the-docker-internals-7ccb052ce9fe)
+
+![Docker](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/docker.png)
+
+Docker use
+
+* Namespaces
+* Cgroups
+* Network
+* Union File Systems
+
+to isolate all kinds of resources.
+
+### Namespaces
+
+Docker makes use of kernel namespaces to provide the isolated workspace called the container. When you run a container, Docker creates a set of namespaces for that container. These namespaces provide a layer of isolation. Each aspect of a container runs in a separate namespace and its access is limited to that namespace.
+Docker Engine uses the following namespaces on Linux:
+
+- `PID` namespace for process isolation.
+- `NET` namespace for managing network interfaces.
+- `IPC` namespace for managing access to IPC resources.
+- `MNT` namespace for managing filesystem mount points.
+- `UTS` namespace for isolating kernel and version identifiers.
+- `USER` namespaces is a feature of Linux that can be used to separate the user IDs and group IDs between the host and containers. It can provide a better isolation and security: the privileged user root in the container can be mapped to a non-privileged user on the host.
+
+#### How to see a docker container's namespaces
+
+```bash
+> docker inspect 195de1a4b33c | grep Pid
+
+	"Pid": 1496,
+	"PidMode": "",
+	"PidsLimit": null,
+
+> cd /proc/1496
+> ls
+
+attr       auxv    clear_refs  comm             cpuset  environ  fd      gid_map  limits    map_files  mem        mounts      net  numa_maps  oom_score      pagemap      personality  root   schedstat  setgroups  smaps_rollup  stat   status   task    timerslack_ns  wchan
+autogroup  cgroup  cmdline     coredump_filter  cwd     exe      fdinfo  io       loginuid  maps       mountinfo  mountstats  ns   oom_adj    oom_score_adj  patch_state  projid_map   sched  sessionid  smaps      stack         statm  syscall  timers  uid_map
+
+> cd ns
+> ls
+
+cgroup  ipc  mnt  net  pid  pid_for_children  user  uts
+
+```
+
+### cgroups
+
+![cgroups](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/cgroups.png)
+
+Control Groups are a Linux feature for organizing processes in hierarchical groups and applying resources limits to them. Docker also makes use of kernel control groups for resource allocation and isolation. A cgroup limits an application to a specific set of resources. Control groups allow Docker Engine to share available hardware resources to containers and optionally enforce limits and constraints.
+Docker Engine uses the following cgroups:
+
+- `Memory` cgroup for managing accounting, limits and notifications.
+- `HugeTBL` cgroup for accounting usage of huge pages by process group.
+- `CPU` cgroup for managing user / system CPU time and usage.
+- `CPUSet` cgroup for binding a group to specific CPU. Useful for real time applications and NUMA systems with localized memory per CPU.
+- `BlkIO` cgroup for measuring & limiting amount of blckIO by group.
+- `net_cls` and net_prio cgroup for tagging the traffic control.
+- `Devices` cgroup for reading / writing access devices.
+- `Freezer` cgroup for freezing a group. Useful for cluster batch scheduling, process migration and debugging without affecting prtrace.
+
+### Union File Systems
+
+![filesystem](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/docker-filesystem.png)
+
+
+Union file systems operate by creating layers, making them very lightweight and fast. Docker Engine uses UnionFS to provide the building blocks for containers. Docker Engine can use multiple UnionFS variants, including AUFS, btrfs, vfs, and devicemapper.
+
+### Container Format
+
+Docker Engine combines the namespaces, control groups and UnionFS into a wrapper called a container format. The default container format is libcontainer.
+### Security
+
+Docker Engine makes use of `AppArmor`, `Seccomp`, `Capabilities` kernel features for security purposes.
+
+- `AppArmor` allows to restrict programs capabilities with per-program profiles.
+- `Seccomp` used for filtering syscalls issued by a program.
+- `Capabilties` for performing permission checks.
+
+
+### Docker VS VM
+
+![Docker vs VM](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/container.png)
+
+## Network
+
+- Bridge
+- Host
+
+## CMD
+
+- has three forms
+- CMD ["executable","param1","param2"]
+	- exec form, preferred
+- CMD ["param1","param2"]
+	- as default parameters to ENTRYPOINT
+- CMD command param1 param2
+	- shell form
+- The main purpose of a CMD is to provide defaults for an executing container. These defaults can include an executable, or they can omit the executable, in which case you must specify an ENTRYPOINT instruction as well.
+- If CMD is used to provide default arguments for the ENTRYPOINT instruction, both the CMD and ENTRYPOINT instructions should be specified with the JSON array format.
+- docker run could override the default specified in CMD.
+
+## ENTRYPOINT
+
+- docker run arguments will as the entrypoint’s parameters. not CMD’s override
+- has two forms
+- ENTRYPOINT ["executable", "param1", "param2"]
+	- exec form, preferred
+- ENTRYPOINT command param1 param2
+	- shell form
+- entrypoint setting can be override by docker run —entrypoint
+
+## Security
+	- https://docs.docker.com/engine/security/security/
+	- use non root user
+	- docker daemon attack surface
+	- kernel security features
+
+## Tips
+
+### difference between `cmd` and `entrypoint`
+
+	`cmd` can be override but entrypoint can't.
+
+### difference between `copy` and `add`
+
+	Basically they are similar, copy files and folder to destination. While ADD has some features (like local-only tar extraction and remote URL support) that are not immediately obvious. Consequently, the best use for ADD is local tar file auto-extraction into the image, as in ADD rootfs.tar.xz /.
+
+### Reduce docker image size
+
+- Use the small base docker image, e.g. `alpine`, `jessie` etc.
+- Change docker image to multi stage, only copy the runtime files and dependencies to the final stage
+
+A example of node js one
+
+**Before**
+
+```Dockerfile
+FROM node:alpine as build	FROM node as build
+
+WORKDIR /app
+
+# copy the app, note .dockerignore	# copy the app, note .dockerignore
+COPY . /app
+
+RUN apk update \
+  && apk add --no-cache git python alpine-sdk\
+  && npm ci \
+  && npm run build \
+  && npm run generate
+
+EXPOSE 3000
+
+CMD [ "npm", "start" ]
+
+```
+**After**
+
+```Dockerfile
+
+FROM node as build
+
+WORKDIR /app
+
+# copy the app, note .dockerignore
+COPY . /app
+
+RUN npm ci \
+  && npm run build \
+  && npm run generate \
+  && npm prune --production \
+  && curl -sf https://gobinaries.com/tj/node-prune | sh \
+  && node-prune
+
+FROM node:alpine
+
+WORKDIR /app
+
+# copy from build image
+COPY --from=build /app ./
+
+EXPOSE 3000
+
+CMD [ "npm", "start" ]
+```
+
+### Inspect image's files
+
+First, just create a container instead of running a container.
+Then export the entire container to a zip file.
+
+```
+docker create --name=some-name image:tag
+docker export tmp_$$ > some-name.zip
+docker rm some-name
+```
@@ -0,0 +1,36 @@
+---
+title: Gcp
+created: 2019-11-27
+updated: 2019-11-27
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/gcp.md
+---
+
+# Google Compute Engine
+
+# Kubernetes Engine
+
+# Cloud Storage
+
+# Cloud SQL
+
+# IAM
+
+# Stackdriver
+
+## Logging
+
+## Log based metrics
+
+## Debug
+
+## Trace
+
+# App Engine
+
+# Cloud Functions
+
+# Cloud Run
+
+Run Containers without caring about infrastructure
@@ -0,0 +1,49 @@
+---
+title: Incident
+created: 2018-03-27
+updated: 2018-03-27
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/incident.md
+---
+
+# Incident Management
+
+## Incident dealing
+
+It's important for a company to have a formal process about how do people dealing with those scenarios. There should some places to list all the common issues and its' solutions.
+During the process, we should records the steps we have, so we can learn from this after the incident.
+Company should have a on call rotation rules to have every engineers on the list and well rotated.
+
+### Some tools
+
+* Pagerduty - on call scheduling and incident resolution
+* Datadog - metrics
+* Stackdriver - logging
+
+## Incident Training
+
+### The purpose for doing this
+
+* It's better to have some training before people get on call
+* We can learn something from the incident training, so we can find some potential issues in production
+* It's a good opportunities to have people from different teams work together to solve the issue
+* Customer first. We should let customer know what happened if the issue has some impact to the customers.
+
+### How to organise an incident training
+
+* Have some check lists for people prepared before training
+* Have a almost production like environment to do this
+* Organiser triggers a issue, then ask trainees to fix the issue and investigate the root clause
+* Let customers knows what happened. The other way round, the customer can give us some more information and also give us some complaint about his loss due to the incident.
+* The on-call team should solve the issue as well as write down some notes to record how to deal with this.
+* Escalate the severity if necessary to involve more people to help to solve this
+* Give enough information to the next people when you are going to hand over it to.
+
+## After incident
+
+* Run a post incident review
+* Write down the incident events timeline
+* Provide a problem statement
+* List the root cause
+* List the ways of how do we prevent this happen again
@@ -0,0 +1,792 @@
+---
+title: Kubernetes
+created: 2020-12-09
+updated: 2020-12-09
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/kubernetes.md
+---
+
+# Kubernetes
+
+## Terminology
+
+### Pod
+
+- Single container
+- Multiple containers
+    - share ip
+    - share volumes
+
+#### Information you get when list pods
+
+```bash
+➜ k get pods
+NAME                                    READY   STATUS      RESTARTS   AGE
+cloud-sql-proxy-66d-71222342354s        1/1     Running     0          42d
+```
+
+### Node
+
+Basically is a Kubernetes client or agent which runs a bunch of pods.
+
+### Service
+
+- ClusterIP (default)
+- NodePort （ access by port）
+- LoadBalancer (external ip)
+- ExternalName dns
+
+### Deployment
+
+A Deployment provides declarative updates for Pods and ReplicaSets.
+
+You describe a desired state in a Deployment, and the Deployment Controller changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
+
+### Replica Set
+
+A ReplicaSet’s purpose is to maintain a stable set of replica Pods running at any given time. As such, it is often used to guarantee the availability of a specified number of identical Pods.
+
+### Service rolling update
+
+### DaemonSet
+
+Running that pod on every node.
+
+- running a cluster storage daemon, such as `glusterd`, `ceph`, on each node
+- running a logs collection daemon on every node, such as `fluentd` or logstash.
+- running a node monitoring daemon on every node, such as Prometheus Node Exporter, collectd, New Relic agent.
+
+### StatefulSet
+
+![statefulset](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/statefulset.png)
+
+StatefulSets represent a set of pods with unique, persistent identities and stable hostnames. The state information and other resilient data for any given StatefulSet Pod is maintained in persistent disk storage associated with the StatefulSet.
+
+StatefulSets are designed to deploy stateful applications and clustered applications that save data to persistent storage, such as Google Compute Engine persistent disks. StatefulSets are suitable for deploying Kafka, MySQL, Redis, ZooKeeper, and other applications needing unique, persistent identities and stable hostnames. For stateless applications, use Deployments.
+
+#### Key features
+
+  - Ordered, graceful deployment and scaling
+  - Stable, persistent storage.
+  - Stable, unique network identifiers.
+  - Ordered, graceful deletion and termination.
+
+#### An example
+
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+  name: nginx
+  labels:
+    app: nginx
+spec:
+  ports:
+  - port: 80
+    name: web
+  clusterIP: None
+  selector:
+    app: nginx
+---
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: web
+spec:
+  selector:
+    matchLabels:
+      app: nginx # Label selector that determines which Pods belong to the StatefulSet
+                 # Must match spec: template: metadata: labels
+  serviceName: "nginx"
+  replicas: 3
+  template:
+    metadata:
+      labels:
+        app: nginx # Pod template's label selector
+    spec:
+      terminationGracePeriodSeconds: 10
+      containers:
+      - name: nginx
+        image: gcr.io/google_containers/nginx-slim:0.8
+        ports:
+        - containerPort: 80
+          name: web
+        volumeMounts:
+        - name: www
+          mountPath: /usr/share/nginx/html
+  volumeClaimTemplates:
+  - metadata:
+      name: www
+    spec:
+      accessModes: [ "ReadWriteOnce" ]
+      resources:
+        requests:
+          storage: 1Gi
+```
+
+#### Update Statefulset
+
+To decide how to handle updates, StatefulSets use a update strategy defined in spec: updateStrategy. There are two strategies, OnDelete and RollingUpdate:
+
+- `OnDelete` does not automatically delete and recreate Pods when the object's configuration is changed. Instead, you must manually delete the old Pods to cause the controller to create updated Pods.
+
+- `RollingUpdate` automatically deletes and recreates Pods when the object's configuration is changed. New Pods must be in Running and Ready states before their predecessors are deleted. With this strategy, changing the Pod specification automatically triggers a rollout. This is the default update strategy for StatefulSets.
+
+StatefulSets update Pods in reverse ordinal order. You can monitor update rollouts by running the following command:
+
+```bash
+kubectl rollout status statefulset [STATEFULSET_NAME]
+```
+#### Partitioning rolling updates
+
+When you partition an update, all Pods with an ordinal greater than or equal to the partition value are updated when you update the StatefulSet’s Pod specification. Pods with an ordinal less than the partition value are not updated and, even if they are deleted, are recreated using the previous version of the specification. If the partition value is greater than the number of replicas, the updates are not propagated to the Pods.
+
+### Proxy
+
+Creates a proxy server or application-level gateway between localhost and the Kubernetes API Server. It also allows
+serving static content over specified HTTP path. All incoming data enters through one port and gets forwarded to the
+remote kubernetes API Server port, except for the path matching the static content path.
+
+### DNS
+
+  - service.namespace
+
+### Secrets
+
+For storing keys, credentials and certificates. For instance: database token, 3rd party API keys.
+
+### Persistent volumes
+
+- Available
+    - a free resource that is not yet bound to a claim
+- Bound
+    -  the volume is bound to a claim
+- Released
+    - the claim has been deleted, but the resource is not yet reclaimed by the cluster
+- Failed
+    - the volume has failed its automatic reclamation
+
+### kubelet
+
+The `kubelet` is the primary “node agent” that runs on each node. It can register the node with the apiserver using one of: the hostname; a flag to override the hostname; or specific logic for a cloud provider. The `kubelet` works in terms of a PodSpec
+
+### CRD(CustomResourceDefinitions)
+
+The CustomResourceDefinition API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.
+
+| RDs | Aggregated API|
+|---|---|
+| Do not require programming. Users can choose any language for a CRD controller. |	Requires programming in Go and building binary and image. Users can choose any language for a CRD controller.|
+| No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail.|
+| No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. |	May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated APIserver. |
+| No need to handle multiple versions of your API. For example: when you control the client for this resource, you can upgrade it in sync with the API. |	You need to handle multiple versions of your API, for example: when developing an extension to share with the world. |
+
+
+### Liveness vs Readiness
+
+#### Liveness
+
+The `kubelet` uses liveness probes to know when to restart a Container. For example, liveness probes could catch a deadlock, where an application is running, but unable to make progress. Restarting a Container in such a state can help to make the application more available despite bugs.
+
+The `kubelet` uses readiness probes to know when a Container is ready to start accepting traffic. A Pod is considered ready when all of its Containers are ready. One use of this signal is to control which Pods are used as backends for Services. When a Pod is not ready, it is removed from Service load balancers.
+
+The kubelet uses startup probes to know when a Container application has started. If such a probe is configured, it disables liveness and readiness checks until it succeeds, making sure those probes don’t interfere with the application startup. This can be used to adopt liveness checks on slow starting containers, avoiding them getting killed by the kubelet before they are up and running.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: goproxy
+  labels:
+    app: goproxy
+spec:
+  containers:
+  - name: goproxy
+    image: k8s.gcr.io/goproxy:0.1
+    ports:
+    - containerPort: 8080
+    readinessProbe:
+      tcpSocket:
+        port: 8080
+      initialDelaySeconds: 5
+      periodSeconds: 10
+    livenessProbe:
+      tcpSocket:
+        port: 8080
+      initialDelaySeconds: 15
+      periodSeconds: 20
+```
+
+#### Readiness
+
+Sometimes, applications are temporarily unable to serve traffic. For example, an application might need to load large data or configuration files during startup, or depend on external services after startup. In such cases, you don’t want to kill the application, but you don’t want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. A pod with containers reporting that they are not ready does not receive traffic through Kubernetes Services.
+
+```yaml
+readinessProbe:
+  exec:
+    command:
+    - cat
+    - /tmp/healthy
+  initialDelaySeconds: 5
+  periodSeconds: 5
+```
+
+### Taints and Tolerations
+
+Node affinity, described here, is a property of pods that attracts them to a set of nodes (either as a preference or a hard requirement). Taints are the opposite -- they allow a node to repel a set of pods.
+
+Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes.
+
+```bash
+kubectl taint nodes node1 key=value:NoSchedule
+```
+By default, no pod will be deploy to the node `node1`
+
+You specify a toleration for a pod in the PodSpec. Both of the following tolerations “match” the taint created by the kubectl taint line above, and thus a pod with either toleration would be able to schedule onto node1:
+
+```yaml
+tolerations:
+- key: "key"
+  operator: "Equal"
+  value: "value"
+  effect: "NoSchedule"
+```
+
+```yaml
+tolerations:
+- key: "key"
+  operator: "Exists"
+  effect: "NoSchedule"
+```
+
+## Several ways to access a service inside Kubernetes cluster
+
+* ClusterIP, Can only be accessed inside the cluster. You can access inside `kubectl proxy`
+* NodePort, You can access the service by a specific public port between `30000–32767`, if the cluster server IP changes, then your service' endpoint url changes.
+* LoadBalancer, it's kind of external service, all the requests go through the loadbalancer. Every service needs a new IP for this.
+* Ingress, can be shared among multiple services, you can have different types of ingress controllers: `gec`, `nginx`, `contour`, `istio` and so on.
+
+Links:
+* [Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?](https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0)
+
+## Kubernetes components
+
+A Kubernetes cluster consists of a set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node.
+
+The worker node(s) host the pods that are the components of the application. The Control Plane manages the worker nodes and the pods in the cluster. In production environments, the Control Plane usually runs across multiple computers and a cluster usually runs multiple nodes, providing fault-tolerance and high availability.
+
+![Components of kubernetes](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/components-of-kubernetes.png)
+
+### Components
+
+#### Control Plane Components
+
+- `kube-apiserver` The API server is a component of the Kubernetes control plane that exposes the Kubernetes API. The API server is the front end for the Kubernetes control plane
+- `etcd` Consistent and highly-available key value store used as Kubernetes’ backing store for all cluster data.
+- `kube-scheduler` Control Plane component that watches for newly created pods with no assigned node, and selects a node for them to run on
+- `kube-controller-manager` Control Plane component that runs controller processes. These controllers include:
+
+  * Node Controller: Responsible for noticing and responding when nodes go down.
+  * Replication Controller: Responsible for maintaining the correct number of pods for every replication controller object in the system.
+  * Endpoints Controller: Populates the Endpoints object (that is, joins Services & Pods).
+  * Service Account & Token Controllers: Create default accounts and API access tokens for new namespaces.
+- `cloud-controller-manager` runs controllers that interact with the underlying cloud providers
+
+#### Node Components
+
+Node components run on every node, maintaining running pods and providing the Kubernetes runtime environment.
+
+- `kubelet` An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.
+- `kube-proxy` kube-proxy is a network proxy that runs on each node in your cluster, implementing part of the Kubernetes Service concept. kube-proxy maintains network rules on nodes. These network rules allow network communication to your Pods from network sessions inside or outside of your cluster.
+kube-proxy uses the operating system packet filtering layer if there is one and it’s available. Otherwise, kube-proxy forwards the traffic itself.
+- `Container Runtime` The container runtime is the software that is responsible for running containers.
+
+#### Addons
+
+Addons use Kubernetes resources (DaemonSet, Deployment, etc) to implement cluster features. Because these are providing cluster-level features, namespaced resources for addons belong within the kube-system namespace
+
+- `DNS` While the other addons are not strictly required, all Kubernetes clusters should have cluster DNS, as many examples rely on it. Cluster DNS is a DNS server, in addition to the other DNS server(s) in your environment, which serves DNS records for Kubernetes services
+- `Web UI (Dashboard)` Dashboard is a general purpose, web-based UI for Kubernetes clusters.
+- `Container Resource Monitoring` Container Resource Monitoring records generic time-series metrics about containers in a central database, and provides a UI for browsing that data.
+- `Cluster-level Logging` A cluster-level logging mechanism is responsible for saving container logs to a central log store with search/browsing interface.
+
+## Kubernetes pod lifecycle
+
+Through its lifecycle, a Pod can attain following states:
+
+- `Pending`: The pod is accepted by the Kubernetes system but its container(s) is/are not created yet.
+
+- `Running`: The pod is scheduled on a node and all its containers are created and at-least one container is in Running state.
+
+- `Succeeded`: All container(s) in the Pod have exited with status 0 and will not be restarted.
+
+- `Failed`: All container(s) of the Pod have exited and at least one container has returned a non-zero status.
+
+- `CrashLoopBackoff`: The container fails to start and is tried again and again.
+
+## Creating Highly Available clusters with kubeadm
+
+### With stacked control plane nodes
+
+- `Init first control plane` kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
+-  `Join the rest control planes` kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
+
+### External etcd nodes
+
+Setting up a cluster with external etcd nodes is similar to the procedure used for stacked etcd with the exception that you should setup etcd first, and you should pass the etcd information in the kubeadm config file.
+
+
+## Flow chart of running a pod
+
+![running a pod](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/running-a-pod.png)
+
+Chart from: <https://blog.heptio.com/core-kubernetes-jazz-improv-over-orchestration-a7903ea92ca>
+
+## Kubernetes termination lifecycle
+
+![termination lifecycle](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/terminate-pod.jpg)
+
+Chart from: <https://dzone.com/articles/kubernetes-lifecycle-of-a-pod>
+
+## Zero down time deployment
+
+Set strategy type to `RollingUpdate` instead of `Recreate`
+
+```yaml
+spec:
+  replicas: 1
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxUnavailable: 50%
+      maxSurge: 1
+```
+
+## Some useful commands
+
+```bash
+kubectl config set-context context-name --namespace default-namespace-name #default active context
+kubectl config view #view config
+kubectl get nodes #get nodes
+kubectl get namespaces #get all namespaces
+kubectl get pods #get pods
+kubectl get deployments #get deployments
+kubectl logs -f <pod> --tail 200 # tail logs from some pod
+kubectl get services #get services
+kubectl get ingress #get ingresses
+kubectl get hpa # get horizontal auto scaling policies
+kubectl get all # get all kinds of units
+kubectl get secrets <secret-id> -o yaml #view a secret details with yaml format, fields are encrypted with base64
+# List all Secrets currently in use by a pod
+kubectl get pods -o json | jq '.items[].spec.containers[].env[]?.valueFrom.secretKeyRef.name' | grep -v null | sort | uniq
+
+# List Events sorted by timestamp
+kubectl get events --sort-by=.metadata.creationTimestamp
+
+kubectl run -i --tty busybox --image=busybox -- sh  # Run pod as interactive shell
+kubectl run nginx --image=nginx --restart=Never -n
+mynamespace                                         # Run pod nginx in a specific namespace
+kubectl run nginx --image=nginx --restart=Never     # Run pod nginx and write its spec into a file called pod.yaml
+--dry-run -o yaml > pod.yaml
+
+kubectl attach my-pod -i                            # Attach to Running Container and get the current running process
+
+kubectl top pod POD_NAME --containers               # Show metrics for a given pod and its containers
+
+kubectl config view -o jsonpath='{.users[].name}'    # display the first user
+kubectl config view -o jsonpath='{.users[*].name}'   # get a list of users
+kubectl config get-contexts                          # display list of contexts
+kubectl config current-context                       # display the current-context
+kubectl config use-context my-cluster-name           # set the default context to my-cluster-name
+
+# add a new cluster to your kubeconf that supports basic auth
+kubectl config set-credentials kubeuser/foo.kubernetes.com --username=kubeuser --password=kubepassword
+
+# permanently save the namespace for all subsequent kubectl commands in that context.
+kubectl config set-context --current --namespace=ggckad-s2
+
+# set a context utilizing a specific username and namespace.
+kubectl config set-context gce --user=cluster-admin --namespace=foo \
+  && kubectl config use-context gce
+
+kubectl config unset users.foo                       # delete user foo
+
+kubectl describe quota # Check quota for current namespace
+
+kubectl taint nodes node1 key=value:NoSchedule # add node taint
+kubectl taint nodes node1 key:NoSchedule- # remove a taint from a node
+
+```
+
+### Some explanation
+
+#### kubectl exec with attach
+
+The main difference is in the process you interact with in the container:
+
+`exec`: any one you want to create
+
+`attach`: the one currently running (no choice)
+
+## Kubernetes port forward
+
+`kubectl port-forward` allows using resource name, such as a pod name, to select a matching pod to port forward to Which is perfect for testing the remote service/pods in your local. e.g. Forward your db port to a local port, you can connect to your remote db even it doesn't have a public IP.
+
+```bash
+kubectl port-forward redis-master-765d459796-258hz 7000:6379
+
+kubectl port-forward pods/redis-master-765d459796-258hz 7000:6379
+
+kubectl port-forward deployment/redis-master 7000:6379
+
+kubectl port-forward rs/redis-master 7000:6379
+
+kubectl port-forward svc/redis-master 7000:6379
+```
+
+## Kubernetes Dashboard
+
+### Deploy Dashboard
+
+```bash
+kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
+```
+### Access dashboard
+
+```bash
+# First fetch cluster info, make sure cluster is running properly
+kubectl cluster-info
+
+# Normally the url for dashboard would be
+http://your_ip:8080/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/
+
+```
+
+## Configure Kubectl to access remote Kubernetes cluster
+
+Here is a very simple config which use HTTP and `not secure`, should only for local testing purpose
+
+### Config
+
+Having a yaml file called `config-demo` in your current folder
+
+```yaml
+apiVersion: v1
+clusters:
+- cluster:
+    server: https://your_cluster_ip:5443
+  name: development
+contexts:
+- context:
+    cluster: development
+    namespace: dev
+    user: developer
+  name: dev
+current-context: dev
+kind: Config
+preferences: {}
+users:
+- name: developer
+```
+
+### Connect to remote cluster
+
+```bash
+kubectl get all --kubeconfig=config-demo --all-namespaces
+```
+For long term usage, you will need to copy the content to your `~/.kube/config` file
+
+## Helm
+
+Add Kubernetes yaml template engine and the package manager for Kubernetes
+
+### How to use
+
+```bash
+helm init
+helm upgrade --install -f abc/values-staging.yaml some-name ./abc
+# abc/values-staging.yaml the value file
+# some-name the release name
+# abc the template folder
+helm delete --purge mqtt  # mqtt the release name
+
+```
+### Install 3rd party packages
+```bash
+helm repo add gitlab https://charts.gitlab.io/ # add remote repo
+helm repo update # update index
+helm install mirantisworkloads/vernemq
+```
+
+### Tips
+
+* Normally when you just updated the configmap the deployment or statefulset pod wouldn't updated, but you can add a label to deployment/statefulset yaml, when the value changes the pods will be recreated
+
+```yaml
+template:
+  metadata:
+    labels:
+      app: vernemq
+      configmapVersion: "{{ .Release.Revision }}"
+```
+
+### Deploy tools comparsion
+
+https://blog.hasura.io/draft-vs-gitkube-vs-helm-vs-ksonnet-vs-metaparticle-vs-skaffold-f5aa9561f948/
+
+## Rolling update
+
+Readiness Probe. Readiness Probe makes sure that the new pods created are ready to take on requests before terminating the old pods. To enable this, first you need to have a route in whatever the application you want to run which would return a 200 on an HTTP GET (Or other type) request.
+
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: hello-dep
+  namespace: default
+spec:
+  replicas: 2
+  strategy:
+  type: RollingUpdate
+  rollingUpdate:
+    maxSurge: 1
+    maxUnavailable: 25%
+  selector:
+    matchLabels:
+      app: hello-dep
+  template:
+    metadata:
+      labels:
+        app: hello-dep
+    spec:
+      containers:
+      - image: gcr.io/google-samples/hello-app:2.0
+        imagePullPolicy: Always
+        name: hello-dep
+        ports:
+        - containerPort: 8080
+        readinessProbe:
+          httpGet:
+             path: /
+             port: 8080
+             initialDelaySeconds: 5
+             periodSeconds: 5
+             successThreshold: 1
+```
+`maxUnavailable` is an optional field that specifies the maximum number of Pods that can be unavailable during the update process. The value can be an absolute number (for example, 5) or a percentage of desired Pods (for example, 10%). The absolute number is calculated from percentage by rounding down. The value cannot be 0 if maxSurge is 0. The default value is 25%.
+
+`maxSurge` is an optional field that specifies the maximum number of Pods that can be created over the desired number of Pods
+
+`initialDelaySeconds`: Number of seconds after the container has started before readiness probes are initiated.
+
+`periodSeconds`: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.
+
+`timeoutSeconds`: Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.
+
+`successThreshold`: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness. Minimum value is 1.
+
+`failureThreshold`: When a Pod starts and the probe fails, Kubernetes will try failureThreshold times before giving up. Giving up in case of liveness probe means restarting the Pod. In case of readiness probe the Pod will be marked Unready. Defaults to 3. Minimum value is 1.
+
+## Horizontal scaling
+
+### V1
+
+```yaml
+apiVersion: autoscaling/v1
+kind: HorizontalPodAutoscaler
+metadata:
+  name: worker-auto-scaling
+  namespace: x-prod
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: <deployment-name>
+  minReplicas: 1
+  maxReplicas: 2
+  targetCPUUtilizationPercentage: 75 # trigger point
+```
+
+### V2
+
+```yaml
+apiVersion: autoscaling/v2beta2
+kind: HorizontalPodAutoscaler
+metadata:
+  name: php-apache
+  namespace: default
+spec:
+  scaleTargetRef:
+    apiVersion: apps/v1
+    kind: Deployment
+    name: php-apache
+  minReplicas: 1
+  maxReplicas: 10
+  metrics:
+  - type: Resource
+    resource:
+      name: cpu
+      target:
+        type: Utilization
+        averageUtilization: 50
+  - type: Pods
+    pods:
+      metric:
+        name: packets-per-second
+      target:
+        type: AverageValue
+        averageValue: 1k
+  - type: Object
+    object:
+      metric:
+        name: requests-per-second
+      describedObject:
+        apiVersion: networking.k8s.io/v1beta1
+        kind: Ingress
+        name: main-route
+      target:
+        type: Value
+        value: 10k
+```
+## Limit CPU and memory for pods
+
+```yaml
+ports:
+- name: db-port
+  containerPort: 2345
+  protocol: TCP
+resources:
+requests:
+  cpu: 30m
+  memory: 64Mi
+limits:
+  cpu: 200m
+  memory: 256Mi
+```
+
+## Kubernetes Operator
+
+A Kubernetes Operator is an abstraction for deploying non-trivial applications on Kubernetes. It wraps the logic for deploying and operating an application using Kubernetes constructs. As an example, the `etcd` operator provides an `etcd` cluster as a first-class object.
+
+### An example Operator
+
+- deploying an application on demand
+- taking and restoring backups of that application’s state
+-handling upgrades of the application code alongside related changes such as database schemas or extra configuration settings
+- publishing a Service to applications that don’t support Kubernetes APIs to discover them
+- simulating failure in all or part of your cluster to test its resilience
+- choosing a leader for a distributed application without an internal member election process
+
+### Use Operator
+
+```
+kubectl get SampleDB                   # find configured databases
+
+kubectl edit SampleDB/example-database # manually change some settings
+```
+
+### Some 3rd party operators
+
+- Operator registry: https://operatorhub.io/
+- Custom operators list: https://gist.github.com/philips/a97a143546c87b86b870a82a753db14c
+- Prometheus operator: https://coreos.com/blog/the-prometheus-operator.html
+
+## Debug on kubernetes
+
+### Examine previous pod logs
+
+View the container logs before crash
+
+```bash
+kubectl logs --previous ${POD_NAME} ${CONTAINER_NAME}
+```
+
+### Debug a pod 
+
+Let's assume you have a pod called test-app, but it doesn't have a shell or utility attched to it.
+
+```bash
+kubectl debug -it ephemeral-demo --image=busybox --target=test-app
+```
+This can attach busybox to the target pod, so you can debug your pod with tools.
+
+### Copy a pod
+
+Sometimes Pod configuration options make it difficult to troubleshoot in certain situations, so we can copy the orginal pod to our debugging pod
+
+```bash
+# source pod
+kubectl run myapp --image=busybox --restart=Never -- sleep 1d
+
+kubectl debug myapp -it --image=ubuntu --share-processes --copy-to=myapp-debug
+```
+The --share-processes allows the containers in this Pod to see processes from the other containers in the Pod.
+
+### Copy a pod with changing command or the image
+
+```bash
+# You can use kubectl debug to create a copy of this Pod with the command changed to an interactive shell:
+
+kubectl debug myapp -it --copy-to=myapp-debug --container=myapp -- sh
+
+kubectl debug myapp --copy-to=myapp-debug --set-image=*=ubuntu
+```
+The syntax of --set-image uses the same container_name=image syntax as kubectl set image. *=ubuntu means change the image of all containers to ubuntu.
+
+
+
+## Tools
+
+### Kubectx and kubens
+
+Switch faster between clusters and namespaces in kubectl https
+
+- kubectx for switching contexts
+- kubens for switching namespaces
+
+Github page: https://github.com/ahmetb/kubectx
+
+
+## Other distributions
+
+### OpenShift(OKD)
+
+https://github.com/openshift/okd
+
+OKD is the Origin community distribution of Kubernetes optimized for continuous application development and multi-tenant deployment. OKD adds developer and operations-centric tools on top of Kubernetes to enable rapid application development, easy deployment and scaling, and long-term lifecycle maintenance for small and large teams. OKD is also referred to as Origin in github and in the documentation. OKD makes launching Kubernetes on any cloud or bare metal a snap, simplifies running and updating clusters, and provides all of the tools to make your containerized-applications succeed.
+
+#### Differences
+
+- More strict on security, for example you can't run docker images with `root` user.
+- Better UI and come with some management tools.
+- Integrated CI/CD
+- It's a Redhat product, OKD is its open-source project
+
+![differences between Kubernetes and Openshift](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/openshift.png)
+
+
+### K3s
+
+https://github.com/rancher/k3s
+
+k3s is intended to be a fully compliant Kubernetes distribution with the following changes:
+
+1. Removed most in-tree plugins (cloud providers and storage plugins) which can be replaced
+   with out of tree addons.
+2. Add sqlite3 as the default storage mechanism. etcd3 is still available, but not the default.
+3. Wrapped in simple launcher that handles a lot of the complexity of TLS and options.
+4. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed). k3s packages required
+   dependencies
+    * containerd
+    * Flannel
+    * CoreDNS
+    * CNI
+    * Host utilities (iptables, socat, etc)
+
+
+## Links
+
+- https://kubernetes.io/docs/concepts/overview/components
+- [Writing a Custom Controller: Extending the Functionality of Your Cluster [I] - Aaron Levy](https://www.youtube.com/watch?v=_BuqPMlXfpE)
+- [kubectl docs](https://kubectl.docs.kubernetes.io/)
@@ -0,0 +1,518 @@
+---
+title: Linux
+created: 2021-11-09
+updated: 2021-11-09
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/linux.md
+---
+
+# Linux
+
+## Useful commands and tips
+
+### Split strings
+
+```bash
+# split ip list string by comma and get first one
+export IP=127.0.0.2,127.0.0.3
+echo $IP | cut -d ',' -f1
+```
+
+### Create a systemd service
+
+Create a file called `/etc/systemd/system/hello.service`
+
+```
+[Unit]
+Description=Hello Service
+After=network.target
+StartLimitIntervalSec=0
+[Service]
+Type=simple
+Restart=always
+RestartSec=1
+User=centos
+ExecStart=/app/hello.sh
+
+[Install]
+WantedBy=multi-user.target
+```
+Automatically start the service when boot the vm
+
+```bash
+sudo systemctl enable hello
+```
+
+Start service
+
+```bash
+sudo systemctl start hello
+```
+Learned from [here](https://medium.com/@benmorel/creating-a-linux-service-with-systemd-611b5c8b91d6)
+
+### ssh
+Do not use default `22` port
+```bash
+ssh user@ip -p 22222
+```
+
+Login without password
+
+```bash
+# adds private key identities to the authentication agent
+ssh-add ~/.ssh/id_rsa
+
+# use locally available keys to authorise logins on a remote machine
+
+ssh-copy-id user@ip -p 22222
+```
+
+### Add user to the sudo group
+
+```bash
+usermod -aG sudo youruser
+```
+
+### IP table
+
+### Cron job
+
+List cron jobs for current user
+
+```bash
+crontab -l
+```
+
+List cron jobs for some user
+
+```bash
+crontab -u user -l
+```
+
+Edit cron jobs
+
+```bash
+crontab -e
+```
+
+#### An example cron job
+
+For example, you can run a backup of all your user accounts
+
+```bash
+# at 5 a.m every week with:
+# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
+```
+#### Check cron jobs logs
+
+```bash
+grep CRON /var/log/syslog
+```
+
+#### Cron job expression
+
+`MIN HOUR DOM MON DOW`
+
+Field    Description    Allowed Value
+MIN      Minute field    0 to 59
+HOUR     Hour field      0 to 23
+DOM      Day of Month    1-31
+MON      Month field     1-12
+DOW      Day Of Week     0-6
+
+### PS
+```bash
+ps aux | grep 'ruby'
+```
+List process, using `grep` to filter processes
+### tail
+Tailing texts from file in realtime
+
+```bash
+tail -f a.log
+```
+`f` means following, auto append texts to console
+### nohup
+Simply running application in background
+
+
+### df
+```bash
+df -h
+```
+Show disk usage and left, `h` means show in human readable way. Normally by MB/GB.
+### free
+```bash
+              total        used        free      shared  buff/cache   available
+Mem:        3960624      416720      270024       22252     3273880     3264060
+Swap:       4104188      109688     3994500
+```
+Show memory usage and
+
+### Watch
+```
+watch kubectl get pods
+```
+Continuous running the commands, watching the changes. By default it refresh the result every `2s`.
+
+### Change timezone
+```bash
+TZ=Asia/Shanghai;
+ln -snf /usr/share/zoneinfo/$TZ /etc/localtime
+```
+
+### Check release
+```bash
+cat /etc/*-release
+```
+
+### Debian check system version
+```bash
+cat /etc/issue
+
+cat /etc/debian_version
+```
+
+### htop
+
+like top, but more
+
+#### mosh
+
+auto reconnect ssh
+
+### show last something time
+
+```bash
+last shutdown
+last reboot
+```
+
+### Count files in current folder
+
+```bash
+find . -type f | wc -l
+```
+### Check if a port is open
+
+Used Netcat
+
+```bash
+if ! nc -z localhost 5672; then
+sleep 3;
+fi
+```
+
+### scp copy files between host
+
+#### Copy files from remote server
+
+```bash
+scp your_username@remotehost.edu:foobar.txt /local/dir
+```
+
+### Copy local files to remote server
+
+```bash
+scp /path/to/local/file user@server:/path/to/remote/file
+```
+
+### Merge video files into one
+
+```bash
+ffmpeg -f concat -safe 0 -i ./files.txt -c copy output.mp4
+```
+
+In the `files.txt` you should list all the video clips like the following
+
+```txt
+file 1.flv
+file 2.flv
+file 3.flv
+```
+
+### Curl
+
+Curl is a very popular and powerful HTTP client, there is also another very good tool called `httpie` written in Python which is even eaiser to use.
+
+The following are some useful commands:
+
+#### Post request with file
+```bash
+curl --data "@$(pwd)/fixure/sample.json" --user "foo:bar" localhost:8080/api/abc
+```
+
+### Mount
+
+#### Mount NFS(Network file system) to local
+
+```bash
+
+mkdir /to-local-folder
+# mount
+mount -t nfs -O user=abc,pass=pass 192.168.1.2:/nfs-folder  /to-local-folder
+
+# remount
+mount -o remount /to-local-folder
+```
+
+#### Mount folder permanently
+
+In the example above, the mounted folder will lost the connection when you restart your system, in order to permanently mount the folder, you will need to edit `/etc/fstab` by adding one more line
+
+```
+192.168.1.2:/nfs-folder   /to-local-folder    nfs    defaults    0 0
+```
+You can specify more options
+
+```
+192.168.0.216:/nfs-folder   /to-local-folder    nfs    defaults,proto=tcp,port=2049    0 0
+```
+
+### Rsync
+
+Sync folders and files between folders and hosts
+
+### sync between directories
+
+```bash
+rsync -avzh /root/rpmpkgs /tmp/backups/
+```
+#### sync folders between hosts
+
+```bash
+rsync -avz --exclude downloads --exclude 'some-folder' /mnt rsync://user@192.168.1.2/rsync/ -v -c
+```
+
+You can export `RSYNC_PASSWORD` to pass password to the command.
+
+#### Check rsync logs
+
+```bash
+grep -ir rsync /var/log
+```
+
+### Check debian version
+
+```bash
+cat /etc/issue
+
+cat /etc/debian_version
+```
+
+### Debian error bash: netstat: command not found
+
+```bash
+apt-get install net-tools
+```
+
+### Create CPU loads
+
+```bash
+dd if=/dev/urandom | gzip -9 >> /dev/null &
+```
+
+### Kill the load process
+
+```bash
+kill %1
+```
+
+### IP Address
+
+```bash
+sudo /sbin/ifconfig
+
+ip address # new command
+```
+
+### IP range, CIDR
+
+```bash
+# ip range  ,CIDR network
+
+/16 /20 /24
+```
+
+### trace route tool
+
+```bash
+sudo apt-get install traceroute
+
+sudo traceroute google.com -I
+```
+
+### linux list all the service with status
+
+```bash
+systemctl --full --type service --all
+```
+
+### To see details about the RAM installed on your VM, run the following command:
+
+```bash
+sudo dmidecode -t 17
+```
+
+### verify the number of processors, run the following command:
+
+```bash
+nproc
+```
+
+### CPU information
+
+```bash
+lscpu
+
+# Check cpu platform
+
+cat /proc/cpuinfo
+```
+
+### Running application in background with screen
+
+```bash
+sudo apt-get install -y screen
+
+sudo screen -S mcs java -Xms1G -Xmx7G -d64 -jar /home/minecraft/minecraft_server.1.11.2.jar nogui
+
+detach
+
+# To detach the screen terminal, press Ctrl+A, D. The terminal continues to run in the background. To reattach the terminal, run the following command:
+
+sudo screen -r mcs
+
+# send commands to screen
+
+sudo screen -r -X stuff '/stop\n'
+```
+
+### ApacheBench To place a load on the load balancer, run the following command
+
+```bash
+ab -n 50000 -c 1000 http://ip
+```
+
+### Mount drives
+
+```bash
+1. Find what the drive is called
+You'll need to know what the drive is called to mount it. To do that fire off one of the following (ranked in order of my preference):
+lsblk
+sudo blkid
+sudo fdisk -l
+You're looking for a partition that should look something like: /dev/sdb1. The more disks you have the higher the letter this is likely to be. Anyway, find it and remember what it's called.
+2. Create a mount point (optional)
+This needs to be mounted into the filesystem somewhere. You can usually use /mnt/ if you're being lazy and nothing else is mounted there but otherwise you'll want to create a new directory:
+sudo  mkdir /media/usb
+3. Mount!
+sudo mount /dev/sdb1 /media/usb
+When you're done, just fire off:
+sudo umount /media/usb
+```
+
+### Change dns server
+
+```bash
+vim /etc/resolv.conf
+```
+
+### Last boot
+
+```bash
+last reboot | less
+last -x shutdown
+```
+
+### Ports scanner
+
+```bash
+#nmap scan ports
+nmap -v 203.192.94.53
+```
+
+### Check and turn off swap
+
+```bash
+# swap
+
+1. Identify configured swap devices and files with cat /proc/swaps.
+2. Turn off all swap devices and files with swapoff -a.
+3. Remove any matching reference found in /etc/fstab.
+4. Optional: Destroy any swap devices or files found in step 1 to prevent their reuse. Due to your concerns about leaking sensitive information, you may wish to consider performing some sort of secure wipe.
+```
+
+### Reset timezone
+
+```bash
+sudo dpkg-reconfigure tzdata
+```
+
+### List all timezones
+
+```bash
+ls /usr/share/zoneinfo/
+
+# check ssh failed retries
+
+grep "Failed password" /var/log/auth.log
+```
+
+### Linux disk analyse
+
+```bash
+###  Ncdu analyse linux disk
+
+ncdu
+```
+
+### VIM setup
+
+```bash
+# https://github.com/amix/vimrc
+
+git clone --depth=1 https://github.com/amix/vimrc.git ~/.vim_runtime
+sh ~/.vim_runtime/install_awesome_vimrc.sh
+```
+
+### check ssh failed retries
+
+```bash
+grep "Failed password" /var/log/auth.log
+
+# In order to display extra information about the failed SSH logins, issue the command as shown in the below example.
+
+ egrep "Failed|Failure" /var/log/auth.log
+```
+
+## Popular Distributions
+
+### Debian
+
+### Ubuntu
+
+### Manjaro
+
+### Fedora
+
+Fedora is the main project, and it’s a community-based, free distro focused on quick releases of new features and functionality.
+
+### CentOS
+
+CentOS is basically the community version of Redhat. So it’s pretty much identical, but it is free and support comes from the community as opposed to Redhat itself.
+
+### RHEL (RedHat Enterprise Linux)
+
+![redhat](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/redhat.webp)
+
+
+RHEL is the corporate version based on the progress of that project, and it has slower releases, comes with support, and isn’t free.
+
+### Arch Linux
+
+## Package manager
+
+* [snap](https://snapcraft.io/)
+* [Brew](https://brew.sh/)
@@ -0,0 +1,18 @@
+---
+title: More Tools
+created: 2021-11-04
+updated: 2021-11-04
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/devops/more-tools.md
+---
+
+# More tools
+
+## HTTP performace tests
+
+Some command line toos which can provide categorized information for http requests.
+
+* [curl_time](https://stackoverflow.com/questions/18215389/how-do-i-measure-request-and-response-times-at-once-using-curl?rq=1)
+* [hey](https://github.com/rakyll/hey)
+* [ab](https://httpd.apache.org/docs/2.4/programs/ab.html)
@@ -0,0 +1,25 @@
+---
+title: Business Spoken English
+created: 2018-03-25
+updated: 2018-03-25
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/english/business-spoken-english.md
+---
+
+# Business English Speaking for Professionals
+
+
+## Some tips for doing presentation
+
+* Talk loud enough
+* Look at people when you are talking
+  * Look at people's eyes
+  * Look at people's noses
+* Change your tones occasionally
+* Be interested in what you are talking about
+* If you are interested, pretend to be
+* Use your face, don't to be a poker face
+* Change your hands while you talking
+* Pick a spot, stand still and tall
+* Ask yourself question, how can I help them?
@@ -0,0 +1,54 @@
+---
+title: Fintech
+created: 2019-12-02
+updated: 2019-12-02
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/fintech.md
+---
+
+# Fintech(Financial technology)
+
+## ISO8583 message
+
+### A example
+
+```
+02004200040000000002161234567890123456060
+9173030123456789ABC1000123456789012345678
+90123456789012345678901234567890123456789
+0123456789012345678901234567890123456789
+```
+Where:
+
+```
+  0200 - MTI (Message Type Indicator),
+  4200040000000002 - primary bitmap,
+  161234567890123456 - field 2, the first 2 digits: 16, is length indicator
+  0609173030 - field 7,
+  123456789ABC - field 22,
+  012345678901234567890123456789012345678901234567890\
+  1234567890123456789012345678901234567890123456789 - field 63.
+```
+
+### Definition
+
+ISO8583 message is used in bank or financial institutes card-originated transactions. But sometime, those not card-originated transactions also use ISO8583.
+
+### Typical Parts
+
+Typical Fields an ISO8583 message
+A typical message might include such fields:
+```
+MTI - Message Type Indicator.
+Bit 2 - Primary Account Number (PAN)
+Bit 3 - Processing Code
+Bit 7 - Date and Time of transmission
+Bit 11 - Audit Code
+Bit 12 - Local time transaction
+Bit 13 - Local transaction date
+Bit 32 - Identification code of the institution of acquisition
+Bit 38 - The authorization code in response
+Bit 39 - Response Code
+Bit 49 - Currency code of transaction
+```
@@ -0,0 +1,438 @@
+---
+title: Go
+created: 2020-04-05
+updated: 2020-04-05
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/go.md
+---
+
+## pass pointer or value
+- value: Variable must not be modified
+- Variable is a large struct  then prefer pointer
+- Variable is a map or slice then prefer value
+- Passing by value often is cheaper
+
+## struct
+
+- make
+```go
+make(T, args) -> T
+```
+- new
+```go
+new(T) -> *T
+```
+- a := T{}
+
+## array
+```go
+var a [1]int
+```
+Array has a exactly length, can't be modified.
+## slice
+
+```go
+var a []string
+[]string{"a", "b"}
+[...]string{"a","b"}
+```
+1. Auto increment length
+```go
+new([]int)
+make([]int, 2, 5)
+```
+2. nil is a valid slice which length is `0`
+
+## Go routine
+
+- you can run more goroutine vs thread
+- go routine have a faster start up time than thread
+- go routine come with built-in primitives to communicate safely by using channels
+
+## Closure
+
+## Channel
+
+- Normal channel
+```go
+messages := make(chan string)
+// _Send_ a value into a channel using the `channel <-`
+go func() { messages <- "ping" }()
+// channel. Here we'll receive the `"ping"` message
+msg := <-messages
+```
+
+- buffered channel
+```go
+ch := make(chan Task, 3)
+```
+
+## Big Numbers
+
+Package big implements arbitrary-precision arithmetic (big numbers).
+For example the int numbers larger than `int64` or the float greater than `float64`.
+
+```
+int16: (-32,768 to +32,767)
+
+int32: (-2,147,483,648 to +2,147,483,647)
+
+int64: (-9,223,372,036,854,775,808 to +9,223,372,036,854,775,807)
+```
+The following numeric types are supported:
+```
+ Int    signed integers
+ Rat    rational numbers
+ Float  floating-point numbers
+```
+
+Some exmaple code
+```go
+import "math/big"
+
+//string to Int
+n1 := new(big.Int)
+n1, ok = n1.SetString(str1, 10)
+
+// Int1 + Int2
+result := new(big.Int)
+result.Add(n1, n2)
+
+// Int to string
+result.String()
+```
+
+## Sync
+
+###  atomic
+
+### Mutex
+
+  - sync.Mutex()
+  - mutex.Lock()
+  - mutx.Unlock()
+
+```go
+package main
+
+import (
+	"fmt"
+	"sync"
+	"time"
+)
+
+// SafeCounter is safe to use concurrently.
+type SafeCounter struct {
+	v   map[string]int
+	mux sync.Mutex
+}
+
+// Inc increments the counter for the given key.
+func (c *SafeCounter) Inc(key string) {
+	c.mux.Lock()
+	// Lock so only one goroutine at a time can access the map c.v.
+	c.v[key]++
+	c.mux.Unlock()
+}
+
+// Value returns the current value of the counter for the given key.
+func (c *SafeCounter) Value(key string) int {
+	c.mux.Lock()
+	// Lock so only one goroutine at a time can access the map c.v.
+	defer c.mux.Unlock()
+	return c.v[key]
+}
+
+func main() {
+	c := SafeCounter{v: make(map[string]int)}
+	for i := 0; i < 1000; i++ {
+		go c.Inc("somekey")
+	}
+
+	time.Sleep(time.Second)
+	fmt.Println(c.Value("somekey"))
+}
+```
+
+
+### RWMutex
+!> A RWMutex is a reader/writer mutual exclusion lock. The lock can be held by an arbitrary number of readers or a single writer. The zero value for a RWMutex is an unlocked mutex.
+In other words, readers don't have to wait for each other. They only have to wait for writers holding the lock.
+
+### string literals
+
+```go
+    `aaa bbb ccc`
+```
+### panic / recover
+
+###  `sync.Map` is concurrent/ thread safe `map`, normal `map` isn't
+
+```go
+m := new(sync.Map)
+m.Store("a", "b")
+value, ok := m.Load("a")
+```
+
+### Defer to Clean Up
+
+Use defer to clean up resources such as files and locks.
+```go
+p.Lock()
+defer p.Unlock()
+
+if p.count < 10 {
+  return p.count
+}
+
+p.count++
+return p.count
+
+// more readable
+```
+
+## Monotonic Clocks
+
+Operating systems provide both a `wall clock`, which is subject to changes for clock synchronization, and a `monotonic clock`, which is not. The general rule is that the wall clock is for telling time and the monotonic clock is for measuring time. Rather than split the API, in this package the Time returned by time.Now contains both a wall clock reading and a monotonic clock reading; later time-telling operations use the wall clock reading, but later time-measuring operations, specifically comparisons and subtractions, use the monotonic clock reading.
+
+For example, this code always computes a positive elapsed time of approximately 20 milliseconds, even if the wall clock is changed during the operation being timed:
+
+```go
+start := time.Now()
+... operation that takes 20 milliseconds ...
+t := time.Now()
+elapsed := t.Sub(start)
+```
+
+### wall clock
+
+This clock is subject to potential variations. For example, if it is synchronized with NTP (Network Time Protocol). In this case after synchronization, the local clock of our server can jump backward or forward in time. So measuring a duration from the wall-clock can be biased.
+
+### Monotonic clock
+
+we have a guarantee that the time always moves forward and will not be impacted by variations leading to jumps in time.
+
+Therefore, if we have to measure durations, we must use the monotonic-clock. This rule of thumb is only valid for local duration measurements though. Indeed, the monotonic-clocks of two different servers are by definition not synchronized. So, measuring a distributed execution based on these clocks will not be accurate.
+
+## Frameworks
+
+### db
+
+- xorm
+- gorm
+
+### web
+
+- beego
+- mux
+- gin
+- go kit
+    - full stack micro service framework like spring boot
+
+## Tools
+
+### profiler
+    - go-wrk(wrk)
+        - a http benchmark tool
+    - go-torch
+        - Stochastic flame graph profiler
+### test
+    - Testify <http://github.com/stretchr/testify>
+    - Ginkgo <http://onsi.github.io/ginkgo/>
+### Linter
+    - Golangci-lint https://github.com/golangci/golangci-lint
+
+# Tips
+
+## Sort slice
+
+```go
+// sort users by user age ASC
+sort.Slice(users, func(i, j int) bool {
+  return users[i].age < planets[j].age
+})
+```
+
+## slice
+
+- byte* array //actual data
+- uintgo len
+- uintgo cap
+
+## map
+
+* implement by hash table
+* slice can't be the key of a map, but sized array could. e.g. `var a map[[2]int]string`
+
+## Go has no generics
+
+- performance
+- complexity
+- If C++ and Java are about type hierarchies and the taxonomy of types, Go is about composition.
+- How to solve
+    - use interface
+    - use type assertions
+    - use reflection
+
+## Modify item in range
+
+- use the array index instead of the value
+```go
+    for _, e := range array {
+        e.field = "foo"
+    }
+
+    for idx, _ := range array {
+        array[idx].field = "foo"
+    }
+```
+## merge two array
+
+- a = append(a, b…)
+    - must add … to b, otherwise you can only add one item
+
+## error handling
+
+### check error type
+
+```go
+ErrorSample := errors.New("some error")
+if errors.Is(err, ErrorSample) {
+    // something wasn't found
+}
+```
+
+### error type assertion
+```go
+    if serr, ok := err.(*json.SyntaxError); ok {}
+//or
+
+// var e *QueryError
+if errors.As(err, &e) {
+    // err is a *QueryError, and e is set to the error's value
+}
+
+```
+
+### better error handling
+
+#### custom error type
+
+```go
+type appError struct {
+    Error   error
+    Message string
+    Code    int
+}
+```
+####  concat error check
+
+```go
+if err1() != nil || err2() != nil {}
+```
+#### some error constants
+
+```go
+errNotFound = errors.New("Item not found")
+switch err {
+    case errNotFound:
+}
+```
+
+## date format
+```go
+t := time.Now()
+fmt.Println(t.String())
+fmt.Println(t.Format("2006-01-02 15:04:05"))
+```
+
+- var _ InterfaceX = &InterfaceXImplementation{}
+	- make sure InterfaceX’s implementation works
+
+## reference types in go
+* map
+* channel
+* slice
+
+## value types
+* Array
+
+## Read file line by line
+```go
+package main
+
+import (
+    "bufio"
+    "fmt"
+    "log"
+    "os"
+)
+
+func main() {
+    file, err := os.Open("/path/to/file.txt")
+    if err != nil {
+        log.Fatal(err)
+    }
+    defer file.Close()
+
+    scanner := bufio.NewScanner(file)
+    for scanner.Scan() {
+        fmt.Println(scanner.Text())
+    }
+
+    if err := scanner.Err(); err != nil {
+        log.Fatal(err)
+    }
+}
+```
+
+## commands
+
+### test
+
+- go test ./...
+    -  run all tests in current directory and all of its subdirectories
+- go test foo/...
+    -  run all tests with import path prefixed with foo/:
+- go test foo...
+    - run all tests import path prefixed with foo:
+- go test ...
+    - run all tests in your $GOPATH:
+
+## REPL
+
+REPL stands for read eval print loop, basically it just like the irb in Ruby.
+
+Wiki: <https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop>
+
+* gore <https://github.com/motemen/gore>
+
+# Some code snippets
+
+## defer would still run after panic
+
+```go
+package main
+func main() {
+	for {
+		defer func() {
+			for {
+			}
+		}()
+		panic("yolo")
+	}
+}
+```
+## Some interesting mistakes
+
+### Do not reuse HTTP request when retrying
+
+See more information: https://stackoverflow.com/questions/55385894/http-contentlength-99-with-body-length-0
+
+
+
+# Useful links
+* [Go best practice](https://github.com/golang/go/wiki/CodeReviewComments)
+* [Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
+* [Uber Go code style](https://github.com/uber-go/guide/blob/master/style.md#pointers-to-interfaces)
@@ -0,0 +1,54 @@
+---
+title: Interview
+created: 2022-04-06
+updated: 2022-04-06
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/interviews/interview.md
+---
+
+# Interviews
+
+## System design
+
+### Questions to ask when you receive a question
+
+* How many users will be using it? What's the throughput we need to design for?
+* Do we need to consider accessing by users from multiple countries and regions?
+* Is this for a single tenant for multiple tenants?
+* What's the latency requirements?
+* Do we need to consider logging and montoring?
+
+### Steps to answer the question
+
+* First, briefly talk about what the high level design, the components
+* Briefly calculatt the traffic which relates to what db you choose and system architectures.
+* Talk about the DB choice and why, NOSQL or relational SQL database
+* Keep interation with interviewers, and ask if the answer is good enough, do you want me to dive deeper?
+
+
+## Live coding
+
+### Steps
+
+* Clarify the inputs and outputs
+* Briefly write overall steps
+* I will always write a test case before implementing it
+* Calculate the Algorithm complexity, O(n)
+* Keep talking to the intreviewer and keep interactions
+
+## resources
+
+- [https://github.com/kdn251/interviews](https://github.com/kdn251/interviews)
+
+- [5 Salary Negotiation Rules for Software Developers](https://dev.to/aershov24/5-salary-negotiation-rules-for-software-developers-get-20-on-top-of-your-market-rate-2jii)
+
+
+
+## Some interview questions
+
+### Why ssd is faster than hdd
+
+One is that compared to a spinning hard disk the access (seek) time is miniscule, the hard disk needs to move the head to the right track and then wait for the sector to be under the head (depends on rotation speed) this is on average about 10 milliseconds. The SSD on the other hand needs to send a command to the flash chip on a fast interconnect and data will stream back in a few microseconds.
+
+The other factor is parallelism, the HDD has a single head construction and it can only read from one head at a time so if you send it 32 commands at the same time the last one will only be started after all the others have completed (assuming no reordering). The SSD can access multiple flash chips at the same time and order them all to fetch data, it can even pull the data to its own RAM in parallel from multiple flash chips, most consumer SSDs can handle at least 8 requests in parallel so it can be blazing fast. This is also the different between the SSD and a USB flash drive, the USB flash drive only has one flash chip so it has no parallelization.
@@ -0,0 +1,73 @@
+---
+title: Iot
+created: 2020-04-05
+updated: 2020-04-05
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/iot.md
+---
+
+# IoT
+
+## MQTT
+
+MQTT is a Client Server publish/subscribe messaging transport protocol. It is light weight, open, simple, and designed so as to be easy to implement. These characteristics make it ideal for use in many situations, including constrained environments such as for communication in Machine to Machine (M2M) and Internet of Things (IoT) contexts where a small code footprint is required and/or network bandwidth is at a premium. MQTT means **MQ Telemetry Transport**
+
+### Core features
+
+* Connect, Pub/Sub
+* Quality of Service(QoS)
+* Retained Messages
+* Persistent Session
+* Last Will and Testament
+* Keep Alive
+
+### Why Pub/Sub is good
+
+The most important aspect of pub/sub is the decoupling of the publisher of the message from the recipient(subscriber). This decoupling has several dimensions:
+
+* **Space decoupling**: Publisher and subscriber do not need to know each other (for example, no exchange of IP address and port).
+
+* **Time decoupling**: Publisher and subscriber do not need to run at the same time.
+
+* **Synchronization decoupling**: Operations on both components do not need to be interrupted during publishing or receiving.
+
+!> In summary, the pub/sub model removes direct communication between the publisher of the message and the recipient/subscriber. The filtering activity of the broker makes it possible to control which client/subscriber receives which message. The decoupling has three dimensions: space, time, and synchronization.
+
+MQTT embodies all the aspects of pub/sub that we’ve mentioned:
+
+* MQTT decouples the publisher and subscriber spatially. To publish or receive messages, publishers and subscribers only need to know the hostname/IP and port of the broker
+
+* MQTT decouples by time. Although most MQTT use cases deliver messages in near-real time, if desired, the broker can store messages for clients that are not online. (Two conditions must be met to store messages: the client had connected with a persistent session and subscribed to a topic with a Quality of Service greater than 0).
+
+* MQTT works asynchronously. Because most client libraries work asynchronously and are based on callbacks or a similar model, tasks are not blocked while waiting for a message or publishing a message. In certain use cases, synchronization is desirable and possible. To wait for a certain message, some libraries have synchronous APIs. But the flow is usually asynchronous.
+
+Another thing that should be mentioned is that MQTT is especially easy to use on the client-side. Most pub/sub systems have the logic on the broker-side, but MQTT is really the essence of pub/sub when using a client library and that makes it a light-weight protocol for small and constrained devices.
+
+
+
+### MQTT Brokers
+
+* [HiveMQ](https://www.hivemq.com/)
+* [VerneMQ](https://vernemq.com)
+* [Mosquitto](https://mosquitto.org/)
+
+
+### Difference between MQTT and Message Queue
+
+**A message queue stores message until they are consumed**
+
+When you use a message queue, each incoming message is stored in the queue until it is picked up by a client (often called a consumer). If no client picks up the message, the message remains stuck in the queue and waits to be consumed. In a message queue, it is not possible for a message not to be processed by any client, as it is in MQTT if nobody subscribes to a topic.
+
+**A message is only consumed by one client**
+
+Another big difference is that in a traditional message queue a message can be processed by one consumer only. The load is distributed between all consumers for a queue. In MQTT the behavior is quite the opposite: every subscriber that subscribes to the topic gets the message.
+
+**Queues are named and must be created explicitly**
+
+A queue is far more rigid than a topic. Before a queue can be used, the queue must be created explicitly with a separate command. Only after the queue is named and created is it possible to publish or consume messages. In contrast, MQTT topics are extremely flexible and can be created on the fly.
+
+
+### Useful posts
+
+* [MQTT Essentials](https://www.hivemq.com/blog/mqtt-essentials-part-1-introducing-mqtt)
@@ -0,0 +1,241 @@
+---
+title: Java
+created: 2020-04-08
+updated: 2020-04-08
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/java.md
+---
+
+## Java GC
+
+### GC types
+
+	- Serial GC
+	- Parallel GC
+	- CMS GC
+	- G1 GC
+
+### Object type
+
+	- Young generation  minor GC
+	- Old generation  full GC
+	- System.gc()
+
+## JVM
+
+- Method Area
+	- Class name, Constant, static Constant, Class object
+- Heap
+	- Object instance, array
+
+	- young generation
+
+	- Old generation
+- VM stack
+	- local variable
+	- dynamic links
+	- operation stacks
+- Native method stack
+- program counter
+
+### JVM insight
+![jvm insight](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/0AF6D004-6074-4396-94A0-6141B398BC94.png)
+## Spring
+- Spring boot
+	- Rest controller
+	- Jpa template
+	- Spring scheduler
+	- Annotation bean
+- Spring cloud
+	- Consol
+	- Zuul
+	- Cloud config
+	- Spring cloud zookeeper( discovery and configuration)
+- Spring ecurity
+
+## ORM
+- Hibernate
+	- validation
+- JPA
+	- Sprint jpa tempalte
+- Mybatis
+
+## Build
+- Gradle
+	- groovy task
+- Maven
+	- XML plugins
+	- Maven mirror
+
+	- [https://maven-central.storage.googleapis.com](https://maven-central.storage.googleapis.com)
+
+	- [http://maven.aliyun.com/nexus/content/groups/public/](http://maven.aliyun.com/nexus/content/groups/public/)
+- Ant
+	- task based
+
+## Composition or inheritance
+
+Think of containment as a has a relationship. A car "has an" engine, a person "has a" name, etc.
+
+Think of inheritance as an is a relationship. A car "is a" vehicle, a person "is a" mammal, etc.
+
+## Useful tips
+
+### Iterate map
+	- for (Map.Entry<String, String> entry : map.entrySet()) { }
+###  List to array
+	- int[] arr= new int[list.length](); list.toArray(arr);
+### Object
+	- immutable
+	- object that can’t be modified after it’s created
+		- Don’t allow subclass override methods, class to be final
+		- Don’t provide setter method
+		- Make all field private and final
+	- mutable
+### Default Timezone
+	- TimeZone.setDefault(TimeZone.getTimeZone("GMT+8"))
+### [generics](https://appliedgo.net/generics/)
+	- Why we need?
+
+	- e.g we have sort method, and different type(n) of  objects. then you have to write n kind of search methods.
+### event listener
+	- [https://stackoverflow.com/questions/6270132/create-a-custom-event-in-java](https://stackoverflow.com/questions/6270132/create-a-custom-event-in-java)
+
+### Try with resource
+The try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements java.lang.AutoCloseable, which includes all objects which implement java.io.Closeable, can be used as a resource.
+
+In short no need add manual `finally` block to close resources.
+
+```java
+    try (InputStream caInput = new ByteArrayInputStream(decode)) {
+      // Generate the CA Certificate from the raw resource file
+
+      Certificate ca = CertificateFactory.getInstance("X.509").generateCertificate(caInput);
+
+      // Load the key store using the CA
+      KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
+      keyStore.load(null, null);
+      keyStore.setCertificateEntry("ca", ca);
+
+      // Initialize the TrustManager with this CA
+      TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
+      tmf.init(keyStore);
+
+      // Create an SSL context that uses the created trust manager
+      SSLContext sslContext = SSLContext.getInstance("TLS");
+      sslContext.init(null, tmf.getTrustManagers(), new SecureRandom());
+
+      return sslContext.getSocketFactory();
+
+    } catch (Exception ex) {
+      logger.error(ex.getMessage());
+      throw new RuntimeException(ex);
+    }
+```
+
+## JAVA SE
+
+### Java Collection
+![java collection](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/java-collection.png)
+
+#### HashMap and HashTable
+
+- `Hashtable` is synchronized, whereas `HashMap` is not. This makes HashMap better for non-threaded applications, as unsynchronized Objects typically perform better than synchronized ones.
+
+- `Hashtable` does not allow null keys or values. `HashMap` allows one null key and any number of null values.
+
+- One of HashMap's subclasses is `LinkedHashMap`, so in the event that you'd want predictable iteration order (which is insertion order by default), you could easily swap out the HashMap for a LinkedHashMap. This wouldn't be as easy if you were using Hashtable.
+
+- `HashMap` is not thread-safe, if you do want to have thread-safe one then use `Collections.synchronizedMap()`,
+- HashTable is thread-safe, but is treated as legacy.
+
+### modifier
+
+	* public
+	* no modifier
+	* protected
+	* priivate
+
+### Java modifier diagram
+
+![Java modifier diagram](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/AF058A2F-FEB9-4C6A-837C-2DFF0950CA6C.png)
+
+### NIO
+	- Channels
+	- A channel is a kind of stream, from the channel data can be read into a buffer.
+	- channel implementations
+		- FileChannel
+			- reads data from and to files
+		- DatagramChannel
+			- read and write data over the network via UDP
+		- SocketChannel
+			- read and write data over the network visa TCP
+		- ServerSocketChannel
+			- allows you to listen for incoming TCP connections, like a web server. For each incoming connection a SocketChannel is created.
+	- Buffer
+
+	- code sample
+```java
+        RandomAccessFile aFile = new RandomAccessFile("data/nio-data.txt", "rw");
+            FileChannel inChannel = aFile.getChannel();
+
+            //create buffer with capacity of 48 bytes
+            ByteBuffer buf = ByteBuffer.allocate(48);
+
+            **int bytesRead = inChannel.read(buf**); //read into buffer.
+            while (bytesRead != -1) {
+
+            **buf.flip**();  //make buffer ready for read
+
+            while(buf.hasRemaining()){
+                System.out.print((char) **buf.get**()); // read 1 byte at a time
+            }
+
+            **buf.clear**(); //make buffer ready for writing
+            bytesRead = inChannel.read(buf);
+            }
+            aFile.close();
+```
+	- properties
+		- capacity
+		- position
+		- limit
+
+	- Steps for buffer to read and write data.
+		- write data into buffer
+		- call buffer.flip()
+		- read data out of the buffer
+		- call buffer.clear() or buffer.compact()
+	- Non-blocking IO
+	- Selectors
+
+	- A selector allows a single thread to handle multiple Channels
+	- [http://tutorials.jenkov.com/java-nio/index.html](http://tutorials.jenkov.com/java-nio/index.html)
+
+### SSL
+
+#### Java use TLSv1.2 by default since Java 8
+
+When you saw some error like this `SSL handshake exception: “Algorithm constraints check failed: MD5withRSA”`, probably you used `MD5` as message digest when generate a CA. You should regenerate a new CA using `sha1`, because the `MD5` has been found to be insecure.
+
+Some more info <https://stackoverflow.com/questions/21218217/ssl-handshake-exception-algorithm-constraints-check-failed-md5withrsa>
+
+### Monotonic clock
+
+`Note`: You can go to the `go` page to check out more background knowledge
+
+```java
+long startTime = System.nanoTime();
+// ... the code being measured ...
+long estimatedTime = System.nanoTime() - startTime;
+
+```
+
+Some code which uses the wall clock
+
+```java
+long start = System.currentTimeMillis();
+// Do something exceptional
+long elapsed = System.currentTimeMillis() - start;
+```
@@ -0,0 +1,251 @@
+---
+title: Javascript
+created: 2018-02-15
+updated: 2018-02-15
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/javascript.md
+---
+
+# ES 6/ES 2015
+
+- Arrow function
+	- Array function doesn’t create new this, it grabs this from its surrounding instead.
+
+- Array/Collection methods
+	- map
+
+		```javascript
+			  const numbers = [0, 1, 2, 3, 4, 5, 6];
+			  const doubledNumbers = numbers.map(n => n * 2);
+         ```
+	- filter
+		- const evenNumbers = numbers.filter(n => n % 2 === 0);
+	- reduce
+         ```javascript
+         const sum = numbers.reduce(
+		    function(acc, n) {
+		      return acc + n;
+		    },
+		    0 // accumulator variable value at first iteration step
+		  );
+         ```
+		- reduce takes two parameters
+		  The first parameter is a function that will be called at each iteration step.The second parameter is the value of the accumulator variable (*acc* here) at the first iteration step (read next point to understand)
+
+- Class
+
+- Template String
+	- const name = "Nick";
+	  `Hello ${name}, the following expression is equal to four : ${2+2}`;
+
+- Destructing
+	- Array
+		- const myArray = ["a", "b", "c"];
+		  //without const x = myArray[0];
+		  const y = myArray[1];
+		  const [x, y] = myArray;
+	- Object
+		 ```javascript
+         const person = {
+		    firstName: "Nick",
+		    lastName: "Anderson",
+		    age: 35,
+		    sex: "M"
+		  }
+		  const { firstName: first, age, city = "Paris" } = person;
+         ```
+- Default + Rest + Spread parameters
+
+- Const
+	- can't be reassigned; but not immutable
+
+- Let
+
+- Modules
+
+- Map / Set
+
+- Promise
+	- .then().catch()
+        ```javascript
+        function getGithubUser(username) {
+		    return new Promise((resolve, reject) => {
+		      fetch(`https://api.github.com/users/${username}`)
+		        .then(response => {
+		          const user = response.json();
+		          resolve(user);
+		        })
+		        .catch(err => reject(err));
+		    })
+		  }
+        ```
+
+- Enhanced Object Literals
+
+- Imports/ Exports
+	- export default xx; import xx from ‘..’
+	- export const xx; import {xx} from ‘..’
+
+- Async/Await
+	- await can only used in an async function.
+		- async function getGithubUser(username) { // async keyword allows usage of await in the function and means function returns a promise
+		    try { // this is how errors are handled with async / await
+		      const response = await fetch(`https://api.github.com/users/${username}`); // "synchronously" waiting fetch promise to resolve before going to next line
+		      return response.json();
+		    } catch (err) {
+		      alert(err);
+		    }
+		  }
+		  getGithubUser('mbeaudru').then(user => console.log(user)); // logging user response - cannot use await syntax since this code isn't in async function
+
+# Node.js
+
+- Npm
+
+- Yarn
+
+- Express
+
+- Stream
+
+# This
+
+- Refers who calls
+     ```javascript
+      function myFunc() {
+	    ...
+	  }
+	  // After each statement, you find the value of *this* in myFunc
+	  myFunc.call("myString", "hello") // "myString" -- first .call parameter value is injected into *this*
+	  // In non-strict-mode
+	  myFunc("hello") // window -- myFunc() is syntax sugar for myFunc.call(window, "hello")
+	  // In strict-mode
+	  myFunc("hello") // undefined -- myFunc() is syntax sugar for myFunc.call(undefined, "hello")
+
+    var person = {
+        myFunc: function() { ... }
+        }
+        person.myFunc.call(person, "test") // person Object -- first call parameter is injected into *this*
+        person.myFunc("test") // person Object -- person.myFunc() is syntax sugar for person.myFunc.call(person, "test")
+        var myBoundFunc = person.myFunc.bind("hello") // Creates a new function in which we inject "hello" in *this* value
+        person.myFunc("test") // person Object -- The bind method has no effect on the original method
+        myBoundFunc("test") // "hello" -- myBoundFunc is person.myFunc with "hello" bound to *this*
+     ```
+	- [http://yehudakatz.com/2011/08/11/understanding-javascript-function-invocation-and-this/](http://yehudakatz.com/2011/08/11/understanding-javascript-function-invocation-and-this/)
+
+# Closure
+
+# Frameworks
+
+- Ionic
+
+- Angular
+
+- Vue.js
+	- component
+		- .sync modifier, A kind of two way binding
+			- :abc.sync=“”; $tis.emit(“update:abc”, newValue) //trigger in child component
+		- computed
+			- computed attributes
+			- computed over watcher
+		- watchers
+			- Monitor data changes
+	- cheat sheet
+		- https://vuejs-tips.github.io/cheatsheet/
+
+- Ember.js
+
+- Backbone
+
+	-
+
+- React.js
+
+- Flux/ Redux
+	- State
+	- Store
+		- Reducer
+	- Action
+	- Dispatch
+
+# Use promise to avoid nested callbacks.
+
+# tips
+
+- object literal
+-   ```javascript
+    Object.keys(object).map(function(objectKey, index) {
+	      var value = object[objectKey];
+	      console.log(value);
+	  });
+    ```javascript
+
+# Application
+
+# Search
+- ElasticSearch
+	- Logstash
+	- Kibana
+
+# APM
+- Newrelic
+- Pinpoint
+
+# Monitor
+- Bosun
+- Prometheus
+
+# API
+
+- RPC(remote procedure call)
+	- disadvantages
+		- coupling
+		- clients needs to know the request procedure names
+	- advantages
+		- more freedoms to define any requests.
+	- examples
+		- [https://api.slack.com/methods](https://api.slack.com/methods)
+
+- Restful /Representational State Transfer
+	- Easy to cache
+	- every url represent a resource
+	- disadvantages
+		- some special method is hard to use restful to name it. like /login, /resetpassword
+	- examples
+		- [https://developer.github.com/v3/](https://developer.github.com/v3/)
+	- methods
+		- GET
+		- POST
+		- PUT
+		- DELETE
+
+- GraphQL
+	- Can define fields you want
+
+- gRPC
+	- protocol buffers
+		 ```javascript
+         // The greeter service definition.
+		  service Greeter {
+		    // Sends a greeting
+		    rpc SayHello (HelloRequest) returns (HelloReply) {}
+		  }
+		  // The request message containing the user's name.
+		  message HelloRequest {
+		    string name = 1;
+		  }
+		  // The response message containing the greetings
+		  message HelloReply {
+		    string message = 1;
+		  }
+        ```javascript
+	- A opensource  remote procedure call framework based on HTTP2 and protobuf
+	- advantages
+		- baed on HTTP2, better performance
+		- call remote like local method, and support multiple language
+		- protobuf serialization and deserialization is faster than JSON
+
+# Log and Error
+
+- Sentry
@@ -0,0 +1,40 @@
+---
+title: Kotlin
+created: 2021-05-07
+updated: 2021-05-07
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/kotlin.md
+---
+
+# Kotlin
+
+## Some tips
+
+### Builder mode in kotlin
+
+```kotlin
+class Car(
+  val model: String?,
+  val color: String?,
+  val type: String?) {
+
+    data class Builder(
+      var model: String? = null,
+      var color: String = "pink",
+      var type: String? = null) {
+
+        fun model(model: String) = apply { this.model = model }
+        fun color(color: String) = apply { this.color = color }
+        fun type(type: String) = apply { this.type = type }
+        fun build() = Car(model, color, type)
+    }
+}
+//use
+val car = Car.Builder()
+  .model("Ford Focus")
+  .color("Black")
+  .type("Type")
+  .build()
+```
+Based on [stackoverflow](https://stackoverflow.com/questions/36140791/how-to-implement-builder-pattern-in-kotlin)
@@ -0,0 +1,71 @@
+---
+title: Microservice
+created: 2019-12-03
+updated: 2019-12-03
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/microservice.md
+---
+
+# Microservice
+
+- benefits
+
+	- Each application is relatively small
+	- Easy to scale
+	- Improved fault isolation
+		- One service fails won’t influence others
+	- You can use different stack for each service
+	- Each service can be deployed independently
+- drawbacks
+
+	- Increased memory consumption
+	- Developers must deal with extra complexity of creating a distributed system
+	- Deployment complexity
+		- To deploy different types of services
+
+## Service Mesh
+
+The term service mesh is used to describe the network of microservices that make up such applications and the interactions between them. As a service mesh grows in size and complexity, it can become harder to understand and manage. Its requirements can include discovery, load balancing, failure recovery, metrics, and monitoring. A service mesh also often has more complex operational requirements, like A/B testing, canary rollouts, rate limiting, access control, and end-to-end authentication.
+
+![service mesh](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/service-mesh.png)
+
+
+### Opensource Projects
+
+* [Istio](https://istio.io/)
+* [Consul](https://www.consul.io/)
+
+## CircuitBreaker Design
+
+### Why we need CircuitBreaker
+
+In case we have serviceB down, serviceA should still try to recover from this and try to do one of the followings:
+- **Custom fallback**: Try to get the same data from some other source. If not possible, use its own cache value.
+- **Fail fast**: If serviceA knows that serviceB is down, there is no point waiting for the timeout and consuming its own resources. It should return ASAP “knowing” that serviceB is down
+- **Don’t crash**: As we saw in this case, serviceA should not have crashed.
+- **Heal automatic**: Periodically check if serviceB is working again.
+- **Other APIs should work**: All other APIs should continue to work.
+
+### What is circuit breaker design?
+The idea behind is simple:
+- Once serviceA “knows” that serviceB is down, there is no need to make request to serviceB. serviceA should return cached data or timeout error as soon as it can. This is the OPEN state of the circuit
+- Once serviceA “knows” that serviceB is up, we can CLOSE the circuit so that request can be made to serviceB again.
+- Periodically make fresh calls to serviceB to see if it is successfully returning the result. This state is HALF-OPEN.
+
+![CircuitBreaker](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/circuitbreaker.png)
+
+
+### More links
+
+- <https://martinfowler.com/bliki/CircuitBreaker.html>
+- <https://itnext.io/understand-circuitbreaker-design-pattern-with-simple-practical-example-92a752615b42>
+
+
+### Self-contained System (SCS)
+
+The Self-contained System (SCS) approach is an architecture that focuses on a separation of the functionality into many independent systems, making the complete logical system a collaboration of many smaller software systems. This avoids the problem of large monoliths that grow constantly and eventually become unmaintainable. Over the past few years, we have seen its benefits in many mid-sized and large-scale projects.
+
+The idea is to break a large system apart into several smaller self-contained systems, or SCSs, that follow certain rules.
+
+![Self contained system](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/scs.png)
@@ -0,0 +1,127 @@
+---
+title: Ruby
+created: 2018-06-14
+updated: 2018-06-14
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/ruby.md
+---
+
+# Objects inheritances
+
+![objects](https://toozhao.com/knowledge-mind-mapping/Knowledge.mindnode/resources/0F278DB4-4103-44BA-BCBF-EB702C63D097.png)
+
+# Background jobs
+- Resque
+	- Single threaded process
+- Delayed Job
+	- Single process
+- Sidekiq
+	- Redis and multiple threaded process
+      ```ruby
+        while(!empty(queue)) {
+		       = get(queue); //从任务队列中取一个(涉及加锁等)
+		      q->callback(); //执行该任务
+          }
+      ```
+
+# Test
+- Unit Test
+	- Mini Test
+	- RSpec
+- Browser Test
+	- Capybara
+	- Cucumber
+- Mock
+	- Mocha
+
+# Auth
+- OmniAuth
+- Devise
+
+# Web Server
+- Thin
+- Unicorn
+- Puma
+- Passenger
+- Pow
+
+# Language
+- Syntax
+	- lambda
+	- yield
+	- respond_to
+	- send
+	- <=>
+		- return 1,0, -1
+	- * before array
+		- splat operator: split an array into a list of arguments
+	- array << obj
+		- append obj to new arry
+- block {} do end
+	- Block is not object
+	- use yield pass block
+- Proc
+	- Proc is object
+	- greeting = Proc.new { puts "hello, world" }  greeting.call
+- Module
+	- Can't be new
+	- include
+	- can't extend
+- Class
+
+# Code analysis
+- Rubocop
+
+# GC
+- Minor GC
+- Major GC
+- GC.stat
+
+# Bundler
+
+## Use groups in bundler
+
+```ruby
+# These gems are in the :default group
+gem 'nokogiri'
+gem 'sinatra'
+
+gem 'wirble', :group => :development
+
+group :test do
+  gem 'faker'
+  gem 'rspec'
+end
+
+group :test, :development do
+  gem 'capybara'
+  gem 'rspec-rails'
+end
+
+gem 'cucumber', :group => [:cucumber, :test]
+```
+when you install bundles, you can do
+
+```bash
+bundle install --without test development
+```
+# Web frameworks
+## Sintra
+
+## Ruby on Rails
+
+### How Rails application works
+
+![Rails flow](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/01193273-3E02-4AB0-ADE9-037F1577CCB0.png)
+
+	- ORM
+		- Active record
+		- Mongoid
+	- Websocket
+	- Assets pipeline
+	- Rails console
+
+# Gem mirrors
+- [http://mirrors.aliyun.com/rubygems/](http://mirrors.aliyun.com/rubygems/)
+- [https://gems.ruby-china.org](https://gems.ruby-china.org)
@@ -0,0 +1,135 @@
+---
+title: Software Design
+created: 2020-04-08
+updated: 2020-04-08
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/software-design.md
+---
+
+
+## Refactoring
+
+- when to refactor
+
+	- long method
+	- too many parameters
+	- duplicated code
+	- large class
+	- too many if else or case when
+	- temporary variable
+	- coupling
+
+## OOP
+
+Object-oriented programming (or OOP) is a paradigm or pattern of programming whereby the solution to a programming problem is modelled as a collection of collaborating objects. Objects collaborate by sending messages to each other. It is most suitable for managing large, complex problems.
+
+The four principles of object-oriented programming are
+- `encapsulation` Object encapsulates data and the functions that operate on that data
+- `abstraction` You don't need to what happened under the hood, just know how to use the public methods is enough.
+- `inheritance`  Sharing common behaviours
+- `polymorphism` `Polymorphism means that when an object receives a message, the correct method is called, based on the object’s class. That method may belong to the parent, or it may be one that is customized for this class.`
+
+- Polymorphism
+	- Code to interface, not an implementation
+
+- Composition over inheritance
+
+	- objects have high cohesion and low coupling with other objects
+-  Duck typing
+
+- SOLID
+
+	- Single responsibility
+	- Open/Closed
+	- Liskov substitution
+	- Interface segregation
+	- Dependency inversion
+
+## Functional programming
+
+- pure functions
+
+	- giving a same input, always return same output
+	- produce no side effects
+	- reply on no external mutable state
+- function composition
+- avoid share state
+- avoid mutating state
+- avoid side effects
+
+
+## Serverless
+
+- Lambda
+
+	- A **lambda** is just an anonymous function - a function defined with no name
+
+## Closure
+
+- a function value that references variables from outside its body
+	- In Javascript, not all anonymous function are lamdba, and vise verse
+
+## Event Sourcing
+
+### Kafka
+
+Apache Kafka is a distributed streaming platform
+
+![Kafka APIs](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/kafka-apis.png)
+
+There are 4 core APIs in kafka:
+
+* Producer API
+* Consumer API
+* Streams API
+* Connector API
+
+## Command Query Responsibility Segregation (CQRS)
+
+Every changes you made to the system is a event, events are been persisted and can be replayed to generate the materialized view.
+
+In his book "Object Oriented Software Construction," Betrand Meyer introduced the term "Command Query Separation" to describe the principle that an object's methods should be either commands or queries. A query returns data and does not alter the state of the object; a command changes the state of an object but does not return any data. The benefit is that you have a better understanding what does, and what does not, change the state in your system.
+
+**More**:
+
+- [Reference 2: Introducing the Command Query Responsibility Segregation Pattern](https://msdn.microsoft.com/en-us/library/jj591573.aspx)
+
+## UML
+
+- many_to_many
+- one to many
+
+![example](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/18F27D1D-F7CB-4E8B-BE47-90847CAEBC91.png)
+
+### One UML example
+
+![uml](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/656E28B3-4CCA-40F8-A95A-0F901E61AFA1.png)
+
+
+## Java Design pattern
+
+- [https://github.com/iluwatar/java-design-patterns](https://github.com/iluwatar/java-design-patterns)
+
+
+## How to do Data Migration
+
+1. Add logic to support both new and old data structures
+
+2. Add Logic should read both old and new data but only write data to new schema/structure
+
+2. Then migrate the old data schema to new one
+
+3. Last, delete the logic which handle the old data schema.
+
+
+## Some tips
+
+- concurrency vs parallelism
+
+	- parallelism
+		- doing a lot of things at once
+		- about exection
+	- concurrency
+		- dealing with lots of things at once
+		- about structure
@@ -0,0 +1,84 @@
+---
+title: Team Work
+created: 2018-09-06
+updated: 2018-09-06
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/team-work.md
+---
+
+# Agile
+
+- Retrospective
+- Iteration meeting
+- Showcase
+- Code review
+- Pair programming
+- Instant feedback
+
+## Retrospective
+
+### Steps
+* Review actions of last retro
+* Draw a timeline of events happy between last retro and now (Optional, some people say this can help team to get clear mind about what happened and what are the important things)
+* Choose the retro format. The most simple and popular one is(good, bad, questions)
+* Around 5 minutes to write down your idea
+* Quick go through all the cards and group the cards
+* Vote cards
+* Pick the cards which get most votes and talk about them
+* Write down the action is requires to
+
+
+## Good practices
+
+### In the team
+
+* Regular team one on one feedback session. Every pair have around 5 minutes, 2 minutes to come up with ideas and 3 minutes to talk. Then one person switch to another persion.
+* Regular team retrospective, each retro should check the previous actions, and have some actions as outputs.
+* Daily standup
+* Have some story wall, no matter physical or on the website.
+* Team health check survey and metrics report. About how do people feel about the team? How to people like to work in team? Any obstacles?
+* Regular team outing/ team building. The simplest one is the team lunch, so people can know more to each other.
+* Regular team technology people catch up. Talk about the tech decisions, tech debts and improvments.
+
+### Among teams/Company
+
+* Regular whole office stand up, one people stands for each team.
+* Regular office update, Q&A
+* Regular office tech people meeting.
+
+
+# Kanban
+
+# Tools
+
+- ThoughtWorks Tech Radar
+- CI
+- CD
+- Gantt chart
+- monitoring/debuging
+    - NewRelic
+    - Sentry
+    - Google analytics
+    - Prometheus
+        - cadvisor
+            - get metrics of containers
+        - alertmanager
+            - alert
+        - node exporter
+            - exporters for machine metrics
+        - grafana
+            - UI charts analytics
+        - docker-compose
+            - [https://github.com/vegasbrianc/prometheus](https://github.com/vegasbrianc/prometheus)
+    - bosun
+        - components
+            - opentsdb
+            - cadvisor
+- Cross-functional team
+	- One role can do others works
+		- Dev can do test
+		- QA can write BDD and automation tests
+	- One Delivery team
+	- High spped
+		- Team members can address problems more quickly
@@ -0,0 +1,33 @@
+---
+title: Testing
+created: 2018-06-13
+updated: 2018-06-13
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/testing.md
+---
+
+# Testing
+
+## Types of testing methodologies
+
+- unit test
+- integration test
+- contract test
+- smoke test
+- functional test
+	- selenium
+	- cucumber
+
+# Test layers
+
+![Good tests layers](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/test_layers.png)
+
+
+## Functional test
+
+### Selenium
+
+#### How does selenium works
+
+![selenium](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/Knowledge.mindnode/resources/3D938731-E99C-4E75-B2A2-B4B96870EFAB.png)
@@ -0,0 +1,18 @@
+---
+title: Uncategorised
+created: 2018-06-12
+updated: 2018-06-12
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/uncategorised.md
+---
+
+# Uncategorised
+
+
+## Some interesting questions
+
+### UUID or UUID4
+
+* `UUID` use current time and your computer MAC address to generate the id
+* `UUID4` which is much simpler. Each and every bit of a UUID v4 is generated randomly and with no inherent logic. With the sheer number of possible combinations (2^128), it would be almost impossible to generate a duplicate unless you are generating trillions of IDs every second, for many years.
@@ -0,0 +1,49 @@
+---
+title: Web Scraping
+created: 2022-04-07
+updated: 2022-04-07
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/web-scraping.md
+---
+
+# Web Scraping
+
+## A good scraper
+
+- content downloading, download images, contents and so on.
+- links retrieve, find new links
+- URL management, avoid endless loop
+- content analysis and management, export as csv or chart.
+
+## good frameworks
+
+- colly
+	- https://github.com/asciimoo/colly
+- pyspider
+	- [http://docs.pyspider.org/en/latest/](http://docs.pyspider.org/en/latest/)
+- scrapy
+	- [https://github.com/scrapy/scrapy](https://github.com/scrapy/scrapy)
+
+## anti scraper blocking
+
+
+- Ddon't use default user-agent, instead use real browser user agent
+- Proper interval, normally means reduce your interval
+- multiple ips
+- different headers
+- dynamic pages -> prentend to be a real man. /phantom.js/selenium/chrome headless
+- set proper cookie
+- verification code -> ocr
+- fetch data from mobile webpage
+
+## anti scraper
+
+- block ip
+- http headers, such as user-agent
+- cookie
+- return fake data
+
+## How does web scraper work
+
+TODO
@@ -0,0 +1,166 @@
+---
+title: Web
+created: 2018-06-14
+updated: 2018-06-14
+type: summary
+tags: [tech, reference]
+external: https://github.com/wahyd4/knowledge/blob/master/categories/web.md
+---
+
+# HTTP
+
+## HTTP Status code
+
+* `200` OK
+* `201` created
+* `204` No content, The server has successfully fulfilled the request and that there is no additional content to send in the response payload body.
+* `301` Moved Permanently
+* `400` Bad request, the request body syntactically wrong
+* `401` Unauthorized, generally means you didn't login or specify the credential
+* `403` Forbidden, we know who you are, but you just don't have enough rights
+* `415` Unsupported Media Type，the content type consumer passed in is not valid
+* `422` Unprocessable Entity, means the request body is syntactically correct but semantically incorrect
+* `500` Internal error, some unknown or unhandled error happened from the server side
+* `502` Bad Gateway, The server, while acting as a gateway or proxy, received an invalid response from an inbound server it accessed while attempting to fulfill the request.
+* `503` Service Unavailable, The server is currently unable to handle the request due to a temporary overload or scheduled maintenance, which will likely be alleviated after some delay.
+* `504` Gateway timeout, The server, while acting as a gateway or proxy, did not receive a timely response from an upstream server it needed to access in order to complete the request.
+
+
+## HTTP 2
+### HTTP server push
+
+ HTTP/2 Push allows a web server to send resources to a web browser before the browser gets to request them. It is, for the most part, a performance technique that can help some websites load faster.
+
+ HTTP/2 Push is not a mechanism for the server to notify things to the browser. Instead, pushed contents are used by the browser when it may have otherwise produced a request to get the resource anyway. But if the browser does not request the resource, the pushed contents become wasted bandwidth.
+
+Server push some related resource before the browser sends the request. .e.g. Browser request index.html, and then server response with index.html and also take the initiative to push style.css and index.js to frontend.
+
+Frames:
+
+* HEADERS
+* PUSH_PROMISE
+* DATA
+* RST_STREAM
+
+
+## HTTP server-sent events
+
+
+### Links
+* [A Comprehensive Guide To HTTP/2 Server Push](https://www.smashingmagazine.com/2017/04/guide-http2-server-push/)
+
+
+# CSS3
+
+- transition
+- transaction
+- border-radius
+- rgba
+- flex
+- box-shadow
+- svg
+
+# CORS
+
+-  Nginx proxy
+- Server add headers
+
+# CDN
+
+- Reduce request amount
+- Reduce server network traffic
+- Speed up load
+
+# Websocket
+
+WebSocket is a computer communications protocol, providing full-duplex communication channels over a single TCP connection. The WebSocket protocol was standardized by the IETF as RFC 6455 in 2011, and the WebSocket API in Web IDL is being standardized by the W3C. WebSocket is a different TCP protocol from HTTP.
+
+## Frameworks and tools
+
+- Pusher
+- Socket.IO
+- Pubnub
+
+## How Websocket works
+
+![Diagram](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/websocket-diagram.png)
+
+
+# json-schema
+
+- Describe data format
+- Clear, machine and human readable
+- Complete structure validation
+
+# Auth Related
+
+- session
+	- store session_id in cookie, server side has a map in memory
+- JWT Token
+	- expire
+- cookie
+- local storage
+
+# Browser
+
+## How browser works?
+
+How browser render web page
+
+![how browser render web page](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/how-browser-works.png)
+
+The flow diagram when you access a website
+
+![Flow](https://raw.githubusercontent.com/wahyd4/knowledge-mind-mapping/master/assets/images/http-flow.jpg)
+
+# CSS
+
+##  tips
+
+  * inline element can’t set width, modify the display to inline-block
+
+## inline element
+	* i
+	* span
+	* a
+	* image
+	* label
+	* input
+	* button
+	* textarea
+	* br
+
+## CSS custom properties(variables)
+
+```css
+# global variable
+:root {
+  font-size: 16px;
+}
+element {
+  --main-bg-color: brown;
+}
+```
+Then you use it like:
+
+```css
+  element {
+    background-color: var(--main-bg-color);
+  }
+```
+
+Some advanced usage, you can have default value when variable is not set
+
+```css
+.two {
+  color: var(--my-var, red); /* Red if --my-var is not defined */
+}
+
+.three {
+  background-color: var(--my-var, var(--my-background, pink)); /* pink if my-var and --my-background are not defined */
+}
+
+.three {
+  background-color: var(--my-var, --my-background, pink); /* Invalid: "--my-background, pink" */
+}
+```