home/knowledge
2
0
Fork 0
mirror of https://github.com/wahyd4/knowledge.git synced 2026-08-09 05:06:28 +10:00
Code Issues Packages Projects Releases Wiki Activity

adding more pages

Browse Source
This commit is contained in:
junv
2018-01-31 17:57:24 +11:00
parent 022a89a208
commit fc3a0660db
13 changed files with 939 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
-2
View File
@@ -2,7 +2,5 @@
A mind mapping chart to record the things that I learned.
The File format is in [MindNode](https://mindnode.com/)
## Preview
_sidebar.md
+9 -9
View File
@@ -2,15 +2,15 @@
- [Home](/)
- [Go](categories/go.md)
- [Java](golang.md)
- [JavaScript](golang.md)
- [Ruby](golang.md)
- [Interview](golang.md)
- [Software Design](golang.md)
- [Message Queue](golang.md)
- [Database](golang.md)
- [Application](golang.md)
- [Web Scraping](golang.md)
- [Java](categories/java.md)
- [JavaScript](categories/javascript.md)
- [Ruby](categories/ruby.md)
- [Interview](categories/interview.md)
- [Software Design](categories/software-design.md)
- [Message Queue](categories/message-queue.md)
- [Database](categories/database.md)
- [Application](categories/application.md)
- [Web Scraping](categories/web-scraping.md)
- [Devops](categories/devops.md)
- [Web](categories/web.md)
- [Success Team Work](categories/team-work.md)
categories/application.md
+66
View File
@@ -0,0 +1,66 @@
## Search
- ElasticSearch
- Logstash
- Kibana
## APM
- Newrelic
- Pinpoint
## Monitor
- Bosun
- Prometheus
## API
- RPC(remote procedure call)
- disadvantages
- coupling
- clients needs to know the request procedure names
- advantages
- more freedoms to define any requests.
- examples - [https://api.slack.com/methods](https://api.slack.com/methods)
- Restful /Representational State Transfer
- Easy to cache
- every url represent a resource
- disadvantages
- some special method is hard to use restful to name it. like /login, /resetpassword
- examples
- [https://developer.github.com/v3/](https://developer.github.com/v3/)
- methods
- GET
- POST
- PUT - DELETE
- GraphQL - Can define fields you want
- gRPC
- protocol buffers
```go
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
```
- A opensource remote procedure call framework based on HTTP2 and protobuf
- advantages
- baed on HTTP2, better performance
- call remote like local method, and support multiple language
- protobuf serialization and deserialization is faster than JSON
## Log and Error
- Sentry
categories/database.md
+41
View File
@@ -0,0 +1,41 @@
## Relation
- Mysql
- Postgres
- Microsoft SQL Server
- Oracle
- Sqlite
## NoSql
- Monodb
- Replica Set
- Primary
- Secondary
- Hidden
- Redis
- Cluster
- Strong recommend 3 masters and 3 slaves
- Hbase
- Cassendra
## SQL
- Index
- Index is unique but not mandatory
- join
- MYSQL
- tips
- datetime
- 8 bytes
`1000-01-01 00:00:00 ~ 9999-12-31 23:59:59`
- store a absolute value
- timestamp
- 4 bytes `1970-01-01 08:00:01 ~ 2038-01-19 11:14:07`
- will impacted by time zone
- index by timestamp will be faster than datetime due to 4 bytes.
## Message Queue
## Rabbitmq
## activeMQ
## Redis
categories/devops.md
+155
View File
@@ -0,0 +1,155 @@
## AWS
- EC2
- EC2 container
- Cloudfront
- S3
- RDS
## CI
- Jenkins
- Gitlab CI
- Travis
## Docker
- Orchestration
- Rancher
- Kubernetes
- Pod
- Single container
- Multiple containers
- share ip
- share volumes
- Node
- Service
- ClusterIP (default)
- NodePort （ access by port）
- LoadBalancer (external ip)
- ExternalName dns
- Deployment
- Replica Set
- Service rolling update
- DaemonSet
- running a cluster storage daemon, such as glusterd, ceph, on each node
- running a logs collection daemon on every node, such as fluentd or logstash.
- running a node monitoring daemon on every node, such as Prometheus Node Exporter, collectd, New Relic agent, or Ganglia gmond.
- StatefulSets
- Ordered, graceful deployment and scaling
- Stable, persistent storage.
- Stable, unique network identifiers.
- Ordered, graceful deletion and termination.
- Proxy
- DNS
- service.namespace
- Secrets
- Persistent volumes
- Available
- a free resource that is not yet bound to a claim
- Bound
- the volume is bound to a claim
- Released
- the claim has been deleted, but the resource is not yet reclaimed by the cluster
- Failed
- the volume has failed its automatic reclamation
- benefits
- escape the app dependency matrix
- Solve the environment issue. it works on my machine.
- Network
- Bridge
- Host
- Tips
- difference between cmd and entrypoint
- CMD
- has three forms
- CMD ["executable","param1","param2"]
- exec form, preferred
- CMD ["param1","param2"]
- as default parameters to ENTRYPOINT
- CMD command param1 param2
- shell form
- The main purpose of a CMD is to provide defaults for an executing container. These defaults can include an executable, or they can omit the executable, in which case you must specify an ENTRYPOINT instruction as well.
- If CMD is used to provide default arguments for the ENTRYPOINT instruction, both the CMD and ENTRYPOINT instructions should be specified with the JSON array format.
- docker run could override the default specified in CMD.
- ENTRYPOINT
- docker run arguments will as the entrypoint’s parameters. not CMD’s override
- has two forms
- ENTRYPOINT ["executable", "param1", "param2"]
- exec form, preferred
- ENTRYPOINT command param1 param2
- shell form
- entrypoint setting can be override by docker run —entrypoint
- Security
- https://docs.docker.com/engine/security/security/
- use non root user
- docker daemon attack surface
- kernel security features
## Linux
- ssh
- IP table
- Cron job
- ps aux
- tail
- nohup
- df -h
- free
- Tips
- Change timezone
- TZ=Asia/Shanghai;ln -snf /usr/share/zoneinfo/$TZ /etc/localtime
- Check release
- cat /etc/*-release
- Packages
- htop
- like top, but more
- mosh
- auto reconnect ssh
## Everything is code
- infrastructure as code
- CI config as code
- Dockerfile
## CD
- Capistrano
- Ansible
- Puppet
## HA High Availability
- LB - haproxy
- Keepalived -- vrrp
- Nginx + Keepalived
- Heartbeat
## Network
- IP
- Private IP Addresses
- 10.0.0.0 – 10.255.255.255
- 172.16.0.0 – 172.31.255.255
- 192.168.0.0 – 192.168.255.255
## tools
- git
- cherrypick
- rebase
- commit
- checkout
- checkout -b br com
- push
- --tags
- push tags
- —all
- push all branchs
- -f
- force push
- branch -d xx
- delete a local branch
- remote
- show xxx
- show info of a remote branch
- add xxx
- add a remote repo
- branch
- list all the branchs
- oh-my-zsh
- IDE/tools
- Intellij
- vscode
categories/interview.md
+3
View File
@@ -0,0 +1,3 @@
# resources
- [https://github.com/kdn251/interviews](https://github.com/kdn251/interviews)
categories/java.md
+138
View File
@@ -0,0 +1,138 @@
## Java GC
- GC types
- Serial GC
- Parallel GC
- CMS GC
- G1 GC
- Object type
- Young generation minor GC
- Old generation full GC
- System.gc()
## JVM
- Method Area
- Class name, Constant, static Constant, Class object
- Heap
- Object instance, array
- young generation
- Old generation
- VM stack
- local variable
- dynamic links
- operation stacks
- Native method stack
- program counter
## Spring
- Spring boot
- Rest controller
- Jpa template
- Spring scheduler
- Annotation bean
- Spring cloud
- Consol
- Zuul
- Cloud config
- Spring cloud zookeeper( discovery and configuration)
- Spring ecurity
## ORM
- Hibernate
- validation
- JPA
- Sprint jpa tempalte
- Mybatis
## Build
- Gradle
- groovy task
- Maven
- XML plugins
- Maven mirror
- [https://maven-central.storage.googleapis.com](https://maven-central.storage.googleapis.com)
- [http://maven.aliyun.com/nexus/content/groups/public/](http://maven.aliyun.com/nexus/content/groups/public/)
- Ant
- task based
## Useful tips
- Iterate map
- for (Map.Entry<String, String> entry : map.entrySet()) { }
- List to array
- int[] arr= new int[list.length](); list.toArray(arr);
- Object
- immutable
- object that can’t be modified after it’s created
- Don’t allow subclass override methods, class to be final
- Don’t provide setter method
- Make all field private and final
- mutable
- Default Timezone
- TimeZone.setDefault(TimeZone.getTimeZone("GMT+8"))
- [generics](https://appliedgo.net/generics/)
- Why we need?
- e.g we have sort method, and different type(n) of objects. then you have to write n kind of search methods.
- event listener
- [https://stackoverflow.com/questions/6270132/create-a-custom-event-in-java](https://stackoverflow.com/questions/6270132/create-a-custom-event-in-java)
## JAVA se
- modifier
- public
- no modifier
- protected
- priivate
- NIO
- Channels
- A channel is a kind of stream, from the channel data can be read into a buffer.
- channel implementations
- FileChannel
- reads data from and to files
- DatagramChannel
- read and write data over the network via UDP
- SocketChannel
- read and write data over the network visa TCP
- ServerSocketChannel
- allows you to listen for incoming TCP connections, like a web server. For each incoming connection a SocketChannel is created.
- Buffer
- code sample
```java
RandomAccessFile aFile = new RandomAccessFile("data/nio-data.txt", "rw");
FileChannel inChannel = aFile.getChannel();
//create buffer with capacity of 48 bytes
ByteBuffer buf = ByteBuffer.allocate(48);
**int bytesRead = inChannel.read(buf**); //read into buffer.
while (bytesRead != -1) {
**buf.flip**(); //make buffer ready for read
while(buf.hasRemaining()){
System.out.print((char) **buf.get**()); // read 1 byte at a time
}
**buf.clear**(); //make buffer ready for writing
bytesRead = inChannel.read(buf);
}
aFile.close();
```
- properties
- capacity
- position
- limit
- Steps for buffer to read and write data.
- write data into buffer
- call buffer.flip()
- read data out of the buffer
- call buffer.clear() or buffer.compact()
- Non-blocking IO
- Selectors
- A selector allows a single thread to handle multiple Channels
- [http://tutorials.jenkov.com/java-nio/index.html](http://tutorials.jenkov.com/java-nio/index.html)
categories/javascript.md
+242
View File
@@ -0,0 +1,242 @@
## ES 6/ES 2015
- Arrow function
- Array function doesn’t create new this, it grabs this from its surrounding instead.
- Array/Collection methods
- map
```javascript
const numbers = [0, 1, 2, 3, 4, 5, 6];
const doubledNumbers = numbers.map(n => n * 2);
```
- filter
- const evenNumbers = numbers.filter(n => n % 2 === 0);
- reduce
```javascript
const sum = numbers.reduce(
function(acc, n) {
return acc + n;
},
0 // accumulator variable value at first iteration step
);
```
- reduce takes two parameters
The first parameter is a function that will be called at each iteration step.The second parameter is the value of the accumulator variable (*acc* here) at the first iteration step (read next point to understand)
- Class
- Template String
- const name = "Nick";
`Hello ${name}, the following expression is equal to four : ${2+2}`;
- Destructing
- Array
- const myArray = ["a", "b", "c"];
//without const x = myArray[0];
const y = myArray[1];
const [x, y] = myArray;
- Object
```javascript
const person = {
firstName: "Nick",
lastName: "Anderson",
age: 35,
sex: "M"
}
const { firstName: first, age, city = "Paris" } = person;
```
- Default + Rest + Spread parameters
- Const
- can't be reassigned; but not immutable
- Let
- Modules
- Map / Set
- Promise
- .then().catch()
```javascript
function getGithubUser(username) {
return new Promise((resolve, reject) => {
fetch(`https://api.github.com/users/${username}`)
.then(response => {
const user = response.json();
resolve(user);
})
.catch(err => reject(err));
})
}
```
- Enhanced Object Literals
- Imports/ Exports
- export default xx; import xx from ‘..’
- export const xx; import {xx} from ‘..’
- Async/Await
- await can only used in an async function.
- async function getGithubUser(username) { // async keyword allows usage of await in the function and means function returns a promise
try { // this is how errors are handled with async / await
const response = await fetch(`https://api.github.com/users/${username}`); // "synchronously" waiting fetch promise to resolve before going to next line
return response.json();
} catch (err) {
alert(err);
}
}
getGithubUser('mbeaudru').then(user => console.log(user)); // logging user response - cannot use await syntax since this code isn't in async function
## Node.js
- Npm
- Yarn
- Express
- Stream
## This
- Refers who calls
```javascript
function myFunc() {
...
}
// After each statement, you find the value of *this* in myFunc
myFunc.call("myString", "hello") // "myString" -- first .call parameter value is injected into *this*
// In non-strict-mode
myFunc("hello") // window -- myFunc() is syntax sugar for myFunc.call(window, "hello")
// In strict-mode
myFunc("hello") // undefined -- myFunc() is syntax sugar for myFunc.call(undefined, "hello")
var person = {
myFunc: function() { ... }
}
person.myFunc.call(person, "test") // person Object -- first call parameter is injected into *this*
person.myFunc("test") // person Object -- person.myFunc() is syntax sugar for person.myFunc.call(person, "test")
var myBoundFunc = person.myFunc.bind("hello") // Creates a new function in which we inject "hello" in *this* value
person.myFunc("test") // person Object -- The bind method has no effect on the original method
myBoundFunc("test") // "hello" -- myBoundFunc is person.myFunc with "hello" bound to *this*
```
- [http://yehudakatz.com/2011/08/11/understanding-javascript-function-invocation-and-this/](http://yehudakatz.com/2011/08/11/understanding-javascript-function-invocation-and-this/)
## Closure
## Frameworks
- Ionic
- Angular
- Vue.js
- component
- .sync modifier, A kind of two way binding
- :abc.sync=“”; $tis.emit(“update:abc”, newValue) //trigger in child component
- computed
- computed attributes
- computed over watcher
- watchers
- Monitor data changes
- cheat sheet
- https://vuejs-tips.github.io/cheatsheet/
- Ember.js
- Backbone
-
- React.js
- Flux/ Redux
- State
- Store
- Reducer
- Action
- Dispatch
## Use promise to avoid nested callbacks.
## tips
- object literal
- ```javascript
Object.keys(object).map(function(objectKey, index) {
var value = object[objectKey];
console.log(value);
});
```javascript
## Application
## Search
- ElasticSearch
- Logstash
- Kibana
## APM
- Newrelic
- Pinpoint
## Monitor
- Bosun
- Prometheus
## API
- RPC(remote procedure call)
- disadvantages
- coupling
- clients needs to know the request procedure names
- advantages
- more freedoms to define any requests.
- examples
- [https://api.slack.com/methods](https://api.slack.com/methods)
- Restful /Representational State Transfer
- Easy to cache
- every url represent a resource
- disadvantages
- some special method is hard to use restful to name it. like /login, /resetpassword
- examples
- [https://developer.github.com/v3/](https://developer.github.com/v3/)
- methods
- GET
- POST
- PUT
- DELETE
- GraphQL
- Can define fields you want
- gRPC
- protocol buffers
```javascript
// The greeter service definition.
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {}
}
// The request message containing the user's name.
message HelloRequest {
string name = 1;
}
// The response message containing the greetings
message HelloReply {
string message = 1;
}
```javascript
- A opensource remote procedure call framework based on HTTP2 and protobuf
- advantages
- baed on HTTP2, better performance
- call remote like local method, and support multiple language
- protobuf serialization and deserialization is faster than JSON
## Log and Error
- Sentry
categories/message-queue.md
+5
View File
@@ -0,0 +1,5 @@
## Rabbitmq
## activeMQ
## Redis
categories/ruby.md
+80
View File
@@ -0,0 +1,80 @@
## Background jobs
- Resque
- Single threaded process
- Delayed Job
- Single process
- Sidekiq
- Redis and multiple threaded process
```ruby
while(!empty(queue)) {
= get(queue); //从任务队列中取一个(涉及加锁等)
q->callback(); //执行该任务
}
```
## Test
- Unit Test
- Mini Test
- RSpec
- Browser Test
- Capybara
- Cucumber
- Mock
- Mocha
## Auth
- OmniAuth
- Devise
## Web Server
- Thin
- Unicorn
- Puma
- Passenger
- Pow
## Language
- Syntax
- lambda
- yield
- respond_to
- send
- <=>
- return 1,0, -1
- * before array
- splat operator: split an array into a list of arguments
- array << obj
- append obj to new arry
- block {} do end
- Block is not object
- use yield pass block
- Proc
- Proc is object
- greeting = Proc.new { puts "hello, world" } greeting.call
- Module
- Can't be new
- include
- can't extend
- Class
## Code analysis
- Rubocop
## GC
- Minor GC
- Major GC
- GC.stat
## Web frameworks
- Sintra
- Ruby on Rails
- ORM
- Active record
- Mongoid
- Websocket
- Assets pipeline
- Rails console
## Gem mirrors
- [http://mirrors.aliyun.com/rubygems/](http://mirrors.aliyun.com/rubygems/)
- [https://gems.ruby-china.org](https://gems.ruby-china.org)
categories/software-design.md
+93
View File
@@ -0,0 +1,93 @@
## Refactoring
- when to refactor
- long method
- too many parameters
- duplicated code
- large class
- too many if else or case when
- temporary variable
- coupling
## Microservice
- benefits
- Each application is relatively small
- Easy to scale
- Improved fault isolation
- One service fails won’t influence others
- You can use different stack for each service
- Each service can be deployed independently
- drawbacks
- Increased memory consumption
- Developers must deal with extra complexity of creating a distributed system
- Deployment complexity
- To deploy different types of services
## OOP
- Polymorphism
- Code to interface, not an implementation
- Composition over inheritance
- objects have high cohesion and low coupling with other objects
- Duck typing
- SOLID
- Single responsibility
- Open/Closed
- Liskov substitution
- Interface segregation
- Dependency inversion
## Functional programming
- pure functions
- giving a same input, always return same output
- produce no side effects
- reply on no external mutable state
- function composition
- avoid share state
- avoid mutating state
- avoid side effects
## Serverless
- Lambda
- A **lambda** is just an anonymous function - a function defined with no name
## Closure
- a function value that references variables from outside its body
- In Javascript, not all anonymous function are lamdba, and vise verse
## UML
- many_to_many
- one to many
## Java Design pattern
- [https://github.com/iluwatar/java-design-patterns](https://github.com/iluwatar/java-design-patterns)
## some tips
- concurrency vs parallelism
- parallelism
- doing a lot of things at once
- about exection
- concurrency
- dealing with lots of things at once
- about structure
categories/web-scraping.md
+32
View File
@@ -0,0 +1,32 @@
## A good scraper
- content downloading, download images, contents and so on.
- links retrieve, find new links
- URL management, avoid endless loop
- content analysis and management, export as csv or chart.
## good frameworks
- colly
- https://github.com/asciimoo/colly
- pyspider
- [http://docs.pyspider.org/en/latest/](http://docs.pyspider.org/en/latest/)
- scrapy
- [https://github.com/scrapy/scrapy](https://github.com/scrapy/scrapy)
## anti scraper blocking
- multiple ips
- different headers
- proper interval
- dynamic pages -> prentend to be a real man. /phantom.js/selenium/chrome headless
- set proper cookie
- verification code -> ocr
- fetch data from mobile webpage
## anti scraper
- block ip
- http headers, such as user-agent
- cookie
- return fake data
categories/web.md
+75
View File
@@ -0,0 +1,75 @@
## HTML
- HTML5
- audio
- video
- drag and drop
- more input type
- more elements
- section
- article
- footer
- header
- localstorage
- websocket
- canvas
- HTTPS
## CSS3
- transition
- transaction
- border-radius
- rgba
- flex
- box-shadow
- svg
## CORS
- Nginx proxy
- Server add headers
## CDN
- Reduce request amount
- Reduce server network traffic
- Speed up load
## Websocket
- Pusher
- Socket.IO
## json-schema
- Describe data format
- Clear, machine and human readable
- Complete structure validation
## Auth Related
- session
- store session_id in cookie, server side has a map in memory
- JWT Token
- expire
- cookie
- local storage
## Browser
## CSS
- tips
- inline element can’t set width, modify the display to inline-block
- inline element
* i
* span
* a
* image
* label
* input
* button
* textarea
* br