[Kubernetes] Monitoring & Logging

 

 

Prometheus Monitoring

: CPU, Memoty, Network IO, Disk IO 모니터링

• Heapster(k8s 자체 모니터링 도구) + InfluxDB
• metrics-server : DB 없음 → 실시간으로만 확인가능
    -  CPU, Memory만 측정 가능
• Prometheus (k8s의 표준이라고 할 수 있음) → soundcloud가 개발

 

 

https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack

GitHub - prometheus-community/helm-charts: Prometheus community Helm charts

 

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

 

prom-value.yaml

grafana:
  service:
    type: LoadBalancer

 

* 별도의 name space를 만들어서 분리시켜주는게 좋음

kubectl create ns monitor

 

helm install prom prometheus-community/kube-prometheus-stack -f prom-values.yaml -n monitor

 

 

모니터링 웹브라우저 
http://192.168.100.24X
ID: admin
PWD: prom-operator

 

 

EFK Logging

ELK Stack : Elasticsearch + Logstash (Java) + Kibana
EFK Stack : Elasticsearch + Fluentd (Ruby) + Kibana
                     Elasticsearch + Fluent Bit (C) + Kibana
Elastic Stack : Elasticsearch + Beat + Kibana

 

 

Elasticsearch

:  데이터 저장소에서 검색하는 검색 엔진

helm repo add elastic https://helm.elastic.co
helm repo update

 

helm show values elastic/elasticsearch > es-value.yaml

es-value.yaml

 18 replicas: 1
 19 minimumMasterNodes: 1
 
 80 resources:
 81   requests:
 82     cpu: "500m"
 83     memory: "1Gi"
 84   limits:
 85     cpu: "500m"
 86     memory: "1Gi"

 

kubectl create ns logging

 

helm install elastic elastic/elasticsearch -f es-value.yaml -n logging

 

Fluent Bit

: 로그 수집기

https://github.com/fluent/fluent-bit-kubernetes-logging

GitHub - fluent/fluent-bit-kubernetes-logging: Fluent Bit Kubernetes Daemonset

git clone https://github.com/fluent/fluent-bit-kubernetes-logging.git

 

cd  fluent-bit-kubernetes-logging

 

kubectl create -f fluent-bit-service-account.yaml
kubectl create -f fluent-bit-role-1.22.yaml
kubectl create -f fluent-bit-role-binding-1.22.yaml

 

kubectl create -f output/elasticsearch/fluent-bit-configmap.yaml

 

output/elasticsearch/fluent-bit-ds.yaml

 32         - name: FLUENT_ELASTICSEARCH_HOST
 33           value: "elasticsearch-master"

 

kubectl create -f output/elasticsearch/fluent-bit-ds.yaml

 

Kibana

: data visualization

helm show values elastic/kibana > kibana-value.yaml

kibana-value.yaml

 49 resources:
 50   requests:
 51     cpu: "500m"
 52     memory: "1Gi"
 53   limits:
 54     cpu: "500m"
 55     memory: "1Gi"
 
119 service:
120   type: LoadBalancer

 

helm install kibana elastic/kibana -f kibana-value.yaml -n logging

 

http://192.168.100.X:5601

• 햄버거 → Management → Stack Management
  • Kibana → Index Pattern
    • Create Index Pattern 우상
      • Name: logstash-*
      • Timestamp: @timestamp
  • 햄버거 → Analystics → Discover