[설치] 07. Aggregating Container Logs

07. Aggregating Container Logs

클러스터 관리자는 EFK Stack으로 Openshift 내 다양한 서비스의 로그를 집계한다.

▶ ETK Stack이란?

• Elasticsearch: An object store where all logs are stored.

• Fluentd: Gathers logs from nodes and feeds them to Elasticsearch.

• Kibana: A web UI for Elasticsearch.

가. 사전준비

logging 프로젝트 생성에 앞서 없다면 생성, 있다면 다음 단계로 넘어간다.

[프로젝트 생성]

$ oadm new-project logging --node-selector=""
$ oc project logging

[node-selector 확인]

지정되 있을경우 특정 logging이 배포되는 위치가 제한되므로 지정하지 않는것을 권장

$ oc describe project logging
Name:			logging
Namespace:		<none>
Created:		19 hours ago
Labels:			<none>
Annotations:		openshift.io/description=
			openshift.io/display-name=
			openshift.io/node-selector=
			openshift.io/sa.scc.mcs=s0:c7,c4
			openshift.io/sa.scc.supplemental-groups=1000050000/10000
			openshift.io/sa.scc.uid-range=1000050000/10000
Display Name:		<none>
Description:		<none>
Status:			Active
Node Selector:		<none>
Quota:			<none>
Resource limits:	<none>

[계정 생성 및 권한 부여]

$ oc new-app logging-deployer-account-template
--> Deploying template "logging-deployer-account-template" in project "openshift"

     logging-deployer-account-template
     ---------
     Template for creating the deployer account and roles needed for the aggregated logging deployer. Create as cluster-admin.

--> Creating resources with label app=logging-deployer-account-template ...
    serviceaccount "logging-deployer" created
    serviceaccount "aggregated-logging-kibana" created
    serviceaccount "aggregated-logging-elasticsearch" created
    serviceaccount "aggregated-logging-fluentd" created
    serviceaccount "aggregated-logging-curator" created
    clusterrole "oauth-editor" created
    clusterrole "daemonset-admin" created
    rolebinding "logging-deployer-edit-role" created
    rolebinding "logging-deployer-dsadmin-role" created
--> Success
    Run 'oc status' to view your app.

$ oadm policy add-cluster-role-to-user oauth-editor \
 system:serviceaccount:logging:logging-deployer

$ oadm policy add-cluster-role-to-user cluster-reader \
 system:serviceaccount:logging:aggregated-logging-fluentd

$ oc create secret generic logging-deployer
secret "logging-deployer" created

$ oc edit scc privileged

~ 중략 ~
- system:serviceaccount:management-infra:inspector-admin
- system:serviceaccount:default:registry
- system:serviceaccount:openshift:logging-deployer

나. 설치

[Configration Master]

$ vi /etc/origin/master/master-config.yaml 

admissionConfig:
apiLevels:
- v1
apiVersion: v1
assetConfig:
  logoutURL: ""
  masterPublicURL: https://master1.ocp.com
  publicURL: https://master1.ocp.com/console/
  metricsPublicURL: "https://hawkular-metrics.app.ocp.com/hawkular/metrics"
  loggingPublicURL: "https://kibana.app.ocp.com"

~ 중략 ~

$ systemctl restart atomic-openshift-master.service

또는 

$ systemctl restart atomic-openshift-master-api.service
$ systemctl restart atomic-openshift-master-controllers.service

[logging 생성]

MASTER_URL 의 경우 HA Proxy 서버가 별도 존재할 경우 해당 hostname으로 변경하여 적용

$ oc new-app logging-deployer-template \
--param IMAGE_PREFIX=registry.access.redhat.com/openshift3/ \
--param KIBANA_HOSTNAME=kibana.app.ocp.com \
--param KIBANA_OPS_HOSTNAME=kibana-ops.app.ocp.com \
--param MASTER_URL=https://master1.ocp.com:443 \
--param PUBLIC_MASTER_URL=https://master1.ocp.com:443 \
--param INSECURE_REGISTRY=true \
--param ES_OPS_INSTANCE_RAM=1G \
--param MODE=install

실행과 동시에 컨테이너가 생성되는 화면

[확인 - View achive 버튼 활성화]

kibana 접속화면

다. 제거

$ oc new-app logging-deployer-template \
--param IMAGE_PREFIX=registry.access.redhat.com/openshift3/ \
--param KIBANA_HOSTNAME=kibana.app.ocp.com \
--param KIBANA_OPS_HOSTNAME=kibana-ops.app.ocp.com \
--param MASTER_URL=https://master1.ocp.com:443 \
--param PUBLIC_MASTER_URL=https://master1.ocp.com:443 \
--param INSECURE_REGISTRY=true \
--param ES_OPS_INSTANCE_RAM=1G \
--param MODE=uninstall