[kubernetes] 스케줄링 명령어 연습문제

[kubernetes] 쿠버네티스 목차

ㅁ 들어가며

  스캐줄링에 관련된 명령어 연습

## 목록 ##
메뉴열 스케줄링
Lable와 Selector,
Taint와 Tolerations
Node Affinity
Resource Limits
DaemonSets
Static Pod
Multiple Schedulers

 

 ㅁ Manual Scheduling, node를 지정하여 nginx 생성

$ cat nginx.yaml 
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  nodeName: node01
  containers:
  -  image: nginx
     name: nginx

$ k apply -f nginx.yaml 
pod/nginx created

 ㅇ node를 지정하거나 변경할 경우 nodeName을 수정하면 된다.

 

ㅁ Label을 이용한 Selector Scheduling

# pod 라벨 확인
$ k get po --show-labels
NAME          READY   STATUS    RESTARTS   AGE   LABELS
auth          1/1     Running   0          50s   bu=finance,env=prod

# 라벨 selector
$ kubectl get pods --show-labels  --selector env=dev
NAME          READY   STATUS    RESTARTS   AGE     LABELS
db-1-6zmhj    1/1     Running   0          2m43s   env=dev,tier=db


# 라벨 selector Count
$ kubectl get pods --selector env=dev --no-headers | wc -l

# 전체 Object 라벨
$ kubectl get all --selector env=prod --show-labels 
NAME              READY   STATUS    RESTARTS   AGE     LABELS
pod/auth          1/1     Running   0          5m53s   bu=finance,env=prod
pod/db-2-c2nkq    1/1     Running   0          5m53s   bu=finance,env=prod,tier=db
pod/app-1-zzxdf   1/1     Running   0          5m53s   bu=finance,env=prod,tier=frontend
pod/app-2-hx944   1/1     Running   0          5m54s   env=prod,tier=frontend

NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE     LABELS
service/app-1   ClusterIP   10.43.34.195   <none>        3306/TCP   5m53s   bu=finance,env=prod

NAME                    DESIRED   CURRENT   READY   AGE     LABELS
replicaset.apps/db-2    1         1         1       5m54s   env=prod
replicaset.apps/app-2   1         1         1       5m54s   env=prod

# 다중 라벨 검색
$ k get all --selector env=prod,bu=finance,tier=frontend
NAME              READY   STATUS    RESTARTS   AGE
pod/app-1-zzxdf   1/1     Running   0          8m2s

# 라벨 selector 리플리카셋 
$ cat replicaset.yaml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
   name: replicaset-1
spec:
   replicas: 2
   selector:
      matchLabels:
        tier: front-end
   template:
     metadata:
       labels:
        tier: front-end
     spec:
       containers:
       - name: nginx
         image: nginx 

# 적용
$ k apply -f replicaset-definition-1.yaml 
replicaset.apps/replicaset-1 created

 

 

ㅁ Taints And Tolerations 스케줄링

# node의 taint 확인하기
$ kubectl describe node node01 | grep -i taints

# taint 생성하기  key=value:effect
$ kubectl taint nodes node01 spray=mortein:NoSchedule

# 샘플 Pod 생성
$ k run mosquito --image=nginx
pod/mosquito created

# Pod 상태 확인, pending
$ k get po
NAME       READY   STATUS    RESTARTS   AGE
mosquito   0/1     Pending   0          44s

# Pending 이유 분석: taint에 걸려 스케줄이 되지 않음.
$ k describe po mosquito
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  103s  default-scheduler  0/2 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }, 1 node(s) had untolerated taint {spray: mortein}. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling..
  
  
# tolerations 추가 pod 생성
# 기본 yaml 생성
$ k run bee --image=nginx --dry-run=client -o yaml > bee.yaml
$ cat bee.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: bee
  name: bee
spec:
  containers:
  - image: nginx
    name: bee
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}


# tolerations 추가 편집
  tolerations:
  - key: spray
    value: mortein
    effect: NoSchedule
    operator: Equal

# Pod 생성
$ k apply -f bee.yaml 
pod/bee created

# 생성 확인
$ k get po
NAME       READY   STATUS    RESTARTS   AGE
bee        1/1     Running   0          32s
mosquito   0/1     Pending   0          9m7s


# taint 제거하기 생성문에 -를 붙임
$ k describe nodes controlplane | grep Taint
Taints:             node-role.kubernetes.io/control-plane:NoSchedule

$ k taint node controlplane node-role.kubernetes.io/control-plane:NoSchedule-
node/controlplane untainted

$ k describe nodes controlplane | grep Taint
Taints:             <none>


# mosquito 스케줄링 확인, controlplane의 taint가 제거되어 스켸줄됨.
$ k get po -o wide
NAME       READY   STATUS    RESTARTS   AGE     IP           NODE           NOMINATED NODE   READINESS GATES
bee        1/1     Running   0          3m42s   10.244.1.2   node01         <none>           <none>
mosquito   1/1     Running   0          12m     10.244.0.4   controlplane   <none>           <none>

 

ㅁ 노드 어피니티

노드 어피니티는 개념적으로 nodeSelector 와 비슷하며, 노드의 레이블을 기반으로 파드가 스케줄링될 수 있는 노드를 제한할 수 있다. 노드 어피니티에는 다음의 두 종류가 있다.

• requiredDuringSchedulingIgnoredDuringExecution: 규칙이 만족되지 않으면 스케줄러가 파드를 스케줄링할 수 없다. 이 기능은 nodeSelector와 유사하지만, 좀 더 표현적인 문법을 제공한다.
• preferredDuringSchedulingIgnoredDuringExecution: 스케줄러는 조건을 만족하는 노드를 찾으려고 노력한다. 해당되는 노드가 없더라도, 스케줄러는 여전히 파드를 스케줄링한다.

 

ㅁ Node Affinity 스케줄링

# 노드의 라벨 이쁘게 보는 방법
$ k get no node01 -o json | jq .metadata.labels
{
  "beta.kubernetes.io/arch": "amd64",
  "beta.kubernetes.io/os": "linux",
  "kubernetes.io/arch": "amd64",
  "kubernetes.io/hostname": "node01",
  "kubernetes.io/os": "linux"
}


# 노드 라벨 생성
$ kubectl label node node01 color=blue
node/node01 labeled


# 테스트용 deployment 생성
$ k create deployment blue --image nginx --replicas 3
deployment.apps/blue created


# 모든 노드에 스케줄됨.
$ k get po -o wide
NAME                    READY   STATUS    RESTARTS   AGE     IP           NODE           NOMINATED NODE   READINESS GATES
blue-5d9664c795-mqgf6   1/1     Running   0          7m39s   10.244.0.4   controlplane   <none>           <none>
blue-5d9664c795-rqbs2   1/1     Running   0          7m40s   10.244.1.2   node01         <none>           <none>
blue-5d9664c795-sl447   1/1     Running   0          7m39s   10.244.1.3   node01         <none>           <none>


# 선호하는 노드 어피니티를 사용해 파드 스케줄하기 
# 조건
NodeAffinity: requiredDuringSchedulingIgnoredDuringExecution
Key: color
value: blue

# 참조
# https://kubernetes.io/ko/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity
# https://kubernetes.io/ko/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/#선호하는-노드-어피니티를-사용해-파드-스케줄하기

# affinity 부분 추가
$ cat blue-deploy.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: blue
  name: blue
spec:
  replicas: 3
  selector:
    matchLabels:
      app: blue
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: blue
    spec:
      containers:
      - image: nginx
        name: nginx
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: color
                operator: In
                values:
                - blue  
                
# 노드 스케줄링 확인: node01으로 이동됨.
$ k get po -o wide
NAME                   READY   STATUS    RESTARTS   AGE   IP           NODE     NOMINATED NODE   READINESS GATES
blue-f69d4c887-bltzx   1/1     Running   0          12s   10.244.1.5   node01   <none>           <none>
blue-f69d4c887-mz4xq   1/1     Running   0          10s   10.244.1.6   node01   <none>           <none>
blue-f69d4c887-wwj5x   1/1     Running   0          14s   10.244.1.4   node01   <none>           <none>

# Node의 라벨에 메칭하여 affinity
# 라벨 확인
$ k get nodes controlplane -o json | jq .metadata.labels
{
  "beta.kubernetes.io/arch": "amd64",
  "beta.kubernetes.io/os": "linux",
  "kubernetes.io/arch": "amd64",
  "kubernetes.io/hostname": "controlplane",
  "kubernetes.io/os": "linux",
  "node-role.kubernetes.io/control-plane": "",
  "node.kubernetes.io/exclude-from-external-load-balancers": ""
}

# affinity yaml 작성
$ cat red.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: red
  name: red
spec:
  replicas: 2
  selector:
    matchLabels:
      app: red
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: red
    spec:
      containers:
      - image: nginx
        name: nginx
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-role.kubernetes.io/control-plane
                operator: Exists
                
# 적용 확인
$ k get po -o wide
NAME                   READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
blue-f69d4c887-bltzx   1/1     Running   0          21m   10.244.1.5   node01         <none>           <none>
blue-f69d4c887-mz4xq   1/1     Running   0          20m   10.244.1.6   node01         <none>           <none>
blue-f69d4c887-wwj5x   1/1     Running   0          21m   10.244.1.4   node01         <none>           <none>
red-5c5768844c-v62h7   1/1     Running   0          19s   10.244.0.5   controlplane   <none>           <none>
red-5c5768844c-vj57t   1/1     Running   0          19s   10.244.0.6   controlplane   <none>           <none>

 

ㅁ Resource Limits 

# pod의 리소스 정보 확인
$ k get po rabbit -o json| jq .spec.containers[0].resources
{
  "limits": {
    "cpu": "2"
  },
  "requests": {
    "cpu": "1"
  }
}


# OOMKilled 오류분석
$ k describe po elephant 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    1
      Started:      Wed, 17 Jan 2024 13:46:34 +0000
      Finished:     Wed, 17 Jan 2024 13:46:34 +0000
    Ready:          False
    Restart Count:  3
    Limits:
      memory:  10Mi
    Requests:
      memory:     5Mi
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


# limit memory 증설작업
$ cat elephant.yaml
apiVersion: v1
kind: Pod
metadata:
  name: elephant
  namespace: default
spec:
  containers:
  - args:
    - --vm
    - "1"
    - --vm-bytes
    - 15M
    - --vm-hang
    - "1"
    command:
    - stress
    image: polinux/stress
    name: mem-stress
    resources:
      limits:
        memory: 20Mi
      requests:
        memory: 5Mi
        
# 메모리 확장 후 정상작동 확인
$ k get po
NAME       READY   STATUS    RESTARTS   AGE
elephant   1/1     Running   0          70s

 

 

ㅁ DaemonSet

 ㅇ 전체 daemonset 조회

# 전체 daemonset 조회
$ k get daemonsets.apps -A
NAMESPACE      NAME              DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-flannel   kube-flannel-ds   1         1         1       1            1           <none>                   3m10s
kube-system    kube-proxy        1         1         1       1            1           kubernetes.io/os=linux   3m13s

 

ㅁ Daemonset 샘플 yaml 얻는 방법

1. delplyment 형태로 우선 샘플 획득

$ kubectl create deployment elasticsearch \
 --image=registry.k8s.io/fluentd-elasticsearch:1.20 \
 -n kube-system --dry-run=client -o yaml > fluentd.yaml

 

2. 내용 수정

 ㅇ replicas, strategy, status를 제거 하고 Deployment을 Daemonset으로 변경

 

3. 내용 검수

$ k apply -f fluentd.yaml --dry-run=server
daemonset.apps/elasticsearch created (server dry run)

 ㅇ dry-run 옵션으로 적용 전 yaml이 문법상 오류가 없는 지 검수할 수 있다.

 

4. 적용

$ k apply -f fluentd.yaml
daemonset.apps/elasticsearch created