[kubernetes] Kubectl 명령어 모음

 

 

[kubernetes] 개발환경 목차

1.  기본 Pod 조회

사용 가능한 모든 포드를 조회하는 명령어로 기본 Default Namespace를 조회합니다.

$ kubectl get pod                                                                                                                                                                         ✔  6785  07:35:22
NAME    READY   STATUS             RESTARTS         AGE
redis   0/1     CrashLoopBackOff   76 (3m39s ago)   6h10m

 

2. kube-system Pod 조회

 -n kube-system 옵션을 사용하여  kube-system namespace의 모든 Pod의 정보를 확인합니다. 

$ kubectl get pod -n kube-system
NAME                               READY   STATUS    RESTARTS        AGE
coredns-5d78c9869d-f2fwk           1/1     Running   0               6d20h
etcd-minikube                      1/1     Running   0               6d20h
kindnet-9xh4s                      1/1     Running   0               6d20h
kindnet-npkfz                      1/1     Running   0               6d20h
kube-apiserver-minikube            1/1     Running   0               6d20h
kube-controller-manager-minikube   1/1     Running   0               6d20h
kube-proxy-9ng9v                   1/1     Running   0               6d20h
kube-proxy-cwr9m                   1/1     Running   0               6d20h
kube-scheduler-minikube            1/1     Running   0               6d20h
metrics-server-844d8db974-hkbqv    1/1     Running   0               6d20h
storage-provisioner                1/1     Running   1 (6d20h ago)   6d20h

 

3. persistent volumes 조회

persistent volumes을 조회합니다. 조회 시 용량에 따른 Sort 옵션을 추가하였습니다.

$ kubectl get pv --sort-by=.spec.capacity.storage
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                 STORAGECLASS     REASON   AGE
pvc-3a26a602-7726-41f6-aaf5-c420fc57e928   8Gi        RWO            Delete           Bound    default/data-mysql-1685599291-0       cinder-tnt-csi            91d
pvc-3abd70bf-5a54-479d-ab65-15d3b7a0265b   8Gi        RWO            Delete           Bound    default/redis-data-redis-replicas-1   cinder-tnt-csi            135d
pvc-4a5c24a1-c315-4ed9-b975-4c164ea9e492   8Gi        RWO            Delete           Bound    default/redis-data-redis-master-0     cinder-tnt-csi            135d
pvc-6bdb7bab-a2ec-4534-a6ce-1eddcb174d4a   8Gi        RWO            Delete           Bound    default/redis-data-redis-replicas-2   cinder-tnt-csi            135d
pvc-7b3b4ece-cd1f-435b-85e3-32f14f7944a8   8Gi        RWO            Delete           Bound    default/redis-data-redis-replicas-0   cinder-tnt-csi            135d
pvc-2af3674a-f805-4945-985c-c78e573e152f   10Gi       RWO            Delete           Bound    priday-grafana/grafana-pvc            cinder-tnt-csi            51d

 

4. services 조회

모든 서비스를 조회합니다. 여기서 클러스터 IP, 외부 IP, 포트 세부 정보 및 포드 수명과 같은 유용한 정보를 얻을 수 있습니다. 이 출력은 현재 실행 중인 서비스와 현재 작동 중지된 서비스를 식별하는 데 도움이 됩니다.

$ kubectl get service
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   6d20h

 

5. 확장된 Pod 조회

-o wide옵션을 통해 추가적인 Pod의 상세정보를 확인 할 수 있습니다. 확장된 정보는 Pod의 IP정보, 위치한 Node정보 등을 확인할 수 있습니다.

$ kubectl get pod -o wide
NAME    READY   STATUS             RESTARTS         AGE     IP           NODE           NOMINATED NODE   READINESS GATES
redis   0/1     CrashLoopBackOff   77 (3m10s ago)   6h15m   10.244.1.3   minikube-m02   <none>           <none>

 

6. Pod 상세정보 조회(YAML)

 kubectl get pod <pod_name> -o yaml 옵션은 Pod의 상세정보를 YAML 형태로 제공합니다.

$ kubectl get pod redis -o yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2023-08-31T16:25:36Z"
  labels:
    run: redis
  name: redis
  namespace: default
  resourceVersion: "534906"
  uid: 29676686-3228-4f77-8a8e-a71c2a7c8f02
spec:
  containers:
  - args:
    - bash
    image: redis
    imagePullPolicy: Always
    name: redis
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-api-access-jh29w
      readOnly: true

 

7. Pod 상세정보 조회(Describe)

kubectl describe pod <pod_name> 옵션은 오류로 인해 Pod가 표시되지 않을 때 발생한 오류 정보를 알려줍니다.

$ kubectl describe pod redis
Name:             redis
Namespace:        default
Priority:         0
Service Account:  default
Node:             minikube-m02/192.168.49.3
Start Time:       Fri, 01 Sep 2023 01:25:36 +0900
Labels:           run=redis
Annotations:      <none>
Status:           Running
IP:               10.244.1.3
IPs:
  IP:  10.244.1.3
Containers:
  redis:
    Container ID:  docker://4847c1bd47887885f2a15b43e32fe3a298d3e96642537cc6f9d138581f9fa6f5
    Image:         redis
    Image ID:      docker-pullable://redis@sha256:c45b9ac48fde5e7ffc59e785719165511b1327151c392c891c2f552a83446847
    Port:          <none>
    Host Port:     <none>
    Args:
      bash
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 01 Sep 2023 07:53:01 +0900
      Finished:     Fri, 01 Sep 2023 07:53:01 +0900
    Ready:          False
    Restart Count:  80
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jh29w (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  kube-api-access-jh29w:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason   Age                       From     Message
  ----     ------   ----                      ----     -------
  Warning  BackOff  3m37s (x1763 over 6h28m)  kubelet  Back-off restarting failed container redis in pod redis_default(29676686-3228-4f77-8a8e-a71c2a7c8f02)

 

8. Node 정보조회

Kubernetes 클러스터의 모든 노드를 확인할 수 있습니다. minikube 환경에서 2개의 노드를 확인할 수 있습니다.

$ kubectl get node
NAME           STATUS   ROLES           AGE     VERSION
minikube       Ready    control-plane   6d20h   v1.27.3
minikube-m02   Ready    <none>          6d20h   v1.27.3

 

9. Pod의 라벨 확인

모든 Pod의 레이블을 보려면 --show-labels 옵션을 사용합니다.

$ kubectl get pod --show-labels
NAME    READY   STATUS             RESTARTS         AGE     LABELS
redis   0/1     CrashLoopBackOff   82 (4m36s ago)   6h42m   run=redis

 

10. List all the secrets used by Pods

If you want to see all the secrets used by all the pods, then you need to use below command. This command will be helpful in identifying all the secrets used in all the pods.

$ kubectl get pods -o json | jq '.items[].spec.containers[].env[]?.valueFrom.secretKeyRef.name' | grep -v null | sort | uniq
"aws-secrets"
"test-cert"
"hocsecret"
"ingress-crt"

 

11. Pod 로그 확인

Pod의 전체 로그를 nginx.log 파일로 저장하여 로그 파일을 확인 할 수 있습니다.

$ kubectl logs nginx > nginx.log 
$ cat nginx.log
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/09/01 09:38:11 [notice] 1#1: using the "epoll" event method
2023/09/01 09:38:11 [notice] 1#1: nginx/1.25.2
2023/09/01 09:38:11 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14)
2023/09/01 09:38:11 [notice] 1#1: OS: Linux 5.15.49-linuxkit-pr
2023/09/01 09:38:11 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/09/01 09:38:11 [notice] 1#1: start worker processes
2023/09/01 09:38:11 [notice] 1#1: start worker process 29
2023/09/01 09:38:11 [notice] 1#1: start worker process 30
2023/09/01 09:38:11 [notice] 1#1: start worker process 31
2023/09/01 09:38:11 [notice] 1#1: start worker process 32

 

12. Pod 내부 명령어 실행

kubectl exec <pod_name> -- <commands> 명령어를 통해 Pod 내부에서 명령을 실행 할 수 있습니다.

$ kubectl exec nginx -- date
Fri Sep  1 12:15:20 UTC 2023

 

13. Pod의 로그 모니터링

If you want to see streaming logs of a specific pod, then you need to use -f option as shown in below example. This command will be very useful in troubleshooting any of the pod error.

$ kubectl logs nginx 
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/09/01 09:38:11 [notice] 1#1: using the "epoll" event method
2023/09/01 09:38:11 [notice] 1#1: nginx/1.25.2
2023/09/01 09:38:11 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14)
2023/09/01 09:38:11 [notice] 1#1: OS: Linux 5.15.49-linuxkit-pr
2023/09/01 09:38:11 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/09/01 09:38:11 [notice] 1#1: start worker processes
2023/09/01 09:38:11 [notice] 1#1: start worker process 29
2023/09/01 09:38:11 [notice] 1#1: start worker process 30
2023/09/01 09:38:11 [notice] 1#1: start worker process 31
2023/09/01 09:38:11 [notice] 1#1: start worker process 32

 

14. Node, Pod의 Metrics 정보확인

노드, Pod의 Metric 정보 확인

# 노드의 Metric 정보
$ kubectl top no
NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
minikube       103m         2%     785Mi           19%
minikube-m02   35m          0%     419Mi           10%

# 파드의 Metric 정보
$ kubectl top po -A
NAMESPACE     NAME                               CPU(cores)   MEMORY(bytes)
default       nginx                              0m           15Mi
kube-system   coredns-5d78c9869d-f2fwk           2m           20Mi
kube-system   etcd-minikube                      17m          98Mi
kube-system   kindnet-9xh4s                      1m           11Mi
kube-system   kindnet-npkfz                      1m           12Mi
kube-system   kube-apiserver-minikube            28m          258Mi
kube-system   kube-controller-manager-minikube   15m          57Mi
kube-system   kube-proxy-9ng9v                   1m           21Mi
kube-system   kube-proxy-cwr9m                   1m           23Mi
kube-system   kube-scheduler-minikube            2m           28Mi
kube-system   metrics-server-844d8db974-hkbqv    3m           23Mi
kube-system   storage-provisioner                1m           16Mi

 

15. Pod 내부 bash 

Pod에 로그인하려면 아래 kubectl 명령을 사용하여 test-pod-0 Pod에서 bash 프롬프트를 열어야 합니다.

$ kubectl exec -it nginx bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@nginx:/#

-it 옵션을 사용해 배시 셸을 유지합니다.

 

16. Cluster 정보 조회

Kubernetes 클러스터 요약을 확인하려면 아래의 클러스터 정보 옵션을 사용하면 됩니다. 이 명령은 Kubernetes 마스터가 현재 실행 중인 URL에 대해 알려줍니다. 또한 아래에서 볼 수 있듯이 KubeDNS가 가리키는 URL에 대해서도 알려줍니다.

$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:52363
CoreDNS is running at https://127.0.0.1:52363/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

 

17. Cluster 정보  Dump

모든 kubernetes 클러스터 로그를 덤프하려면 아래 명령을 사용해야 합니다. 이 명령은 모든 클러스터 포드에 대한 완전한 세부 정보를 제공하므로 이 명령의 출력은 꽤 길 수 있습니다.

$ kubectl cluster-info dump
{
    "kind": "NodeList",
    "apiVersion": "v1",
    "metadata": {
        "resourceVersion": "536592"
    },
    "items": [
        {
            "metadata": {
                "name": "minikube",
                "uid": "40a56ad3-5464-4adc-beaa-4fd7184ca65c",
                "resourceVersion": "536335",
                "creationTimestamp": "2023-08-25T02:25:09Z",
                "labels": {
                    "beta.kubernetes.io/arch": "arm64",
                    "beta.kubernetes.io/os": "linux",
                    "kubernetes.io/arch": "arm64",
                    "kubernetes.io/hostname": "minikube",
                    "kubernetes.io/os": "linux",

~~~ 생략 ~~~
==== START logs for container coredns of pod kube-system/coredns-5d78c9869d-f2fwk ====
.:53
[INFO] plugin/reload: Running configuration SHA512 = f869070685748660180df1b7a47d58cdafcf2f368266578c062d1151dc2c900964aecc5975e8882e6de6fdfb6460463e30ebfaad2ec8f0c3c6436f80225b3b5b
CoreDNS-1.10.1
linux/arm64, go1.20, 055b2c3
[INFO] 127.0.0.1:58100 - 50027 "HINFO IN 1818280593740948307.1063156711439074247. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.043143417s
[WARNING] plugin/health: Local health request to "http://:8080/health" took more than 1s: 1.203135542s
==== END logs for container coredns of pod kube-system/coredns-5d78c9869d-f2fwk ====
~~~ 생략 ~~~

 

18. node metrics 정보확인

클러스터 노드의 리소스 정보를 확인합니다.  CPU 코어 수, CPU 사용률 % 및 메모리 사용률 %를 확인할 수 있습니다.

$ kubectl top node
NAME           CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
minikube       107m         2%     800Mi           20%
minikube-m02   38m          0%     360Mi           9%

 

19. Pod 삭제

If you want to delete a single pod then you can simply use delete pod <pod_name> option as shown in below example. Usually this command will be used to restart the pods where deleting the pod will recreate the pod after current pod termination as per the rule applied through YAML.

$ kubectl delete po redis
pod "redis" deleted

 

20. Pod 전체 삭제

If you want to delete all the Kubernetes Cluster pods, then awk will be very handy here for choosing all the first columns and feeding those columns as input to xargs command to delete all the pods at once instead of deleting one by one as shown in below example.

$ kubectl get pods --no-headers | awk '{ print $1};' | xargs kubectl delete pods
pod "nginx" deleted

 

21.  kubectl API Resources

모든 API 리소스를 확인하려면 아래 api-resources 옵션을 사용해야 합니다. API 리소스의 단축키 정보도 함께 확인 할 수 있습니다.

$ kubectl api-resources
NAME                              SHORTNAMES   APIVERSION                             NAMESPACED   KIND
bindings                                       v1                                     true         Binding
componentstatuses                 cs           v1                                     false        ComponentStatus
configmaps                        cm           v1                                     true         ConfigMap
endpoints                         ep           v1                                     true         Endpoints
events                            ev           v1                                     true         Event
limitranges                       limits       v1                                     true         LimitRange
namespaces                        ns           v1                                     false        Namespace
nodes                             no           v1                                     false        Node
persistentvolumeclaims            pvc          v1                                     true         PersistentVolumeClaim
persistentvolumes                 pv           v1                                     false        PersistentVolume
pods                              po           v1                                     true         Pod
podtemplates                                   v1                                     true         PodTemplate
replicationcontrollers            rc           v1                                     true         ReplicationController
resourcequotas                    quota        v1                                     true         ResourceQuota
secrets                                        v1                                     true         Secret
serviceaccounts                   sa           v1                                     true         ServiceAccount

 

22. Kubectl Commands 도움받기

아래와 같이 --help 옵션을 사용하여 kubectl 명령과 함께 사용할 수 있는 모든 옵션을 확인할 수도 있습니다. 

예를 들어, kubectl get pods 예시에서 사용할 수 있는 모든 옵션을 확인하려면 아래와 같이 kubectl get pods --help 명령을 사용해야 합니다.

$ kubectl get pods --help
Display one or many resources.

 Prints a table of the most important information about the specified resources. You can filter the list using a label
selector and the --selector flag. If the desired resource type is namespaced you will only see results in your current
namespace unless you pass --all-namespaces.

 By specifying the output as 'template' and providing a Go template as the value of the --template flag, you can filter
the attributes of the fetched resources.

Use "kubectl api-resources" for a complete list of supported resources.

Examples:
  # List all pods in ps output format
  kubectl get pods

~~~ 생략 ~~~~

Options:
    -A, --all-namespaces=false:
	If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even
	if specified with --namespace.

~~~ 생략 ~~~~

Usage:
  kubectl get
[(-o|--output=)json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-as-json|jsonpath-file|custom-columns|custom-columns-file|wide]
(TYPE[.VERSION][.GROUP] [NAME | -l label] | TYPE[.VERSION][.GROUP]/NAME ...) [flags] [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).

 

 

ㅁ 함께 보면 좋은 사이트

22 Best Kubectl Command Examples for Beginners | CyberITHub

Kubernetes for Beginners