[Terraform] kubernetes, minikube 구축

ㅁ 들어가며

ㅇ Terraform으로 minikube를 구축하는 테스트를 진행해 보았습니다.

 

ㅁ minikube 환경

ㅇ minikube의 기본으로 설치하였다.

 

ㅇ minikube를 우선 시작한 후에 terraform으로 내부 리소스를 생성한다.

 

ㅁ Provider 설정

terraform {
  required_providers {
    kubernetes = {
      source = "hashicorp/kubernetes"
      version = "2.11.0"
    }
  }
}

provider "kubernetes" {
  config_path    = "~/.kube/config"
  config_context = "minikube"
}

ㅇ providers.tf 파일을 생성하였다.

 

ㅁ k8s 설정

resource "kubernetes_namespace" "example" {
  metadata {
    name = "k8s-ns-by-tf"
  }
}

resource "kubernetes_deployment" "example" {
  metadata {
    name = "terraform-example"
    labels = {
      test = "MyExampleApp"
    }
    namespace = "k8s-ns-by-tf"
  }

  spec {
    replicas = 2

    selector {
      match_labels = {
        test = "MyExampleApp"
      }
    }

    template {
      metadata {
        labels = {
          test = "MyExampleApp"
        }
      }

      spec {
        container {
          image = "nginx:1.21.6"
          name  = "example"

          resources {
            limits = {
              cpu    = "0.5"
              memory = "512Mi"
            }
            requests = {
              cpu    = "250m"
              memory = "50Mi"
            }
          }
        }
      }
    }
  }
}

ㅇ k8s.tf 생성

ㅇ k8s-ns-by-tf라는 namespace를 생성하고 nginx 2대를 띄우는 deployment를 생성하였다.

 

ㅁ 명령어 정리

# 리소스 생성
terraform init
terraform plan
terraform apply

# 리소스 확인
kubectl get ns
kubectl get deployment -n k8s-ns-by-tf
kubectl get pods -n k8s-ns-by-tf

# 리소스 정리
terraform destroy

ㅇ terraform으로 이루어질 리소스의 생명주기별로 명령어를 정리하였다.

ㅇ 아래에는 각 명령어의 실행과 로그를 정리하였다.

 

ㅁ Terraform init

$ terraform init

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/kubernetes versions matching "2.11.0"...
- Installing hashicorp/kubernetes v2.11.0...
- Installed hashicorp/kubernetes v2.11.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

 

ㅁ terraform plan

$ terraform plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kubernetes_deployment.example will be created
  + resource "kubernetes_deployment" "example" {
      + id               = (known after apply)
      + wait_for_rollout = true

      + metadata {
          + generation       = (known after apply)
          + labels           = {
              + "test" = "MyExampleApp"
            }
          + name             = "terraform-example"
          + namespace        = "k8s-ns-by-tf"
          + resource_version = (known after apply)
          + uid              = (known after apply)
        }

      + spec {
          + min_ready_seconds         = 0
          + paused                    = false
          + progress_deadline_seconds = 600
          + replicas                  = "2"
          + revision_history_limit    = 10

          + selector {
              + match_labels = {
                  + "test" = "MyExampleApp"
                }
            }

          + template {
              + metadata {
                  + generation       = (known after apply)
                  + labels           = {
                      + "test" = "MyExampleApp"
                    }
                  + name             = (known after apply)
                  + resource_version = (known after apply)
                  + uid              = (known after apply)
                }
              + spec {
                  + automount_service_account_token  = true
                  + dns_policy                       = "ClusterFirst"
                  + enable_service_links             = true
                  + host_ipc                         = false
                  + host_network                     = false
                  + host_pid                         = false
                  + hostname                         = (known after apply)
                  + node_name                        = (known after apply)
                  + restart_policy                   = "Always"
                  + service_account_name             = (known after apply)
                  + share_process_namespace          = false
                  + termination_grace_period_seconds = 30

                  + container {
                      + image                      = "nginx:1.21.6"
                      + image_pull_policy          = (known after apply)
                      + name                       = "example"
                      + stdin                      = false
                      + stdin_once                 = false
                      + termination_message_path   = "/dev/termination-log"
                      + termination_message_policy = (known after apply)
                      + tty                        = false

                      + resources {
                          + limits   = {
                              + "cpu"    = "0.5"
                              + "memory" = "512Mi"
                            }
                          + requests = {
                              + "cpu"    = "250m"
                              + "memory" = "50Mi"
                            }
                        }
                    }
                }
            }
        }
    }

  # kubernetes_namespace.example will be created
  + resource "kubernetes_namespace" "example" {
      + id = (known after apply)

      + metadata {
          + generation       = (known after apply)
          + name             = "k8s-ns-by-tf"
          + resource_version = (known after apply)
          + uid              = (known after apply)
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.

 

ㅁ terraform apply

$ terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # kubernetes_deployment.example will be created
  + resource "kubernetes_deployment" "example" {
      + id               = (known after apply)
      + wait_for_rollout = true

      + metadata {
          + generation       = (known after apply)
          + labels           = {
              + "test" = "MyExampleApp"
            }
          + name             = "terraform-example"
          + namespace        = "k8s-ns-by-tf"
          + resource_version = (known after apply)
          + uid              = (known after apply)
        }

      + spec {
          + min_ready_seconds         = 0
          + paused                    = false
          + progress_deadline_seconds = 600
          + replicas                  = "2"
          + revision_history_limit    = 10

          + selector {
              + match_labels = {
                  + "test" = "MyExampleApp"
                }
            }

          + template {
              + metadata {
                  + generation       = (known after apply)
                  + labels           = {
                      + "test" = "MyExampleApp"
                    }
                  + name             = (known after apply)
                  + resource_version = (known after apply)
                  + uid              = (known after apply)
                }
              + spec {
                  + automount_service_account_token  = true
                  + dns_policy                       = "ClusterFirst"
                  + enable_service_links             = true
                  + host_ipc                         = false
                  + host_network                     = false
                  + host_pid                         = false
                  + hostname                         = (known after apply)
                  + node_name                        = (known after apply)
                  + restart_policy                   = "Always"
                  + service_account_name             = (known after apply)
                  + share_process_namespace          = false
                  + termination_grace_period_seconds = 30

                  + container {
                      + image                      = "nginx:1.21.6"
                      + image_pull_policy          = (known after apply)
                      + name                       = "example"
                      + stdin                      = false
                      + stdin_once                 = false
                      + termination_message_path   = "/dev/termination-log"
                      + termination_message_policy = (known after apply)
                      + tty                        = false

                      + resources {
                          + limits   = {
                              + "cpu"    = "0.5"
                              + "memory" = "512Mi"
                            }
                          + requests = {
                              + "cpu"    = "250m"
                              + "memory" = "50Mi"
                            }
                        }
                    }
                }
            }
        }
    }

  # kubernetes_namespace.example will be created
  + resource "kubernetes_namespace" "example" {
      + id = (known after apply)

      + metadata {
          + generation       = (known after apply)
          + name             = "k8s-ns-by-tf"
          + resource_version = (known after apply)
          + uid              = (known after apply)
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

kubernetes_namespace.example: Creating...
kubernetes_deployment.example: Creating...
kubernetes_namespace.example: Creation complete after 1s [id=k8s-ns-by-tf]
kubernetes_deployment.example: Still creating... [10s elapsed]
kubernetes_deployment.example: Creation complete after 16s [id=k8s-ns-by-tf/terraform-example]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

ㅇ 'yes'라고 값을 입력하면 리소스들이 생성된다.

 

ㅁ 리소스 확인

# namespace 확인
$ kubectl get ns
NAME              STATUS   AGE
default           Active   31m
k8s-ns-by-tf      Active   107s
kube-node-lease   Active   31m
kube-public       Active   31m
kube-system       Active   31m

# deployment 확인
$ kubectl get deployment -n k8s-ns-by-tf
NAME                READY   UP-TO-DATE   AVAILABLE   AGE
terraform-example   2/2     2            2           8m48s

# pod 확인
$ kubectl get pods -n k8s-ns-by-tf
NAME                                 READY   STATUS    RESTARTS   AGE
terraform-example-67c59dfb88-7ktbr   1/1     Running   0          8m55s
terraform-example-67c59dfb88-znc4q   1/1     Running   0          8m55s

 

ㅁ 리소스 정리

$ terraform destroy
kubernetes_namespace.example: Refreshing state... [id=k8s-ns-by-tf]
kubernetes_deployment.example: Refreshing state... [id=k8s-ns-by-tf/terraform-example]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # kubernetes_deployment.example will be destroyed
  - resource "kubernetes_deployment" "example" {
      - id               = "k8s-ns-by-tf/terraform-example" -> null
      - wait_for_rollout = true -> null

      - metadata {
          - annotations      = {} -> null
          - generation       = 1 -> null
          - labels           = {
              - "test" = "MyExampleApp"
            } -> null
          - name             = "terraform-example" -> null
          - namespace        = "k8s-ns-by-tf" -> null
          - resource_version = "1873" -> null
          - uid              = "59b642f6-0c63-42bf-aee7-f4e4ec60018b" -> null
            # (1 unchanged attribute hidden)
        }

      - spec {
          - min_ready_seconds         = 0 -> null
          - paused                    = false -> null
          - progress_deadline_seconds = 600 -> null
          - replicas                  = "2" -> null
          - revision_history_limit    = 10 -> null

          - selector {
              - match_labels = {
                  - "test" = "MyExampleApp"
                } -> null
            }

          - strategy {
              - type = "RollingUpdate" -> null

              - rolling_update {
                  - max_surge       = "25%" -> null
                  - max_unavailable = "25%" -> null
                }
            }

          - template {
              - metadata {
                  - annotations      = {} -> null
                  - generation       = 0 -> null
                  - labels           = {
                      - "test" = "MyExampleApp"
                    } -> null
                    name             = null
                    # (4 unchanged attributes hidden)
                }
              - spec {
                  - active_deadline_seconds          = 0 -> null
                  - automount_service_account_token  = true -> null
                  - dns_policy                       = "ClusterFirst" -> null
                  - enable_service_links             = true -> null
                  - host_ipc                         = false -> null
                  - host_network                     = false -> null
                  - host_pid                         = false -> null
                  - node_selector                    = {} -> null
                  - restart_policy                   = "Always" -> null
                  - share_process_namespace          = false -> null
                  - termination_grace_period_seconds = 30 -> null
                    # (5 unchanged attributes hidden)

                  - container {
                      - args                       = [] -> null
                      - command                    = [] -> null
                      - image                      = "nginx:1.21.6" -> null
                      - image_pull_policy          = "IfNotPresent" -> null
                      - name                       = "example" -> null
                      - stdin                      = false -> null
                      - stdin_once                 = false -> null
                      - termination_message_path   = "/dev/termination-log" -> null
                      - termination_message_policy = "File" -> null
                      - tty                        = false -> null
                        # (1 unchanged attribute hidden)

                      - resources {
                          - limits   = {
                              - "cpu"    = "500m"
                              - "memory" = "512Mi"
                            } -> null
                          - requests = {
                              - "cpu"    = "250m"
                              - "memory" = "50Mi"
                            } -> null
                        }
                    }
                }
            }
        }
    }

  # kubernetes_namespace.example will be destroyed
  - resource "kubernetes_namespace" "example" {
      - id = "k8s-ns-by-tf" -> null

      - metadata {
          - annotations      = {} -> null
          - generation       = 0 -> null
          - labels           = {} -> null
          - name             = "k8s-ns-by-tf" -> null
          - resource_version = "1827" -> null
          - uid              = "5b74001f-d3b7-4fc3-a78c-a3b7ecc16ac7" -> null
            # (1 unchanged attribute hidden)
        }
    }

Plan: 0 to add, 0 to change, 2 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

kubernetes_namespace.example: Destroying... [id=k8s-ns-by-tf]
kubernetes_deployment.example: Destroying... [id=k8s-ns-by-tf/terraform-example]
kubernetes_deployment.example: Destruction complete after 0s
kubernetes_namespace.example: Destruction complete after 7s

Destroy complete! Resources: 2 destroyed.

 

ㅁ 함께 보면 좋은 사이트

ㅇ Getting Started with Kubernetes provider

ㅇ terraform_repo / kubernetes