[Elasticsearch] Elasticsearch + Kibana 설치하기 with Docker

ㅁ 들어가며

ㅇ [Elasticsearch] Docker로 Elasticsearch 설치 및 테스트하기에서 Elasticsearch를설치하였다.

ㅇ 공식 문서 - Install Elasticsearch with Docker의 내용을 토대로 docker 설치과정을 맥미니에서 수행해 보았다.

ㅇ 테스트 용으로 단일 노드 클러스터로 진행하였다.

 

ㅁ 관련 글

ㅇ [kotlin] Spring Data Elasticsearch 샘플코드
ㅇ [Elasticsearch] Elasticsearch + Kibana 설치하기 with Docker

ㅇ [Elasticsearch] Docker로 Elasticsearch 설치 및 테스트하기

ㅇ [Elasticsearch] Kibana Query Language 사용법 정리

 

ㅁ elastic 네트워크 생성

$ docker network create elastic

 

ㅁ Elasticsearch Docker image 받기

docker pull docker.elastic.co/elasticsearch/elasticsearch:8.14.3

 

ㅁ Elasticsearch container

docker run -d --name es01 --net elastic -p 9200:9200 -it -m 1GB docker.elastic.co/elasticsearch/elasticsearch:8.14.3

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  f-XHpRfq=L*LE6Q9*MZH

ℹ️  HTTP CA certificate SHA-256 fingerprint:
  153883733a7d00d8707d0429e3a0a771cb64f343ece454884f7d83f2fb70f43f

ℹ️  Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjE4LjAuMjo5MjAwIl0sImZnciI6IjE1Mzg4MzczM2E3ZDAwZDg3MDdkMDQyOWUzYTBhNzcxY2I2NGYzNDNlY2U0NTQ4ODRmN2Q4M2YyZmI3MGY0M2YiLCJrZXkiOiI3OGJpX1pBQkd1anJsdUs1VVBiVDpDcTBWdmVOS1JmMlhOMkVtNVp0MS1RIn0=

ℹ️ Configure other nodes to join this cluster:
• Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjE0LjAiLCJhZHIiOlsiMTcyLjE4LjAuMjo5MjAwIl0sImZnciI6IjE1Mzg4MzczM2E3ZDAwZDg3MDdkMDQyOWUzYTBhNzcxY2I2NGYzNDNlY2U0NTQ4ODRmN2Q4M2YyZmI3MGY0M2YiLCJrZXkiOiI4TWJpX1pBQkd1anJsdUs1VVBiVDo1N2ZOVWd0SVJyQ3VaUTdhd3dUMWVnIn0=

  If you're running in Docker, copy the enrollment token and run:
  `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.14.3`
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

 

 

ㅁ 비밀번호 환경변수 저장

export ELASTIC_PASSWORD="f-XHpRfq=L*LE6Q9*MZH"

 

ㅁ http_ca.crt 복사

docker cp es01:/usr/share/elasticsearch/config/certs/http_ca.crt .

 

ㅁ Elasticsearch API 통신

$ curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
{
  "name" : "dc4b2d8acae5",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "oAEKtPWOR8O4J-aoN9YiUg",
  "version" : {
    "number" : "8.14.3",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "d55f984299e0e88dee72ebd8255f7ff130859ad0",
    "build_date" : "2024-07-07T22:04:49.882652950Z",
    "build_snapshot" : false,
    "lucene_version" : "9.10.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

 

ㅁ Kibana 설치

# kibana 도커 이미지
$ docker pull docker.elastic.co/kibana/kibana:8.14.3

# kibana 컨테이너 실행
$ docker run --name kib01 --net elastic -p 5601:5601 docker.elastic.co/kibana/kibana:8.14.3
~~~~~
[2024-07-29T09:58:15.990+00:00][INFO ][http.server.Preboot] http server running at http://0.0.0.0:5601
[2024-07-29T09:58:16.117+00:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [interactiveSetup]
[2024-07-29T09:58:16.125+00:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: Validating Elasticsearch connection configuration…
[2024-07-29T09:58:16.155+00:00][INFO ][root] Holding setup until preboot stage is completed.


i Kibana has not been configured.

Go to http://0.0.0.0:5601/?code=886978 to get started.

ㅇ Kibana가 시작되면 code가 포함된 링크로 접속한다.

 

ㅇ E/S와 연동한 토큰을 입력하면 로그인 창이 나타난다.

ㅇ 초기 사용자는 elastic이다.

ㅇ 비번은 위에서 자동설정된 f-XHpRfq=L*LE6Q9*MZH이다.

 

ㅁ 간단한 데이터 입력 및 조회 테스트

# 데이터 입력
curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD 
-X POST "https://localhost:9200/test/_doc/1" -H "Content-Type: application/json" 
-d'{
    "name": "John Doe",
    "age": 30
}'

출력:
{
  "_index": "test",
  "_id": "1",
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 0,
  "_primary_term": 2
}

 

# 조회
curl --cacert http_ca.crt -u elastic:$ELASTIC_PASSWORD -X GET "https://localhost:9200/test/_doc/1?pretty"

출력:
{
  "_index" : "test",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "John Doe",
    "age" : 30
  }
}

 

ㅁ Kibana Dev Tool을 이용한 데이터 확인

add data

# add data
POST /test/_doc/2
{
  "name": "Kibana Test",    
  "age": 30
  
}

출력:
{
  "_index": "test",
  "_id": "2",
  "_version": 2,
  "result": "updated",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 2,
  "_primary_term": 2
}

ㅇ 데이터를 등록하면 오른쪽에 결과를 확인할 수 있다.

 

read data

# read data
GET /test/_doc/2?pretty

출력:
{
  "_index": "test",
  "_id": "2",
  "_version": 2,
  "_seq_no": 2,
  "_primary_term": 2,
  "found": true,
  "_source": {
    "name": "Kibana Test",
    "age": 30
  }
}

 

hot thread check

# node hot thread chk
GET /_nodes/hot_threads?pretty

출력:
::: {dc4b2d8acae5}{Ceqf9MNDRwGD9LoXWyGQEQ}{rbYQXhBOSgCU3PpE9iP8Bg}{dc4b2d8acae5}{172.18.0.2}{172.18.0.2:9300}{cdfhilmrstw}{8.14.3}{7000099-8505000}{ml.machine_memory=1073741824, transform.config_version=10.0.0, xpack.installed=true, ml.config_version=12.0.0, ml.max_jvm_size=536870912, ml.allocated_processors_double=8.0, ml.allocated_processors=8}
   Hot threads at 2024-07-29T15:36:34.127Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

ㅇ 노드의 부하 스래드를 체크할 수 있다.

ㅇ [Elasticsearch] Data Node 볼륨 병목현상 확인 및 처리

 

ㅇ 디버깅을 위한 다양한 옵션을 자동완성 기능으로 작성이 가능하다.

 

ㅁ 모니터링

ㅇ 등록된 index의 Document count와 사용 Rate를 확인할 수 있다.

 

ㅁ 제거 방법

# Remove the Elastic network
docker network rm elastic

# Remove Elasticsearch containers
docker rm es01

# Remove the Kibana container
docker rm kib01

 

ㅁ 함께 보면 좋은 사이트

ㅇ 공식 문서 - Install Elasticsearch with Docker