SoFunction
Updated on 2025-03-02

Steps to build a cluster using ElasticSearch

1. Preface

Elasticsearch is an open source distributed search and analysis engine for full-text search, structured search, analytical and visualizing large-scale data.

It is designed as a scalable real-time search engine that can process large-scale data and provides fast search and analysis capabilities.

Here are some features and uses of Elasticsearch:

  • Full-text search: Elasticsearch can store a large amount of data and provide query functions based on full-text search, supporting various complex query operations.
  • Real-time: Elasticsearch can index data in real time, and the query response time is very fast, suitable for scenarios where quick query is required.
  • Distributed Architecture: Elasticsearch is a distributed system where data is distributed on multiple nodes and can be horizontally scaled to process large-scale data.
  • Support for multiple data types: Elasticsearch supports indexing and querying of multiple data types, including text, numbers, dates, etc.
  • Powerful query language: Elasticsearch provides rich and flexible query languages ​​that can meet various complex query needs.
  • Real-time data analysis: In addition to the search function, Elasticsearch also provides powerful data analysis functions, which can aggregate, statistical and visually analyze data.
  • Easily deploy and manage: Elasticsearch provides easy-to-use APIs and management tools to easily deploy, monitor and manage clusters.
  • Integration with other tools: Elasticsearch integrates with many tools and technologies such as Logstash, Kibana, Beats, etc. to build powerful log management and data analysis solutions.

In general, Elasticsearch is a powerful and flexible search and analysis engine suitable for a variety of scenarios, including log analysis, real-time monitoring, full-text search, etc.

This example will build an Elasticsearch cluster on 3 machines:

192.168.72.151  node-1
192.168.72.152  node-2
192.168.72.153  node-3

2. Use RPM to install Elasticsearch

Import Elasticsearch GPG keys

Download and install the public signature key

rpm --import /GPG-KEY-elasticsearch

Install from RPM repository

Created in /etc//

[elasticsearch]
name=Elasticsearch repository for  packages
baseurl=/packages//yum
gpgcheck=1
gpgkey=/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md

Execute the yum install command

yum install --enablerepo=elasticsearch elasticsearch -y

The default generated directory

  • Installation directory: /usr/share/elasticsearch
  • Configuration file directory: /etc/elasticsearch

Start the Elasticsearch command:

systemctl enable 
systemctl start

3. Set basic security

When Elasticsearch is started for the first time, a password will be generated for the user and TLS will be automatically configured for the user. The TLS configuration can be adjusted at any time and the node certificate will be updated.

Generate a certificate

1. Enter the installation directory of Elasticsearch on any node and use elasticsearch-certutil to generate CA for the cluster.

bin/elasticsearch-certutil ca
  • Set the CA file name (it's the default)
  • Set password for CA

2. Use elastic-stack-ca.p12 in the previous step to generate certificates and private keys for the cluster.

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
  • Enter the password of CA
  • Create a password for the certificate and set the file name (default)

3. Copy the certificate to other nodes.

Encrypt inter-node communication using TLS

1. Enter the Elasticsearch configuration directory and edit the file:

: my-cluster
: node-1
: true
: true
.verification_mode: certificate 
.client_authentication: required
: elastic-certificates.p12
: elastic-certificates.p12

2. Execute the following command to save the certificate password to the keystore of Elasticsearch

bin/elasticsearch-keystore add .secure_password
bin/elasticsearch-keystore add .secure_password

3. Complete the previous steps for each node in the cluster.

4. On each node in the cluster, restart Elasticsearch.

4. Encrypt HTTP client communication for Elasticsearch

1. On any single node, run the Elasticsearch HTTP Certificate Tool to generate a certificate signature request (CSR) from the directory where Elasticsearch is installed.

bin/elasticsearch-certutil http
  • Whether to generate CSR, enter n.
  • Whether to use an existing CA, enter y.
  • Enter the path to CA. This is the absolute path to elastic-stack-ca.p12.
  • Enter the password for CA.
  • Enter the validity period of the certificate.
  • To generate a certificate for each node, enter y.
  • Enter the name of each node ().
  • Enter the host name and IP address of all nodes.
  • Enter the private key password.

This command generates a .zip file containing the certificates and keys used by Elasticsearch and Kibana. Each folder contains a , explaining how to use these files.

2. Decompress the generated file. This compressed file contains directories for Elasticsearch and Kibana.

/elasticsearch
|_ 
|_ http.p12
|_ 
/kibana
|_ 
|_ 
|_

3. On each node in the cluster, complete the following steps:

Copy http.p12 in the elasticsearch folder above to the Elasticsearch configuration directory.

Edit , enable HTTPS security, and specify the location of the http.12 file.

: true
: http.p12

Add the private key password to Elasticsearch's security settings.

bin/elasticsearch-keystore add .secure_password

4. Restart Elasticsearch

5. Configure the cluster

edit

The configuration files of the 3 nodes are the same except for the IP and node names.

: my-clusters
: node-1

: /var/lib/elasticsearch
: /var/log/elasticsearch

.index_buffer_size: 20%
.min_index_buffer_size: 96mb
thread_pool:
  search:
    size: 32
  analyze:
    size: 30
    queue_size: 1000
: 2%
: 10%
: 20%
: 40%

: 0.0.0.0
: 9200
: 9300
: true
http.max_content_length: 100mb

discovery.seed_hosts: ["192.168.72.151", "192.168.72.152","192.168.72.153"]

cluster.initial_master_nodes: ["node-1", "node-2","node-3"]
cluster.fault_detection.leader_check.interval: 5s
discovery.cluster_formation_warning_timeout: 10s
: 30s
.node_initial_primaries_recoveries: 16
.node_concurrent_recoveries: 8
.max_bytes_per_sec: 125mb

The cluster is connected through port 9300 by default. Remember to open the firewall 9300.

Start the cluster

Start each node in turn

service elasticsearch start

Set password

bin/elasticsearch-setup-passwords interactive

Check cluster status

curl  -XGET --user elastic:password http://192.168.72.151:9200/_cluster/health?pretty

You can also access it directly in the browser

Please note that the IP address, port and directory path in the above steps should be adjusted according to your actual environment.

Summarize

The above is personal experience. I hope you can give you a reference and I hope you can support me more.