Day01-ElasticSearch的单点部署，集群部署，多实例部署，es-head和postman环境搭建

0、ElasticSearch的简单介绍

Elasticstack：

• Elastic$earch
  • 日志存储和检索
• filebeat：
  • 用于日志收集
• logstash：
  • 日志转换，也可以用于日志采集
• kibana：
  • 用于日志展示
• TB(PB)级别日志处理：
  • kafka
  • zookeeper

ElasticSearch:

• 单点部署

• 集群部署二

• ES集群的常见术语

  • 分片
  • 副本
  • 索引
  • 文档
  • …

• 集群的API基础操作

  • 索引管理
  • 文档管理

• 集群环境准备：

IP地址	配置	主机名
10.0.0.101	CPU:2C MEMORY:4G DISK:50G+	elk101.oldboyedu.com
10.0.0.102	CPU:2C MEMORY:4G DISK:50G+	elk102.oldboyedu.com
10.0.0.103	CPU:2C MEMORY:4G DISK:50G+	elk103.oldboyedu.com

配置好后拍快照

1、ElasticSearch的单点部署

(1)下载ES软件包
https://www.elastic.co/cn/downloads

(2)老男孩线下同学下载

curl -o elasticsearch-7.17.5-x86_64.rpm  http://192.168.15.253/ElasticStack/softwares/rpm/elasticsearch-7.17.5-x86_64.rpm

(3)安装es

rpm -ivh elasticsearch-7.17.5-x86_64.rpm 

(4)修改es的配置文件

vim /etc/elasticsearch/elasticsearch.yml
...
# ES服务监听对外暴露服务的地址
network.host: 0.0.0.0# 指定ES集群的节点IP
discovery.seed_hosts: ["10.0.0.101"]# 指定参与master选举的节点
cluster.initial_master_nodes: ["10.0.0.101"]

(5)启动ES服务

systemctl enable --now elasticsearch

(6)验证节点是否正常工作

[root@elk101 ~]# ss -ntl
State       Recv-Q Send-Q         Local Address:Port                        Peer Address:Port     
...
LISTEN      0      16384                   [::]:9200                                [::]:*        
LISTEN      0      16384                   [::]:9300                                [::]:*        

• 面试题1:
  • 9200端口作用:
    • 对ES集群外部提供http/https服务。可以理解为对客户端提供服务。
  • 9300端口作用:
    • 对ES集群内部进行数据通信传输端口。走的时候tcp协议。

(7)客户端验证

curl 10.0.0.101:9200

如果遇到集群的uuid为"_na_"情况时，可以执行如下操作:

# 两块网卡时，打开指定参与master选举的节点
vim /etc/elasticsearch/elasticsearch.yml
cluster.initial_master_nodes: ["10.0.0.101"]
# 然后执行
systemctl stop elasticsearch.service
rm -rf /var/lib/elasticsearch/* /var/log/elasticsearch/* /tmp/*
systemctl start elasticsearch.service
curl 10.0.0.101:9200

2、ElasticSearch的集群部署

(1)下载ES软件包
https://www.elastic.co/cn/downloads

(2)老男孩线下同学下载

curl -o elasticsearch-7.17.5-x86_64.rpm http://192.168.15.253/ElasticStack/softwares/rpm/elasticsearch-7.17.5-x86_64.rpm

(3)所有节点安装es

rpm -ivh elasticsearch-7.17.5-x86_64.rpm 

(4)elk101.oldboyedu.com做数据清空

systemctl stop elasticsearch.service
rm -rf /var/lib/elasticsearch/* /var/log/elasticsearch/* /tmp/*	

(5)修改es的配置文件

vim /etc/elasticsearch/elasticsearch.yml 
...# 指定ES集群的名称
cluster.name: oldboyedu-linux85# ES服务监听对外暴露服务的地址
network.host: 0.0.0.0# 指定ES集群的节点IP
discovery.seed_hosts: ["10.0.0.101","10.0.0.102","10.0.0.103"]# 指定参与master选举的节点
cluster.initial_master_nodes: ["10.0.0.101","10.0.0.102","10.0.0.103"]

(6)将配置文件分发到其他两个节点

scp /etc/elasticsearch/elasticsearch.yml  10.0.0.102:/etc/elasticsearch/elasticsearch.yml
scp /etc/elasticsearch/elasticsearch.yml  10.0.0.103:/etc/elasticsearch/elasticsearch.yml

(6)所有节点启动ES服务

systemctl enable --now elasticsearch

(7)验证ES集群节点是否正常工作

[[e-mail] ~]# curl 10.0.0.101:9200/_cat/nodes   # *在谁那谁就是老大
10.0.0.103 11 71 51 1.28 0.38 0.13 cdfhilmrstw - elk103.oldboyedu.com
10.0.0.102 15 73 46 1.10 0.34 0.12 cdfhilmrstw - elk102.oldboyedu.com
10.0.0.101  6 92  1 0.07 0.11 0.12 cdfhilmrstw * elk101.oldboyedu.com# 半数以上机器存活则集群正常，为了防止脑裂，半数以上机器不存活的话，集群不提供对外服务[[e-mail] ~]# 
[[e-mail] ~]# curl 10.0.0.101:9200/_cat/nodes?v
ip         heap.percent ram.percent cpu load_1m load_5m load_15 m node.role   master name
10.0.0.103           12          71   0    0.72    0.34     0.13 cdfhilmrstw -      elk103.oldboyedu.com
10.0.0.102           16          73   0    0.61    0.31     0.12 cdfhilmrstw -      elk102.oldboyedu.com
10.0.0.101            7          92   0    0.04    0.10     0.11 cdfhilmrstw *      elk101.oldboyedu.comheap.percent   # 堆内存使用百分比
ram.percent    # 内存百分比

3、基于二进制部署ElasticSearch

3.1 准备阶段

(1)elk101配置集群免密登录及同步脚本
1.1 修改主机列表

cat >> /etc/hosts <<'EOF'
10.0.0.101 elk101.oldboyedu.com
10.0.0.102 elk102.oldboyedu.com
10.0.0.103 elk103.oldboyedu.com
EOF

1.2 elk101节点上生成密钥对

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa -q

1.3 elk101配置所有集群节点的免密登录

for ((host_id=101;host_id<=103;host_id++));do ssh-copy-id elk${host_id}.oldboyedu.com ;done

1.4 链接测试

ssh 'elk101.oldboyedu.com'
ssh 'elk102.oldboyedu.com'
ssh 'elk103.oldboyedu.com'

1.5 所有节点安装rsync数据同步工具

yum -y install rsync

1.6 编写同步脚本

cat > /usr/local/sbin/data_rsync.sh <<'EOF'
#!/bin/bash# Auther: Jason Yinif  [ $# -ne 1 ];thenecho "Usage: $0 /path/to/file(绝对路径)"exit
fi # 判断文件是否存在if [ ! -e $1 ];thenecho "[ $1 ] dir or file not find!"exit
fi# 获取父路径fullpath=`dirname $1`# 获取子路径basename=`basename $1`# 进入到父路径cd $fullpathfor ((host_id=102;host_id<=103;host_id++))do# 使得终端输出变为绿色tput setaf 2echo ===== rsyncing elk${host_id}.oldboyedu.com: $basename =====# 使得终端恢复原来的颜色tput setaf 7# 将数据同步到其他两个节点rsync -apz $basename  `whoami`@elk${host_id}.oldboyedu.com:$fullpathif [ $? -eq 0 ];thenecho "命令执行成功!"fi
done
EOF

1.7 给脚本授权

chmod +x /usr/local/sbin/data_rsync.sh

3.2 部署阶段

(2)elk101下载ES软件包

curl -o elasticsearch-7.17.5-linux-x86_64.tar.gz http://192.168.15.253/ElasticStack/softwares/binary/elasticsearch-7.17.5-linux-x86_64.tar.gz

(3)所有节点创建运行ES服务的用户

useradd -u 2023 oldboyedu

(4)elk101创建ElasticSearch的工作目录
4.1 可以如下操作

mkdir -pv /oldboyedu/{data,logs,softwares}/es7
chown oldboyedu:oldboyedu -R /oldboyedu/{softwares,data,logs}/es7/
ll /oldboyedu/{data,logs,softwares}/es7 -d

4.2 也可以如下操作（推荐）

install -d /oldboyedu/{data,logs,softwares}/es7 -o oldboyedu -g oldboyedu

(5)elk101解压软件包

tar xf elasticsearch-7.17.5-linux-x86_64.tar.gz -C /oldboyedu/softwares/es7/

(6)elk101修改配置文件

[[e-mail] ~]# egrep -v "^#|^$" /oldboyedu/softwares/es7/elasticsearch-7.17.5/config/elasticsearch.yml 
cluster.name: oldboyedu-linux85-binary
path.data: /oldboyedu/data/es7
path.logs: /oldboyedu/logs/es7
network.host: 0.0.0.0
discovery.seed_hosts: ["elk101.oldboyedu.com","elk102.oldboyedu.com","elk103.oldboyedu.com"]
cluster.initial_master_nodes: ["elk101.oldboyedu.com","elk102.oldboyedu.com","elk103.oldboyedu.com"]

(7)elk101同步程序目录

[[e-mail] ~]# data_rsync.sh  /oldboyedu

(8)elk101节点针对ES基础调优
8.1 修改文件打开数量上线，修改后需要断开会话

[[e-mail] ~]# cat /etc/security/limits.d/es7.conf*	soft	nofile	65535
*	hard	nofile	131070
*	hard	nproc	8192[[e-mail] ~]# ulimit -Sn
65535[[e-mail] ~]# ulimit -Hn
131070

8.2 调大内核虚拟内存映射值

[[e-mail] ~]# cat  /etc/sysctl.d/es.conf
vm.max_map_count=524288[[e-mail] ~]# sysctl -q vm.max_map_count
vm.max_map_count = 65530[[e-mail] ~]# sysctl -f  /etc/sysctl.d/es.conf    # 生效
vm.max_map_count = 524288[[e-mail] ~]# sysctl -q vm.max_map_count
vm.max_map_count = 524288

8.3 同步调优文件

[[e-mail] ~]# data_rsync.sh  /etc/sysctl.d/es.conf
[[e-mail] ~]# data_rsync.sh  /etc/security/limits.d/es7.conf 
[[e-mail] ~]# data_rsync.sh /etc/hosts

(9)所有节点启动服务

9.1 先断开连接

让文件打开数量上限生效。

9.2 使得内核参数生效

sysctl -f  /etc/sysctl.d/es.conf	

9.3 后台启动es服务

chown oldboyedu:oldboyedu -R /oldboyedu/{softwares,data,logs}/es7/
su oldboyedu -c '/oldboyedu/softwares/es7/elasticsearch-7.17.5/bin/elasticsearch -d'	

(10)验证服务是否正常

[[e-mail] ~]# curl 10.0.0.101:9200
{"name" : "elk101.oldboyedu.com","cluster_name" : "oldboyedu-linux85-binary","cluster_uuid" : "l0Vl0fYMSmSun_Mnv3dH7g","version" : {"number" : "7.17.5","build_flavor" : "default","build_type" : "tar","build_hash" : "8d61b4f7ddf931f219e3745f295ed2bbc50c8e84","build_date" : "2022-06-23T21:57:28.736740635Z","build_snapshot" : false,"lucene_version" : "8.11.1","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search"
}[[e-mail] ~]# curl 10.0.0.101:9200/_cat/nodes
10.0.0.101 27 96 38 0.68 0.26 0.14 cdfhilmrstw * elk101.oldboyedu.com
10.0.0.102 23 88 38 0.49 0.21 0.14 cdfhilmrstw - elk102.oldboyedu.com
10.0.0.103 21 88 42 1.25 0.42 0.22 cdfhilmrstw - elk103.oldboyedu.com

补充：若集群UUID出现"_na_"状态时，做如下动作，所有节点操作

pkill java
rm -rf  /oldboyedu/data/es7/* /oldboyedu/logs/es7/* /tmp/*

3.3 使用systemctl管理ES服务

1.elk101停止ES服务

pkill java

2.elk101编写ES启动脚本

cat > /usr/lib/systemd/system/es7.service <<EOF
[Unit]
Description=oldboyedu linux85 es7
After=network.target[Service]
Type=simple
ExecStart=/oldboyedu/softwares/es7/elasticsearch-7.17.5/bin/elasticsearch
User=oldboyedu
LimitNOFILE=131070[Install]
WantedBy=multi-user.target
EOF

3.elk101加载脚本

systemctl daemon-reload

4.elk101设置开机自启动

systemctl enable --now es7

5.elk101查看服务是否正常运行

systemctl status es7
ss -ntl
curl 10.0.0.101:9200/_cat/nodes

6.其他节点使用脚本

data_rsync.sh /usr/lib/systemd/system/es7.service

7.其他节点使用脚本管理

pkill java
systemctl enable --now es7
systemctl status es7

3.4 使用oracle JDK管理ES服务

(1)elk101下载JDK环境

curl -o jdk-8u291-linux-x64.tar.gz http://192.168.15.253/ElasticStack/softwares/binary/jdk-8u291-linux-x64.tar.gz

(2)elk101解压JDK软件包

tar xf jdk-8u291-linux-x64.tar.gz -C /oldboyedu/softwares/

(3)elk101配置系统环境变量

[[e-mail] ~]# cat /etc/profile.d/jdk.sh 
#!/bin/bashexport JAVA_HOME=/oldboyedu/softwares/jdk1.8.0_291
export PATH=$PATH:$JAVA_HOME/bin[root@elk101 ~]# source /etc/profile.d/jdk.sh
[root@elk101 ~]# which java
/oldboyedu/softwares/jdk1.8.0_291/bin/java
[[e-mail] ~]# java -version
java version "1.8.0_291"
Java(TM) SE Runtime Environment (build 1.8.0_291-b10)
Java HotSpot(TM) 64-Bit Server VM (build 25.291-b10, mixed mode)

(4)elk101修改启动脚本

cat > /usr/lib/systemd/system/es7.service <<EOF
[Unit]
Description=oldboyedu linux85 es7
After=network.target[Service]
Type=simple
Environment=JAVA_HOME=/oldboyedu/softwares/jdk1.8.0_291
ExecStart=/oldboyedu/softwares/es7/elasticsearch-7.17.5/bin/elasticsearch
User=oldboyedu
LimitNOFILE=131070
LimitNPROC=8192[Install]
WantedBy=multi-user.target
EOFsystemctl daemon-reload
systemctl start es7

(5)elk101重启服务

systemctl restart es7

(6)elk101查看服务是否正常

[[e-mail] ~]# curl 10.0.0.101:9200/_cat/nodes
10.0.0.101 17 74 6 0.17 0.28 0.19 cdfhilmrstw - elk101.oldboyedu.com
10.0.0.102 11 90 0 0.03 0.03 0.05 cdfhilmrstw * elk102.oldboyedu.com
10.0.0.103 21 91 0 0.00 0.01 0.05 cdfhilmrstw - elk103.oldboyedu.com
root@elk101 ~]# jps
3572 Jps
3371 Elasticsearch
[root@elk101 ~]# ps -ef|grep java
oldboye+   3371      1 99 14:02 ?        00:00:35 /oldboyedu/softwares/jdk1.8.0_291/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j2.formatMsgNoLookups=true -Djava.locale.providers=SPI,JRE -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/tmp/elasticsearch-1143701341521397671 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Xms1024m -Xmx1024m -XX:MaxDirectMemorySize=536870912 -Des.path.home=/oldboyedu/softwares/es7/elasticsearch-7.17.5 -Des.path.conf=/oldboyedu/softwares/es7/elasticsearch-7.17.5/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /oldboyedu/softwares/es7/elasticsearch-7.17.5/lib/* org.elasticsearch.bootstrap.Elasticsearch
root       3590   2034  0 14:03 pts/0    00:00:00 grep --color=auto java

(7)elk101同步配置到其他节点

data_rsync.sh /etc/profile.d/jdk.sh
data_rsync.sh /usr/lib/systemd/system/es7.service
data_rsync.sh /oldboyedu/softwares/jdk1.8.0_291

(8)其他节点重启服务

systemctl daemon-reload  && systemctl restart es7

3.5 修改ES环境的堆(heap)内存大小

(1)elk101修改配置文件

[[e-mail] ~]# vim /oldboyedu/softwares/es7/elasticsearch-7.17.5/config/jvm.options
...
-Xms256m
-Xmx256m

(2)elk101重启服务并验证堆内存大小，若不配置，默认1GB

[root@elk101 ~]# systemctl restart es7
[root@elk101 ~]# jps
3987 Jps
3753 Elasticsearch
[root@elk101 ~]# jmap -heap 3753|grep MaxHeapSizeMaxHeapSize              = 268435456 (256.0MB)

(3)elk101同步环境到其他节点

data_rsync.sh /oldboyedu/softwares/es7/elasticsearch-7.17.5/config/jvm.options

(4)其他节点重启服务并验证堆内存大小

[[e-mail] ~]# systemctl restart es7[[e-mail] ~]# jmap -heap `jps | awk '/Elasticsearch/{print $1}'` | grep MaxHeapSizeMaxHeapSize              = 268435456 (256.0MB)

3.6 ES集群的多实例部署

• 需要解决的问题
  • 程序安装目录
  • 数据目录
  • 日志目录
  • 端口冲突：9200|9300

(1)所有节点准备ES6的工作目录

install -d /oldboyedu/{data,logs,softwares}/es6 -o oldboyedu -g oldboyedu

(2)elk101下载ES6软件包

curl -o elasticsearch-6.8.23.tar.gz http://192.168.15.253/ElasticStack/softwares/binary/elasticsearch-6.8.23.tar.gz

(3)elk101解压软件包
tar xf elasticsearch-6.8.23.tar.gz -C /oldboyedu/softwares/es6/
chown oldboyedu:oldboyedu -R /oldboyedu/softwares/es6/elasticsearch-6.8.23/
(4)elk101修改配置文件

[[e-mail] ~]# yy /oldboyedu/softwares/es6/elasticsearch-6.8.23/config/elasticsearch.yml 
cluster.name: oldboyedu-linux85-es6
node.name: elk101.oldboyedu.com
path.data: /oldboyedu/data/es6
path.logs: /oldboyedu/logs/es6
network.host: 0.0.0.0
http.port: 19200
transport.tcp.port: 19300
discovery.zen.ping.unicast.hosts: ["elk101.oldboyedu.com","elk102.oldboyedu.com","elk103.oldboyedu.com"]
discovery.zen.minimum_master_nodes: 2
[[e-mail] ~]# 

(5)elk101修改堆内存大小

[[e-mail] ~]# vim /oldboyedu/softwares/es6/elasticsearch-6.8.23/config/jvm.options 
...
-Xms256m
-Xmx256m

(6)elk101编写启动脚本

cat > /usr/lib/systemd/system/es6.service <<EOF
[Unit]
Description=oldboyedu linux85 es6
After=network.target[Service]
Type=simple
Environment=JAVA_HOME=/oldboyedu/softwares/jdk1.8.0_291
ExecStart=/oldboyedu/softwares/es6/elasticsearch-6.8.23/bin/elasticsearch
User=oldboyedu
LimitNOFILE=131070[Install]
WantedBy=multi-user.target
EOF

(7)elk101同步配置文件

data_rsync.sh /oldboyedu/softwares/es6/
data_rsync.sh /usr/lib/systemd/system/es6.service

(8)修改各节点的配置文件

[[e-mail] ~]# vim  /oldboyedu/softwares/es6/elasticsearch-6.8.23/config/elasticsearch.yml 
...
node.name: elk102.oldboyedu.com[[e-mail] ~]# vim  /oldboyedu/softwares/es6/elasticsearch-6.8.23/config/elasticsearch.yml 
...
node.name: elk103.oldboyedu.com

(9)启动服务

systemctl daemon-reload && systemctl enable --now es6

(10)查看服务状态

[[e-mail] ~]# curl 10.0.0.101:19200/_cat/nodes
10.0.0.102 50 77  1 0.58 0.25 0.14 mdi - elk102.oldboyedu.com
10.0.0.101 59 82  1 0.37 0.25 0.13 mdi * elk101.oldboyedu.com
10.0.0.103 44 76 42 0.46 0.22 0.12 mdi - elk103.oldboyedu.com

查看集群节点的API：

curl 10.0.0.101:9200/_cat/nodes
[root@elk101 ~]# curl 10.0.0.101:9200/_cat/nodes
10.0.0.103 43 75 0 0.00 0.01 0.05 cdfhilmrstw - elk103.oldboyedu.com
10.0.0.101 35 80 0 0.02 0.03 0.05 cdfhilmrstw - elk101.oldboyedu.com
10.0.0.102 57 84 0 0.00 0.01 0.05 cdfhilmrstw * elk102.oldboyedu.com

查看集群状态:

curl 10.0.0.101:9200/_cluster/health 2>/dev/null | jq
[root@elk101 ~]# curl 10.0.0.101:9200/_cluster/health 2>/dev/null | jq
{"cluster_name": "oldboyedu-linux85-binary","status": "green","timed_out": false,"number_of_nodes": 3,"number_of_data_nodes": 3,"active_primary_shards": 2,"active_shards": 4,"relocating_shards": 0,"initializing_shards": 0,"unassigned_shards": 0,"delayed_unassigned_shards": 0,"number_of_pending_tasks": 0,"number_of_in_flight_fetch": 0,"task_max_waiting_in_queue_millis": 0,"active_shards_percent_as_number": 100
}

3.7 常见报错

1.java.lang.RuntimeException: can not run elasticsearch as root	
报错原因:不能以root用户启动ES服务。
解决方案:使用普通用户启动服务即可。

2.bootstrap check failure [1] of [3]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
报错原因:程序默认的的文件打开数量上线过低。
解决方案:调大文件打开数量上限即可。

3.bootstrap check failure [2] of [3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
报错原因：内核参数虚拟内存映射过低。
解决方案：调大内核虚拟内存映射值即可。

4.bootstrap check failure [3] of [3]: max number of threads [3795] for user [oldboyedu] is too low, increase to at least [4096]
报错原因：程序打开的线程数量设置过低。
解决方案:调大程序打开的线程数量即可。

5.initial heap size [268435456] not equal to maximum heap size [1031798784]; this can cause resize pauses and prevents mlockall from locking the entire heap
报错原因：初始化堆内存和最大堆内存大小不一致。
解决方案:观察配置是否生效，建议将"-Xms"和"-Xmx"值配置一致。

今日作业:

• 完成课堂的所有练习并整理思维导图;
• 使用zabbix监控ES服务

扩展作业:
- 使用ansible的playbook一键搭建ES6和ES7多实例服务