在k8s 1.26.6上部署ES集群

 

一、k8s集群架构：
IP    角色，左边是ip，右边是hostname master1 是192.168.1.3 的hostname

192.168.1.3    master1
192.168.1.4    master2
192.168.1.5    master3
192.168.1.6    node1
192.168.1.7    node2

二、部署ES集群
1、配置storageclass，用于动态创建pvc，并自动绑定pv

cat sc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer 

执行一下：

kubectl apply -f sc.yaml

2、创建名称空间

kubectl create ns elasticsearch

3、创建PV

cat pv.yaml 

apiVersion: v1
kind: PersistentVolume
metadata:name: local-storage-pv-1namespace: elasticsearchlabels:name: local-storage-pv-1
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/esnodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- master1
---
apiVersion: v1
kind: PersistentVolume
metadata:name: local-storage-pv-2namespace: elasticsearchlabels:name: local-storage-pv-2
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/esnodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- master2
---
apiVersion: v1
kind: PersistentVolume
metadata:name: local-storage-pv-3namespace: elasticsearchlabels:name: local-storage-pv-3
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/esnodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- master3
---
apiVersion: v1
kind: PersistentVolume
metadata:name: local-storage-pv-4namespace: elasticsearchlabels:name: local-storage-pv-4
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/esnodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
apiVersion: v1
kind: PersistentVolume
metadata:name: local-storage-pv-5namespace: elasticsearchlabels:name: local-storage-pv-5
spec:capacity:storage: 1GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: local-storagelocal:path: /data/esnodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2

一共是5个PV，每个都通过nodeSelectorTerms跟k8s节点绑定。
执行一下：

kubectl apply -f pv.yaml

3、创建StatefulSet，ES属于数据库类型的应用，此类应用适合StatefulSet类型

cat sts.yaml 

apiVersion: apps/v1
kind: StatefulSet
metadata:name: es7-clusternamespace: elasticsearch
spec:serviceName: elasticsearch7replicas: 5selector:matchLabels:app: elasticsearch7template:metadata:labels:app: elasticsearch7spec:containers:- name: elasticsearch7image: elasticsearch:7.16.2resources:limits:cpu: 1000mrequests:cpu: 100mports:- containerPort: 9200name: restprotocol: TCP- containerPort: 9300name: inter-nodeprotocol: TCPvolumeMounts:- name: datamountPath: /usr/share/elasticsearch/dataenv:- name: cluster.namevalue: k8s-logs- name: node.namevalueFrom:fieldRef:fieldPath: metadata.name- name: discovery.zen.minimum_master_nodesvalue: "2"- name: discovery.seed_hostsvalue: "es7-cluster-0.elasticsearch7,es7-cluster-1.elasticsearch7,es7-cluster-2.elasticsearch7"- name: cluster.initial_master_nodesvalue: "es7-cluster-0,es7-cluster-1,es7-cluster-2"- name: ES_JAVA_OPTSvalue: "-Xms1g -Xmx1g"initContainers:- name: fix-permissionsimage: busyboxcommand: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]securityContext:privileged: truevolumeMounts:- name: datamountPath: /usr/share/elasticsearch/data- name: increase-vm-max-mapimage: busyboxcommand: ["sysctl", "-w", "vm.max_map_count=262144"]securityContext:privileged: true- name: increase-fd-ulimitimage: busyboxcommand: ["sh", "-c", "ulimit -n 65536"]volumeClaimTemplates:- metadata:name: dataspec:accessModes: [ "ReadWriteOnce" ]storageClassName: "local-storage"resources:requests:storage: 1Gi

该ES集群通过volumeClaimTemplates来关联storageClass，并自动绑定相应的PV。
执行一下：

kubectl apply -f sts.yaml
1
4、创建NodePort类型的Service来蒋ES集群暴漏出去

cat svc.yaml 

apiVersion: v1
kind: Service
metadata:name: elasticsearch7namespace: elasticsearch
spec:selector:app: elasticsearch7type: NodePortports:- port: 9200nodePort: 30002targetPort: 9200

执行一下：

kubectl apply -f svc.yaml

以上就创建完了，我们来看一下上面创建的资源状态：
（1）storageclass

kubectl get sc

NAME            PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-storage   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  21h

（2）PV

kubectl get pv

NAME                 CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                              STORAGECLASS    REASON   AGE
local-storage-pv-1   1Gi        RWO            Retain           Bound    elasticsearch/data-es7-cluster-1   local-storage            21h
local-storage-pv-2   1Gi        RWO            Retain           Bound    elasticsearch/data-es7-cluster-2   local-storage            21h
local-storage-pv-3   1Gi        RWO            Retain           Bound    elasticsearch/data-es7-cluster-0   local-storage            21h
local-storage-pv-4   1Gi        RWO            Retain           Bound    elasticsearch/data-es7-cluster-4   local-storage            19m
local-storage-pv-5   1Gi        RWO            Retain           Bound    elasticsearch/data-es7-cluster-3   local-storage            19m

kubectl get pvc -n elasticsearch

NAME                 STATUS   VOLUME               CAPACITY   ACCESS MODES   STORAGECLASS    AGE
data-es7-cluster-0   Bound    local-storage-pv-3   1Gi        RWO            local-storage   21h
data-es7-cluster-1   Bound    local-storage-pv-1   1Gi        RWO            local-storage   21h
data-es7-cluster-2   Bound    local-storage-pv-2   1Gi        RWO            local-storage   21h
data-es7-cluster-3   Bound    local-storage-pv-5   1Gi        RWO            local-storage   20m
data-es7-cluster-4   Bound    local-storage-pv-4   1Gi        RWO            local-storage   19m

（3）StatefulSet

kubectl get statefulset -n elasticsearch

NAME          READY   AGE
es7-cluster   5/5     21h

kubectl get pod -n elasticsearch

NAME            READY   STATUS    RESTARTS   AGE
es7-cluster-0   1/1     Running   1          21h
es7-cluster-1   1/1     Running   1          21h
es7-cluster-2   1/1     Running   1          21h
es7-cluster-3   1/1     Running   0          19m
es7-cluster-4   1/1     Running   0          18m

（4）Service

kubectl get svc -n elasticsearch

NAME             TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
elasticsearch7   NodePort   10.97.196.176   <none>        9200:30002/TCP   21h

（5）通过restful接口查看ES集群状态

curl 192.168.1.3:30002/_cat/nodes?v

ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
10.244.136.5             60          39   1    0.19    0.31     0.36 cdfhilmrstw -      es7-cluster-0
10.244.104.1             45          41   1    0.98    0.44     0.25 cdfhilmrstw -      es7-cluster-3
10.244.180.5             40          40   1    0.28    0.32     0.32 cdfhilmrstw -      es7-cluster-2
10.244.166.141           63          41   1    0.06    0.14     0.14 cdfhilmrstw -      es7-cluster-4
10.244.137.68            44          41   1    0.29    0.45     0.59 cdfhilmrstw *      es7-cluster-1

通过网页访问也是正常的：

5、部署kibana
kibana我们使用Deployment类型来部署，而它的Service我们使用NodePort，完整的代码如下：

vi kibana.yaml

apiVersion: v1
kind: Service
metadata:name: kibananamespace: elasticsearchlabels:app: kibana
spec:ports:- port: 5601targetPort: 5601nodePort: 30001type: NodePortselector:app: kibana
---
apiVersion: apps/v1
kind: Deployment
metadata:name: kibananamespace: elasticsearchlabels:app: kibana
spec:selector:matchLabels:app: kibanatemplate:metadata:labels:app: kibanaspec:nodeSelector:node: node2containers:- name: kibanaimage: kibana:7.16.2resources:limits:cpu: 1000mrequests:cpu: 1000menv:- name: ELASTICSEARCH_HOSTSvalue: http://elasticsearch7:9200- name: SERVER_PUBLICBASEURLvalue: "0.0.0.0:5601"- name: I18N.LOCALEvalue: zh-CNports: - containerPort: 5601

上面的Deployment部分，我们通过nodeSelector指定调度到了node2节点，前提是给node2节点加了个node=node2这样一个标签。
执行一下：

 kubectl apply -f kibana.yaml 

再查看一下状态：

kubectl get all -n elasticsearchNAME                          READY   STATUS    RESTARTS   AGE
pod/es7-cluster-0             1/1     Running   0          47m
pod/es7-cluster-1             1/1     Running   0          46m
pod/es7-cluster-2             1/1     Running   0          46m
pod/es7-cluster-3             1/1     Running   0          43m
pod/es7-cluster-4             1/1     Running   0          42m
pod/kibana-768595479f-mhw9q   1/1     Running   0          13mNAME                     TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/elasticsearch7   NodePort   10.108.115.19   <none>        9200:30002/TCP   47m
service/kibana           NodePort   10.111.66.207   <none>        5601:30001/TCP   13mNAME                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kibana   1/1     1            1           13mNAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/kibana-768595479f   1         1         1       13mNAME                           READY   AGE
statefulset.apps/es7-cluster   5/5     47m

看到kibana的Service和Pod都在运行中了，通过页面验证一下kibana能不能访问：