[云原生案例2.2 ] Kubernetes的部署安装 【单master集群架构 ---- （二进制安装部署）】网络插件部分

1. Kubernetes的网络类别

• 节点网络 : nodeIP
• Pod网络: podIP
• Service网络: clusterIP

2. Kubernetes的接口类型

• CRI 容器运行时接口
• CNI 容器网络接口
• CSI 容器存储接口

3. CNI网络插件 ---- Flannel的介绍及部署

3.1 简介

• Flannel 的功能是让集群中的不同节点主机创建的 Docker 容器都具有全集群唯一的虚拟 IP 地址。
• Flannel 是 Overlay 网络的一种，也是将 TCP 源数据包封装在另一种网络包里面进行路由转发和通信

3.2 flannel的三种模式

• UDP ： 出现最早的模式，但是性能较差，基于flanneld应用程序实现数据包的封装/解封装
• VXLAN ： 默认模式，是推荐使用的模式，性能比UDP模式更好，基于内核实现数据帧的封装/解封装，配置简单使用方便
• HOST-GW ： 性能最好的模式，但是配置复杂，且不能跨网段

3.3 flannel的UDP模式工作原理

1. 原始数据包从源主机的Pod容器发出到cni0网桥接口，再由cni0转发到flannel0虚拟接口
2. flanneld服务进程会监听flannel0接口收到的数据，flanneld进程会将原始数据包封装到UDP报文里
3. flanneld进程会根据在etcd中维护的路由表查到目标Pod所在的nodeIP，并在UDP报文外再封装nodeIP报文、MAC报文，再通过物理网卡发送到目标node节点
4. UDP报文通过8285号端口送到目标node节点的flanneld进程进行解封装，再通过flannel0接口转发到cni0网桥，再由cni0转发到目标Pod容器

3.4 flannel的VXLAN模式工作原理

1. 原始数据帧从源主机的Pod容器发出到cni0网桥接口，再由cni0转发到flannel.1虚拟接口
2. flannel.1接口收到数据帧后添加VXLAN头部，并在内核将原始数据帧封装到UDP报文里
3. flanneld进程根据在etcd维护的路由表将UDP报文通过物理网卡发送到目标node节点
4. UDP报文通过8472号端口送到目标node节点的flannel.1接口在内核进行解封装，再通过flannel.1接口转发到cni0网桥，再由cni0转发到目标Pod容器

3.5 Flannel CNI 网络插件部署

3.5.1 上传flannel镜像文件和插件包到node节点

node01节点

cd /opt/##根据本地文件加载镜像
docker load -i flannel.tar

docker load -i flannel-cni-plugin.tar

#解压CNI 插件包
mkdir /opt/cni/bin -p
tar zxvf cni-plugins-linux-amd64-v1.3.0.tgz -C /opt/cni/bin

scp *.tar *.tgz node02:`pwd`

node02

for i in `ls *.tar`
> do
> docker load -i $i
> done
#加载镜像

#解压CNI 插件包
mkdir /opt/cni/bin -p
tar zxvf cni-plugins-linux-amd64-v1.3.0.tgz -C /opt/cni/bin

3.5.2 在master01节点部署 CNI 网络

#前往master01节点
#上传 kube-flannel.yml 文件到 /opt/k8s 目录中，部署 CNI 网络
cd /opt/k8svim kube-flannel.ymlapiVersion: v1
kind: Namespace
metadata:labels:k8s-app: flannelpod-security.kubernetes.io/enforce: privilegedname: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:k8s-app: flannelname: flannelnamespace: kube-flannel
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:k8s-app: flannelname: flannel
rules:
- apiGroups:- ""resources:- podsverbs:- get
- apiGroups:- ""resources:- nodesverbs:- get- list- watch
- apiGroups:- ""resources:- nodes/statusverbs:- patch
- apiGroups:- networking.k8s.ioresources:- clustercidrsverbs:- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:k8s-app: flannelname: flannel
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: flannel
subjects:
- kind: ServiceAccountname: flannelnamespace: kube-flannel
---
apiVersion: v1
data:cni-conf.json: |{"name": "cbr0","cniVersion": "0.3.1","plugins": [{"type": "flannel","delegate": {"hairpinMode": true,"isDefaultGateway": true}},{"type": "portmap","capabilities": {"portMappings": true}}]}net-conf.json: |{"Network": "10.244.0.0/16","Backend": {"Type": "vxlan"}}
kind: ConfigMap
metadata:labels:app: flannelk8s-app: flanneltier: nodename: kube-flannel-cfgnamespace: kube-flannel
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app: flannelk8s-app: flanneltier: nodename: kube-flannel-dsnamespace: kube-flannel
spec:selector:matchLabels:app: flannelk8s-app: flanneltemplate:metadata:labels:app: flannelk8s-app: flanneltier: nodespec:affinity:nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/osoperator: Invalues:- linuxcontainers:- args:- --ip-masq- --kube-subnet-mgrcommand:- /opt/bin/flanneldenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: EVENT_QUEUE_DEPTHvalue: "5000"image: docker.io/flannel/flannel:v0.21.5name: kube-flannelresources:requests:cpu: 100mmemory: 50MisecurityContext:capabilities:add:- NET_ADMIN- NET_RAWprivileged: falsevolumeMounts:- mountPath: /run/flannelname: run- mountPath: /etc/kube-flannel/name: flannel-cfg- mountPath: /run/xtables.lockname: xtables-lockhostNetwork: trueinitContainers:- args:- -f- /flannel- /opt/cni/bin/flannelcommand:- cpimage: docker.io/flannel/flannel-cni-plugin:v1.1.2name: install-cni-pluginvolumeMounts:- mountPath: /opt/cni/binname: cni-plugin- args:- -f- /etc/kube-flannel/cni-conf.json- /etc/cni/net.d/10-flannel.conflistcommand:- cpimage: docker.io/flannel/flannel:v0.21.5name: install-cnivolumeMounts:- mountPath: /etc/cni/net.dname: cni- mountPath: /etc/kube-flannel/name: flannel-cfgpriorityClassName: system-node-criticalserviceAccountName: flanneltolerations:- effect: NoScheduleoperator: Existsvolumes:- hostPath:path: /run/flannelname: run- hostPath:path: /opt/cni/binname: cni-plugin- hostPath:path: /etc/cni/net.dname: cni- configMap:name: kube-flannel-cfgname: flannel-cfg- hostPath:path: /run/xtables.locktype: FileOrCreatename: xtables-lock

#将 kube-flannel.yml 文件中的配置应用到 Kubernetes 集群中
#创建或更新在 YAML 文件中定义的所有资源对象
kubectl apply -f kube-flannel.yml

#这个 YAML 文件包含了创建 Flannel 网络插件所需的资源对象。以下是文件中包含的资源对象的简要说明：#- namespace: kube-flannel：创建一个名为 kube-flannel 的命名空间，用于隔离 Flannel 插件的相关资源。
#- serviceaccount: flannel：创建一个名为 flannel 的服务账户，用于授权 Flannel 插件与 Kubernetes API 服务器进行通信。
#- clusterrole: flannel：创建一个名为 flannel 的集群角色，定义了 Flannel 插件所需的权限。
#- clusterrolebinding: flannel：创建一个名为 flannel 的集群角色绑定，将 flannel 服务账户与 flannel 集群角色绑定，以授予 flannel 服务账户所需的权限。
#- configmap: kube-flannel-cfg：创建一个名为 kube-flannel-cfg 的配置映射，其中包含了 Flannel 插件的配置信息。
#- daemonset: kube-flannel-ds：创建一个名为 kube-flannel-ds 的 DaemonSet，用于在集群中的每个节点上部署和管理 Flannel 插件的 Pod。通过应用这个 YAML 文件,创建了 Flannel 网络插件所需的各种资源对象。
可以使用 `kubectl get` 命令来查看这些资源对象的状态和详细信息。

#获取 kube-flannel 命名空间中的所有 Pod 的信息
kubectl get pods -n kube-flannel
或者
kubectl get pods -A#-A显示所有信息

#补充，卸载插件
kubectl delete -f kube-flannel.yml

3.5.3 查看集群的节点状态

kubectl get nodes

#切换到node节点
ifconfig

4.CoreDNS 的简单介绍与部署

4.1 简介

• CoreDNS 是 K8S 的默认 DNS 实现
• 根据 service 资源名称 解析出对应的 clusterIP
• 根据 statefulset 控制器创建的 Pod 资源名称 解析出对应的 podIP

4.2 CoreDNS的部署

4.2.1 构建coredns镜像 ---- 所有node节点

#上传 coredns.tar 到 /opt 目录中
cd /opt
docker load -i coredns.tar

scp coredns.tar node02:/opt

cd /opt
docker load -i coredns.tar

4.2.2 编写CoreDNS配置文件

vim coredns.yaml# __MACHINE_GENERATED_WARNING__apiVersion: v1
kind: ServiceAccount
metadata:name: corednsnamespace: kube-systemlabels:kubernetes.io/cluster-service: "true"addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:kubernetes.io/bootstrapping: rbac-defaultsaddonmanager.kubernetes.io/mode: Reconcilename: system:coredns
rules:
- apiGroups:- ""resources:- endpoints- services- pods- namespacesverbs:- list- watch
- apiGroups:- ""resources:- nodesverbs:- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:annotations:rbac.authorization.kubernetes.io/autoupdate: "true"labels:kubernetes.io/bootstrapping: rbac-defaultsaddonmanager.kubernetes.io/mode: EnsureExistsname: system:coredns
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:coredns
subjects:
- kind: ServiceAccountname: corednsnamespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:name: corednsnamespace: kube-systemlabels:addonmanager.kubernetes.io/mode: EnsureExists
data:Corefile: |.:53 {errorshealth {lameduck 5s}readykubernetes cluster.local in-addr.arpa ip6.arpa {pods insecurefallthrough in-addr.arpa ip6.arpattl 30}prometheus :9153forward . /etc/resolv.conf {max_concurrent 1000}cache 30loopreloadloadbalance}
---
apiVersion: apps/v1
kind: Deployment
metadata:name: corednsnamespace: kube-systemlabels:k8s-app: kube-dnskubernetes.io/cluster-service: "true"addonmanager.kubernetes.io/mode: Reconcilekubernetes.io/name: "CoreDNS"
spec:# replicas: not specified here:# 1. In order to make Addon Manager do not reconcile this replicas parameter.# 2. Default is 1.# 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.strategy:type: RollingUpdaterollingUpdate:maxUnavailable: 1selector:matchLabels:k8s-app: kube-dnstemplate:metadata:labels:k8s-app: kube-dnsspec:securityContext:seccompProfile:type: RuntimeDefaultpriorityClassName: system-cluster-criticalserviceAccountName: corednsaffinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- weight: 100podAffinityTerm:labelSelector:matchExpressions:- key: k8s-appoperator: Invalues: ["kube-dns"]topologyKey: kubernetes.io/hostnametolerations:- key: "CriticalAddonsOnly"operator: "Exists"nodeSelector:kubernetes.io/os: linuxcontainers:- name: corednsimage: registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0imagePullPolicy: IfNotPresentresources:limits:memory: 170Mirequests:cpu: 100mmemory: 70Miargs: [ "-conf", "/etc/coredns/Corefile" ]volumeMounts:- name: config-volumemountPath: /etc/corednsreadOnly: trueports:- containerPort: 53name: dnsprotocol: UDP- containerPort: 53name: dns-tcpprotocol: TCP- containerPort: 9153name: metricsprotocol: TCPlivenessProbe:httpGet:path: /healthport: 8080scheme: HTTPinitialDelaySeconds: 60timeoutSeconds: 5successThreshold: 1failureThreshold: 5readinessProbe:httpGet:path: /readyport: 8181scheme: HTTPsecurityContext:allowPrivilegeEscalation: falsecapabilities:add:- NET_BIND_SERVICEdrop:- allreadOnlyRootFilesystem: truednsPolicy: Defaultvolumes:- name: config-volumeconfigMap:name: corednsitems:- key: Corefilepath: Corefile
---
apiVersion: v1
kind: Service
metadata:name: kube-dnsnamespace: kube-systemannotations:prometheus.io/port: "9153"prometheus.io/scrape: "true"labels:k8s-app: kube-dnskubernetes.io/cluster-service: "true"addonmanager.kubernetes.io/mode: Reconcilekubernetes.io/name: "CoreDNS"
spec:selector:k8s-app: kube-dnsclusterIP: 10.0.0.2ports:- name: dnsport: 53protocol: UDP- name: dns-tcpport: 53protocol: TCP- name: metricsport: 9153protocol: TCP

4.2.3 部署 CoreDNS ---- master01节点

#上传 coredns.yaml 文件到 /opt/k8s 目录中，部署 CoreDNS 
cd /opt/k8s#Kubernetes读取配置文件并更新 CoreDNS 的配置
kubectl apply -f coredns.yaml

#查看运行在 kube-system 命名空间中的 Kubernetes Pod 列表
kubectl get pods -n kube-system 

#在Kubernetes 集群中启动一个临时的 Busybox 容器，并进入其交互式终端
kubectl run -it --rm dns-test --image=busybox:1.28.4 shIf you don't see a command prompt, try pressing enter.
/ # nslookup kubernetes