k8s 集群搭建详细教程

参考： Kubernetes 文档 / 入门 / 生产环境 / 使用部署工具安装 Kubernetes / 使用 kubeadm 引导集群 / 安装 kubeadm

---

B. 准备开始

• 一台兼容的 Linux 主机。Kubernetes 项目为基于 Debian 和 Red Hat 的 Linux 发行版以及一些不提供包管理器的发行版提供通用的指令
• 每台机器 2 GB 或更多的 RAM （如果少于这个数字将会影响你应用的运行内存)
• 2 CPU 核或更多
• 集群中的所有机器的网络彼此均能相互连接(公网和内网都可以)
• 节点之中不可以有重复的主机名、MAC 地址或 product_uuid。请参见这里了解更多详细信息。
• 开启机器上的某些端口。请参见这里 了解更多详细信息。
• 禁用交换分区。为了保证 kubelet 正常工作，你 必须 禁用交换分区。

U. 确保每个节点上 MAC 地址和 product_uuid 的唯一性

• 你可以使用命令 ip link 或 ifconfig -a 来获取网络接口的 MAC 地址
• 可以使用 sudo cat /sys/class/dmi/id/product_uuid 命令对 product_uuid 校验

一般来讲，硬件设备会拥有唯一的地址，但是有些虚拟机的地址可能会重复。 Kubernetes 使用这些值来唯一确定集群中的节点。 如果这些值在每个节点上不唯一，可能会导致安装 失败。

---

V. 虚拟机

新建… / 创建自定虚拟机 /
​ Linux / Ubuntu 64位

• 设置过程

ID	『虚拟机』设置	建议配置	默认值	说明
1	处理器	-	2	最低要求
2	内存	-	4096 MB	节约内存
3	显示器	取消复选加速 3D 图形	复选	节约内存
4	网络适配器	-	nat	需上网
5	硬盘	40GB	20 GB	保证练习容量
6	选择固件类型	UEFI	传统 BIOS	VMware Fusion 支持嵌套虚拟化

• 设置结果

ID	Your computer’s name	CPU 核	RAM	DISK	NIC
1	k8s-master	4 或更多	8 GB或更多	40 GB	nat
2	k8s-worker1	同上	2 GB或更多	同上	同上
3	k8s-worker2	同上	同上	同上	同上

---

I. 安装 Ubuntu 22.04 LTS

1. Willkommen! Bienvenue! Welcome! Welkom!

       [ English ]
   

2. Installer update available

    [ Continue without updating ]
   

3. Keyboard configuration

   [ Done ]
   

4. Choose type of install

   (X) Ubuntu Server (minimized)
   / [ Done ]
   

5. Network connections

   [ Done ]
   

6. Configure proxy

   [ Done ]
   

7. Configure Ubuntu archive mirror

   Mirror address: http://mirror.nju.edu.cn/ubuntu
   / [ Done ]
   

8. Guided storage configuration

   [ Done ]

   

9. Storage configuration

   [ Done ]
   

10. Profile setup

                             Your name: kiosk
             Your server 's name: k8s-master
                   Pick a username: kiosk
              Choose a password: ubuntu
        Confirm your password: ubuntu
        / [ Done ]

    

11. SSH Setup

    ​ [X] Install OpenSSH server
    ​ / [ Done ]
    

12. Featured Server Snaps

    [ Done ]
    

13. Install complete!

    🅰️ [ Cancel update and reboot ]

    

    🅱️ [ Reboot Now ]

    

14. 建议（可选）

    关机后，做个快照

---

P. 准备工作

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

1. 设置当前用户sudo免密

   sudo tee /etc/sudoers.d/$USER >/dev/null <<EOF
   $USER ALL=(ALL) NOPASSWD:ALL
   EOF
   

2. 使用国内镜像仓库

   # 国内镜像仓库
   MIRROR_URL=http://mirror.nju.edu.cn/ubuntu# 生成软件仓库源
   sudo tee /etc/apt/sources.list >/dev/null <<EOF
   deb $MIRROR_URL jammy main restricted universe multiverse
   deb $MIRROR_URL jammy-updates main restricted universe  multiverse
   deb $MIRROR_URL jammy-backports main restricted universe multiverse
   deb $MIRROR_URL jammy-security main restricted universe multiverse
   EOF
   

3. 安装相关软件

   # 更新
   sudo apt -y update# 安装
   sudo apt install -y openssh-server \vim sshpass nfs-common \bash-completion netcat-openbsd \open-vm-tools
   

[kiosk@k8s-master]$

4. 设置静态IP

   # 配置IP
   sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
   network:ethernets:ens33:dhcp4: falseaddresses: [192.168.147.128/24]gateway4: 192.168.147.2nameservers:addresses: [8.8.8.8]version: 2
   EOF# dns
   sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf# active
   sudo netplan apply
   

[kiosk@k8s-worker1]$

5. 设置静态IP

   # 配置IP
   sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
   network:ethernets:ens33:dhcp4: falseaddresses: [192.168.147.129/24]gateway4: 192.168.147.2nameservers:addresses: [8.8.8.8]version: 2
   EOF# dns
   sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf# active
   sudo netplan apply
   

[kiosk@k8s-worker2]$

6. 设置静态IP

   # 配置IP
   sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
   network:ethernets:ens33:dhcp4: falseaddresses: [192.168.147.130/24]gateway4: 192.168.147.2nameservers:addresses: [8.8.8.8]version: 2
   EOF# dns
   sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf# active
   sudo netplan apply
   

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

7. 编辑 hosts

   sudo tee -a /etc/hosts >/dev/null <<EOF
   192.168.147.128	k8s-master
   192.168.147.129 k8s-worker1
   192.168.147.130 k8s-worker2
   EOF# 设置 root 密码
   (echo ubuntu; echo ubuntu) | sudo passwd root
   echo PermitRootLogin yes | sudo tee -a /etc/ssh/sshd_config
   sudo systemctl restart sshd
   

[kiosk@k8s-master]$

8. ssh免密

   # 生成 keypair
   ssh-keygen -f ~/.ssh/id_rsa -N ''# 拷贝公钥
   for i in k8s-master k8s-worker1 k8s-worker2; dosshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no kiosk@$isshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no root@$i
   done
   

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

9. 禁用swap

   # 交换文件
   SWAPF=$(awk '/swap/ {print $1}' /etc/fstab)# 立即禁用
   sudo swapoff $SWAPF# 永久禁用
   sudo sed -i '/swap/d' /etc/fstab# 删除交换文件
   sudo rm $SWAPF
   

10. 扩容

    # 逻辑卷名
    export LVN=$(sudo lvdisplay | awk '/Path/ {print $3}')# 扩容
    sudo lvextend -l 100%PVS $LVN# 立即生效
    sudo resize2fs $LVN# 验证
    df -h /
    

11. 模块支持

    # 安装
    sudo apt -y install bridge-utils# 立即生效
    sudo modprobe br_netfilter# 内核支持
    sudo tee /etc/sysctl.d/k8s.conf >/dev/null <<EOF
    net.ipv4.ip_forward=1
    vm.swappiness=0
    vm.overcommit_memory=1
    vm.panic_on_oom=0
    EOF# 立即生效
    sudo sysctl -p /etc/sysctl.d/k8s.conf
    

• docker: k8s-master, k8s-worker1
• containerd: k8s-worker2

[kiosk@k8s-master|k8s-worker1]$

12. 安装运行时

    # 创建文件夹
    sudo mkdir -p /etc/docker# 生成配置文件
    sudo tee /etc/docker/daemon.json >/dev/null <<EOF
    {"exec-opts": ["native.cgroupdriver=systemd"],"log-driver": "json-file","log-opts": {"max-size": "100m","max-file": "10"},"registry-mirrors": ["https://docker.nju.edu.cn/"]
    }
    EOF# 安装 runtime
    sudo apt -y install docker.io# 开机自启
    sudo systemctl enable docker# 立即重启
    sudo systemctl restart docker
    

    # 安装命令 cri-dockerd
    curl -# https://vmcc.xyz:8443/k8s/cri-docker/cri-dockerd-0.2.5.amd64.tgz \-o cri-dockerd-0.2.5.amd64.tgz
    tar -xf cri-dockerd-0.2.5.amd64.tgz
    sudo cp cri-dockerd/cri-dockerd /usr/bin/# 安装服务 cri-docker.service
    sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.service \-o /usr/lib/systemd/system/cri-docker.service
    sudo sed -i '/ExecStart/s+$+ --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.8+' /usr/lib/systemd/system/cri-docker.service# 安装 cri-docker.socket
    sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.socket \-o /usr/lib/systemd/system/cri-docker.socket# 启动服务 cri-dockerd
    sudo systemctl daemon-reload
    sudo systemctl enable cri-docker
    sudo systemctl restart cri-docker# 安装 crictl 命令
    curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz \-o crictl-v1.24.2-linux-amd64.tar.gz
    tar -xf crictl-v1.24.2-linux-amd64.tar.gz
    sudo cp crictl /usr/bin/
    # crictl 配置文件
    sudo tee /etc/crictl.yaml >/dev/null <<EOF
    runtime-endpoint: unix:///var/run/cri-dockerd.sock
    image-endpoint: unix:///var/run/cri-dockerd.sock
    timeout: 10
    debug: false
    pull-image-on-create: true
    EOF
    

[kiosk@k8s-worker2]$

13. 安装运行时

    # 安装 containerd
    sudo apt install -y containerd# 创建目录
    sudo mkdir /etc/containerd# 生成默认配置文件
    containerd config default | \
    sudo tee /etc/containerd/config.toml >/dev/null# 修改配置文件
    sudo sed -i \
    -e '/sandbox_image/s?k8s.gcr.io?registry.aliyuncs.com/google_containers?' \
    -e '/SystemdCgroup/s?false?true?' \
    -e '/registry.mirrors/a\        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]' \
    -e '/registry.mirrors/a\          endpoint = ["https://docker.nju.edu.cn/"]' /etc/containerd/config.toml# 服务重启
    sudo systemctl restart containerd# 安装 crictl 命令
    curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz \-o crictl-v1.24.2-linux-amd64.tar.gz
    tar -xf crictl-v1.24.2-linux-amd64.tar.gz
    sudo cp crictl /usr/bin/
    # crictl 配置文件
    sudo tee /etc/crictl.yaml >/dev/null <<EOF
    runtime-endpoint: unix:///run/containerd/containerd.sock
    image-endpoint: unix:///run/containerd/containerd.sock
    timeout: 10
    debug: false
    pull-image-on-create: true
    EOF

K. 安装 k8s

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

1. 安装 kubeadm、kubelet 和 kubectl

   # 更新 apt 包索引并安装使用 Kubernetes apt 仓库所需要的包
   sudo apt -y install apt-transport-https ca-certificates curl# 下载 Google Cloud 公开签名秘钥
   curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -# 添加 Kubernetes apt 仓库
   MIRROR_URL=https://mirror.nju.edu.cn/kubernetes/apt/
   sudo tee /etc/apt/sources.list.d/kubernetes.list >/dev/null <<EOF
   deb $MIRROR_URL kubernetes-xenial main
   EOF# 更新 apt 包索引
   sudo cp /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d
   sudo apt update -y
   sudo apt-cache madison kubelet | grep 1.24# 安装 kubelet、kubeadm 和 kubectl 考试版本
   sudo apt install -y kubelet=1.24.1-00 kubeadm=1.24.1-00 kubectl=1.24.1-00# 锁定版本
   sudo apt-mark hold kubelet kubeadm kubectl

[kiosk@k8s-worker2]$

# 增加 k8s 支持sudo sed -i '/ExecStart=\//s|$| --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --cgroup-driver=systemd|' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# 重启 kubelet 服务
sudo systemctl daemon-reload
sudo systemctl restart kubelet

[kiosk@k8s-master]$

2. 初始化

   # 生成初始文件
   sudo kubeadm config print init-defaults > kubeadm-config.yaml# 修改文件
   sudo sed -i \-e "/advertiseAddress/s?:.*?: 192.168.147.128?" \-e "/name/s?:.*?: k8s-master?" \-e "/clusterName/s?:.*?: ck8s?" \-e "/imageRepository/s?:.*?: registry.aliyuncs.com/google_containers?" \-e "/criSocket/s+containerd/containerd+cri-dockerd+" kubeadm-config.yaml# 使用初始文件，初始化集群
   sudo kubeadm init --config kubeadm-config.yaml
   

   …输出省略…
   Your Kubernetes control-plane has initialized successfully!

   To start using your cluster, you need to run the following as a regular user:

   mkdir -p $HOME/.kube
   sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown $(id -u):$(id -g) $HOME/.kube/config
   

   Alternatively, if you are the root user, you can run:

   export KUBECONFIG=/etc/kubernetes/admin.conf
   

   You should now deploy a pod network to the cluster.
   Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:
   https://kubernetes.io/docs/concepts/cluster-administration/addons/

   Then you can join any number of worker nodes by running the following on each as root:

   kubeadm join 192.168.147.128:6443 --token abcdef.0123456789abcdef \
   --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
   

3. 配置文件

   # 创建目录
   mkdir -p $HOME/.kube# user 复制配置文件
   sudo \cp /etc/kubernetes/admin.conf $HOME/.kube/config
   sudo chown $(id -u):$(id -g) $HOME/.kube/config# root 变量
   sudo tee -a ~root/.bashrc >/dev/null <<EOF
   export KUBECONFIG=/etc/kubernetes/admin.conf
   EOF
   

4. 创建网络

   kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
   

5. 命令补全

   # 立即生效
   source <(kubectl completion bash)# 永久生效
   mkdir ~/.kube
   kubectl completion bash > ~/.kube/completion.bash.inc
   printf "
   # Kubectl shell completion
   source '$HOME/.kube/completion.bash.inc'
   " >> $HOME/.bash_profile
   source $HOME/.bash_profile
   

6. 命令别名

   # 永久生效
   tee -a $HOME/.bashrc >/dev/null <<EOF
   alias k='kubectl'
   complete -F __start_kubectl k
   EOF# 立即生效
   source $HOME/.bashrc
   

[kiosk@k8s-worker1]$

7. 加入集群

   sudo kubeadm join 192.168.147.128:6443 \--token abcdef.0123456789abcdef \--discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f \--cri-socket unix://var/run/cri-dockerd.sock
   

[kiosk@k8s-worker2]$

8. 加入集群

   sudo kubeadm join 192.168.147.128:6443 \--token abcdef.0123456789abcdef \--discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
   

---

C. 确认环境正常

[kiosk@k8s-master]

$ kubectl get componentstatuses
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE                         ERROR
scheduler           `Healthy`  ok
controller-manager  `Healthy`  ok
etcd-0              `Healthy`  {"health":"true","reason":""}$ kubectl get nodes
NAME          STATUS   ROLES                  AGE     VERSION
k8s-worker1  `Ready`  <none>                 4m4s    `v1.24.1`
k8s-worker2  `Ready`  <none>                 4m44s   `v1.24.1`
k8s-master   `Ready`  control-plane,master   13m     `v1.24.1`$ kubectl -n kube-system get pod -w
NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-555bc4b957-8ccgh   1/1     Running   0          27m
calico-node-5qqcq                          1/1     Running   0          9m29s
calico-node-7qclz                          1/1     Running   0          27m
calico-node-kcvt5                          1/1     Running   0          9m29s
coredns-74586cf9b6-69fn7                   1/1     Running   0          156m
coredns-74586cf9b6-8mgl9                   1/1     Running   0          156m
etcd-k8s-master                            1/1     Running   0          156m
kube-apiserver-k8s-master                  1/1     Running   0          156m
kube-controller-manager-k8s-master         1/1     Running   0          156m
kube-proxy-8j248                           1/1     Running   0          9m29s
kube-proxy-g7r55                           1/1     Running   0          9m29s
kube-proxy-rbdcp                           1/1     Running   0          156m
kube-scheduler-k8s-master                  1/1     Running   0          156m
<Ctrl-C>