二十一、Ingress 进阶实践

架构参考

使用hostnetwork，推荐的方式，使用单独的物理服务器，不部署业务pod的主机。

一、Ingress Nginx Controller 安装

采用helm的安装方式，进行部署。

官网地址：
https://kubernetes.github.io/ingress-nginx/deploy/

github地址：
https://github.com/kubernetes/ingress-nginx

1. 首先安装 Helm(已安装)

# wget https://get.helm.sh/helm-v3.6.3-linux-amd64.tar.gz
# tar -zxvf helm-v3.0.0-linux-amd64.tar.gz
# mv linux-amd64/helm /usr/local/bin/helm[root@k8s-master01 ingress-nginx]#helm version
version.BuildInfo{Version:"v3.15.1", GitCommit:"e211f2aa62992bd72586b395de50979e31231829", GitTreeState:"clean", GoVersion:"go1.22.3"}

下载 Ingress Nginx Controller 安装包

[root@k8s-master01 ingress-up]#helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories[root@k8s-master01 ingress-up]#helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "grafana" chart repository
Update Complete. ⎈Happy Helming!⎈[root@k8s-master01 ingress-up]#helm repo list
NAME            URL
bitnami         https://charts.bitnami.com/bitnami
grafana         https://grafana.github.io/helm-charts
ingress-nginx   https://kubernetes.github.io/ingress-nginx#查看现有包版本
[root@k8s-master01 ingress]#helm search repo ingress-nginx
NAME                            CHART VERSION   APP VERSION     DESCRIPTION
ingress-nginx/ingress-nginx     4.11.2          1.11.2          Ingress controller for Kubernetes using NGINX a...[root@k8s-master01 ingress-up]#helm pull ingress-nginx/ingress-nginx --version 4.0.1
[root@k8s-master01 ingress-up]#ls
helm-v3.6.3-linux-amd64.tar.gz  ingress-nginx-4.0.1.tgz

更改对应的配置

[root@k8s-master01 ingress-up]#tar xf ingress-nginx-4.0.1.tgz
[root@k8s-master01 ingress-up]#cd ingress-nginx/
[root@k8s-master01 ingress-nginx]#ls
CHANGELOG.md  Chart.yaml  ci  OWNERS  README.md  templates  values.yaml
[root@k8s-master01 ingress-nginx]#vim values.yaml

需要修改的位置

1. Controller 和 admissionWebhook 的镜像地址，需要将公网镜像同步至公司内网镜像仓库（需要自行同步 gcr 镜像，一样的版本可以直接用registry.cn-beijing.aliyuncs.com/dotbalo/controller:v1.0.0 和 registry.cn-beijing.aliyuncs.com/dotbalo/kube-webhook-certgen:v1.0 ）
2. 镜像的 digest 值注释#
3. hostNetwork 设置为 true
4. dnsPolicy 设置为 ClusterFirstWithHostNet，改了就能解析了
5. NodeSelector 添加 ingress: "true"部署至指定节点
6. 类型更改为 kind: DaemonSet
7. 将 ingress nginx 设置为默认的default:true，如果有多个可以不用设置默认，不加需要再ingress配置文件上单独写指定。

# 部署 ingress，给需要部署 ingress 的节点上打标签
[root@k8s-master01 ingress-nginx]#kubectl label node k8s-node02 ingress=true
node/k8s-node02 labeled# 创建命名空间
[root@k8s-master01 ingress-nginx]#kubectl create ns ingress-nginx
Error from server (AlreadyExists): namespaces "ingress-nginx" already exists# 检查有没用helm装过
[root@k8s-master01 ingress-nginx]#helm list -n ingress-nginx
NAME  NAMESPACE REVISION UPDATED STATUS CHART APP VERSION[root@k8s-master01 ingress-nginx]#helm install ingress-nginx -n ingress-nginx .# 注意有点.
# 其他命令，删除，更新
# helm delete
# helm upgrade# 检查
[root@k8s-master01 ingress]#kubectl get po -n ingress-nginx
NAME                             READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-wmxnm   1/1     Running   0          8m22s
[root@k8s-master01 ingress]#ssh k8s-node02
Last login: Thu Jun 13 17:48:55 2024 from 10.1.3.92
[root@k8s-node02 ~]# netstat -lnpt |grep 80
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      15851/nginx: master
tcp6       0      0 :::80                   :::*                    LISTEN      15851/nginx: master
[root@k8s-node02 ~]# ps aux | grep nginx
......
controller --publish-service=ingress-nginx/ingress-nginx-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --configmap=ingress-nginx/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key

二、Ingress Nginx入门使用

创建一个用于学习 Ingress 的 Namespace，之后所有的操作都在此 Namespace 进行：

[root@k8s-master01 ingress]# kubectl create ns study-ingress
namespace/study-ingress created

创建一个简单的 Nginx 模拟 Web 服务：

[root@k8s-master01 ingress]#kubectl create deploy nginx --image=registry.cn-beijing.aliyuncs.com/dotbalo/nginx:1.15.12 -n study-ingress
deployment.apps/nginx created

然后创建该 Web 容器的 Service：

[root@k8s-master01 ingress]# kubectl expose deploy nginx --port 80 -n study-ingress
service/nginx exposed
[root@k8s-master01 ingress-up]#kubectl get svc -n study-ingress
NAME    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
nginx   ClusterIP   10.96.226.26   <none>        80/TCP    132m#测试连接
[root@k8s-master01 ingress]#curl 10.96.226.26
.....
<style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;
.....
</body>
</html>

之后创建 Ingress 指向上面创建的 Service：

# vim web-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-ingressnamespace: study-ingress
spec:rules:- host: nginx.test.comhttp:paths:- backend:service:name: nginx                      # pod 的service 名称port:number: 80path: /pathType: ImplementationSpecific# 查看，ingress和service是同一个命名空间
[root@k8s-master01 ingress-up] # kubectl get ingress -n study-ingress
NAME            CLASS   HOSTS            ADDRESS   PORTS   AGE
nginx-ingress   nginx   nginx.test.com             80      6m28s
[root@k8s-master01 ingress-up] # kubectl get svc -n study-ingress
NAME    TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
nginx   ClusterIP   10.96.226.26   <none>        80/TCP    136m# ingress-nginx-controller是全局的，能够自动识别ingress
[root@k8s-master01 ingress-up] # kubectl get po -n ingress-nginx
NAME                             READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-wmxnm   1/1     Running   0          15h[root@k8s-master01 ingress-up] # kubectl exec -it ingress-nginx-controller-wmxnm -n ingress-nginx -- bash
bash-5.1$ grep "nginx.test.com"  nginx.conf## start server nginx.test.comserver_name nginx.test.com ;## end server nginx.test.com
# 可以看见已经有配置了。

创建该 Ingress：

[root@k8s-master01 ingress-up]# kubectl create -f web-ingress.yaml
ingress.networking.k8s.io/nginx-ingress created

注意：只能通过ingress-controller去访问，所以，在哪台上安装的controller就是访问哪台。此处安装在node02上

[root@k8s-master01 ingress-up]#kubectl get po -n ingress-nginx -owide
NAME                             READY   STATUS    RESTARTS   AGE   IP          NODE         NOMINATED NODE   READINESS GATES
ingress-nginx-controller-wmxnm   1/1     Running   0          15h   10.1.3.35   k8s-node02   <none>           <none>

curl验证

[root@k8s-master01 ingress-up]#curl -H "Host:nginx.test.com" 10.1.3.35
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
</body>
</html>

修改host验证

查看日志

[root@k8s-master01 ingress-up]#kubectl get po -n study-ingress
NAME                     READY   STATUS    RESTARTS   AGE
nginx-745cfd564b-qlqr7   1/1     Running   0          155m
[root@k8s-master01 ingress-up]#kubectl logs -f nginx-745cfd564b-qlqr7 -n study-ingress
172.16.32.128 - - [07/Jul/2024:06:08:44 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
172.16.58.192 - - [07/Jul/2024:06:49:38 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0"