K8s集群里Nginx和Traefik怎么和平共处？一个真实场景下的双Ingress Controller配置实战

Kubernetes集群中Nginx与Traefik双Ingress Controller共存实践在Kubernetes生产环境中我们经常会遇到需要同时运行多个Ingress Controller的场景。比如一个已经稳定运行Nginx Ingress Controller的集群现在希望引入Traefik来管理特定Namespace的流量。本文将深入探讨如何实现这一目标解决实际部署中的各种挑战。1. 双Ingress Controller共存的核心挑战在Kubernetes集群中同时部署Nginx和Traefik作为Ingress Controller主要面临以下几个技术难点端口冲突问题默认情况下两个Ingress Controller都会尝试监听相同的HTTP(80)和HTTPS(443)端口。如果不做特殊处理后启动的Controller会因为端口已被占用而无法正常工作。路由分层管理我们需要明确哪些流量由Nginx处理哪些由Traefik处理。常见的做法是让Nginx作为第一层入口将特定域名的流量转发给Traefik进行第二层路由。权限与RBAC配置Traefik需要足够的权限来监控和处理Ingress资源特别是在多Namespace环境下需要仔细配置ServiceAccount、ClusterRole和ClusterRoleBinding。配置热更新机制Traefik以其动态配置能力著称我们需要确保这一特性在双Controller环境下仍然能够正常工作。2. 解决方案架构设计针对上述挑战我们采用以下架构方案外部请求 → Nginx Ingress (80/443) → 根据Host头路由 → - 常规流量: 直接由Nginx路由到后端服务 - 特殊流量: 转发到Traefik Service → Traefik Ingress → 最终后端服务这种分层架构的优势在于保持现有Nginx配置不变最小化对生产环境的影响可以逐步将特定服务迁移到Traefik管理利用Nginx成熟的稳定性作为第一层防护充分发挥Traefik的动态配置优势处理特定业务3. 详细实施步骤3.1 准备工作与命名空间规划首先我们为Traefik创建专用的命名空间kubectl create namespace traefik-system建议的命名空间规划ingress-nginx: 现有的Nginx Ingress Controllertraefik-system: 新部署的Traefik组件apps: 业务应用部署的命名空间3.2 部署Traefik CRD和RBACTraefik使用Custom Resource Definitions(CRD)来扩展Kubernetes API我们需要先部署这些CRDkubectl apply -f https://raw.githubusercontent.com/traefik/traefik/v2.11/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml然后创建RBAC配置确保Traefik有足够的权限# traefik-rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: namespace: traefik-system name: traefik-ingress-controller --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: traefik-ingress-controller rules: - apiGroups: - resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - networking.k8s.io resources: - ingresses verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - ingressroutes - middlewares - tlsoptions verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: traefik-system应用RBAC配置kubectl apply -f traefik-rbac.yaml3.3 部署Traefik Controller创建Traefik的配置文件ConfigMap# traefik-config.yaml apiVersion: v1 kind: ConfigMap metadata: name: traefik-config namespace: traefik-system data: traefik.yaml: | log: level: INFO accessLog: {} api: insecure: true dashboard: true providers: kubernetesCRD: {} kubernetesIngress: {} entryPoints: web: address: :8000 # 使用非标准端口避免冲突 websecure: address: :8443 # 使用非标准端口避免冲突 dashboard: address: :8080然后部署Traefik Deployment# traefik-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: namespace: traefik-system name: traefik labels: app: traefik spec: replicas: 2 selector: matchLabels: app: traefik template: metadata: labels: app: traefik spec: serviceAccountName: traefik-ingress-controller containers: - name: traefik image: traefik:v2.11 ports: - name: web containerPort: 8000 - name: websecure containerPort: 8443 - name: dashboard containerPort: 8080 volumeMounts: - name: config mountPath: /etc/traefik readOnly: true volumes: - name: config configMap: name: traefik-config创建Traefik Service# traefik-service.yaml apiVersion: v1 kind: Service metadata: name: traefik namespace: traefik-system spec: selector: app: traefik ports: - name: web port: 8000 targetPort: web - name: websecure port: 8443 targetPort: websecure - name: dashboard port: 8080 targetPort: dashboard应用所有配置kubectl apply -f traefik-config.yaml -f traefik-deployment.yaml -f traefik-service.yaml3.4 配置Nginx转发规则现在我们需要配置Nginx Ingress将特定域名的流量转发到Traefik。这里以traefik.example.com为例# nginx-to-traefik-ingress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: traefik-proxy namespace: traefik-system annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules: - host: traefik.example.com http: paths: - path: / pathType: Prefix backend: service: name: traefik port: number: 8000这个配置告诉Nginx Ingress将所有发送到traefik.example.com的请求转发到traefik-system命名空间中的traefikService的8000端口即转发到我们部署的Traefik Controller3.5 测试Traefik路由配置为了验证Traefik是否正常工作我们可以部署一个测试服务# whoami-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: whoami namespace: default spec: selector: matchLabels: app: whoami replicas: 2 template: metadata: labels: app: whoami spec: containers: - name: whoami image: containous/whoami ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: whoami namespace: default spec: selector: app: whoami ports: - protocol: TCP port: 80 targetPort: 80然后创建Traefik特有的IngressRoute# whoami-ingressroute.yaml apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami namespace: default spec: entryPoints: - web routes: - match: Host(traefik.example.com) PathPrefix(/whoami) kind: Rule services: - name: whoami port: 80应用这些配置后我们可以测试curl http://traefik.example.com/whoami应该能看到whoami服务的响应证明整个链路已经打通请求首先到达Nginx IngressNginx根据Host头转发到Traefik ServiceTraefik根据IngressRoute规则将请求路由到whoami服务4. 高级配置与优化4.1 启用HTTPS支持为了启用HTTPS我们需要为Traefik配置TLS证书。这里以Lets Encrypt为例首先创建Lets Encrypt生产环境的ClusterIssuer# letsencrypt-prod.yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: email: your-emailexample.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-prod solvers: - http01: ingress: class: nginx然后更新Traefik的IngressRoute# whoami-ingressroute-tls.yaml apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: whoami-tls namespace: default spec: entryPoints: - websecure routes: - match: Host(traefik.example.com) PathPrefix(/whoami) kind: Rule services: - name: whoami port: 80 tls: certResolver: letsencrypt4.2 监控与指标收集Traefik内置了Prometheus指标端点我们可以通过以下配置启用更新Traefik的ConfigMap# 在traefik.yaml中添加 metrics: prometheus: entryPoint: metrics然后添加新的entryPointentryPoints: metrics: address: :8082更新Service暴露metrics端口# 在traefik-service.yaml中添加 - name: metrics port: 8082 targetPort: metrics4.3 资源限制与HPA为确保Traefik稳定运行建议设置资源限制# 在traefik-deployment.yaml的container部分添加 resources: limits: cpu: 1 memory: 512Mi requests: cpu: 100m memory: 128Mi同时可以配置Horizontal Pod AutoscalerapiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: traefik namespace: traefik-system spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: traefik minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 705. 生产环境注意事项在实际生产环境中部署双Ingress Controller时还需要考虑以下因素性能考量两层代理会增加约10-20ms的延迟确保节点有足够的资源处理额外流量监控两个Ingress Controller的指标确保没有性能瓶颈安全最佳实践限制Traefik Dashboard的访问仅允许内部IP为不同环境使用不同的证书定期轮换TLS证书和凭据故障排查技巧检查Nginx和Traefik的日志是否显示正常转发使用kubectl get ingress,ingressroute确认资源状态通过kubectl port-forward直接访问服务隔离问题版本升级策略先在一个非关键环境测试新版本保持Nginx和Traefik版本的兼容性准备好回滚方案特别是CRD变更可能不向后兼容