【k8s】Kubernetes版本v1.17.3 kubesphere 3.1.1 默认用户登录失败

1.发帖： 

Kubernetes版本v1.17.3 kubesphere 3.11 默认用户登录失败 - KubeSphere 开发者社区

2. 问题日志： 

2.1问题排查方法 ： 

用户无法登录

http://192.168.56.100:30880/

 2.2查看用户状态 

 kubectl get users

[root@k8s-node1 ~]# kubectl get users
NAME    EMAIL                 STATUS
admin   [e-mail]  

正常的应该是： 

 

2.3 检查 ks-controller-manager 是否正常运行，是否有异常日志：

kubectl -n kubesphere-system logs -l app=ks-controller-manager

kubectl -n kubesphere-system logs -l app=ks-controller-manager
I0911 11:49:58.686749       1 clusterrolebinding_controller.go:188] Successfully synced key:system:controller:pod-garbage-collector
I0911 11:49:58.686755       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"ks-controller-manager-rolebinding", UID:"aefd6d7b-953a-4967-a0a5-86ff738396ab", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"105248", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.686779       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"kubesphere-prometheus-operator", UID:"b1a2af46-1fd9-42a3-a954-118e0fa3b824", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"117259", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.686811       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"system:controller:pod-garbage-collector", UID:"4c2913ad-0dc2-4ce4-9555-79884b51f3ad", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"119", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.696411       1 globalrolebinding_controller.go:204] Successfully synced key:admin
I0911 11:49:58.696471       1 event.go:278] Event(v1.ObjectReference{Kind:"GlobalRoleBinding", Namespace:"", Name:"admin", UID:"e599dd3d-efa7-49c1-9e29-f734ed7c2fd3", APIVersion:"iam.kubesphere.io/v1alpha2", ResourceVersion:"104977", FieldPath:""}): type: 'Normal' reason: 'Synced' GlobalRoleBinding synced successfully
I0911 11:49:58.700995       1 clusterrolebinding_controller.go:188] Successfully synced key:admin-cluster-admin
I0911 11:49:58.701351       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"admin-cluster-admin", UID:"8c1bc9f3-7e49-4198-b951-846c594a04ab", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"107126", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
E0911 11:50:25.054212       1 user_controller.go:239] Internal error occurred: failed calling webhook "users.iam.kubesphere.io": Post https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=4s: context deadline exceeded
E0911 11:50:25.054314       1 basecontroller.go:132] error syncing 'admin' in user-controller: Internal error occurred: failed calling webhook "users.iam.kubesphere.io": Post https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=4s: context deadline exceeded, requeuing 
[root@k8s-node1 ~]# kubectl delete secret -n cattle-system cattle-webhook-tls
Error from server (NotFound): secrets "cattle-webhook-tls" not found

 关键信息; error syncing 'admin' in user-controller: Internal error occurred: failed calling webhook "users.iam.kubesphere.io": Post https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2

3. 问题解决 

参考： 在修改密码和添加用户时报错 - KubeSphere 开发者社区

set ks-controller-manage hostNetwork: true

kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io users.iam.kubesphere.io

问题解决日志： 最后reboot 重启

[root@k8s-node1 ~]# kubectl get users
NAME    EMAIL                 STATUS
admin   [e-mail]   
[root@k8s-node1 ~]# kubectl -n kubesphere-system logs -l app=ks-controller-manager
I0911 11:49:58.686749       1 clusterrolebinding_controller.go:188] Successfully synced key:system:controller:pod-garbage-collector
I0911 11:49:58.686755       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"ks-controller-manager-rolebinding", UID:"aefd6d7b-953a-4967-a0a5-86ff738396ab", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"105248", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.686779       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"kubesphere-prometheus-operator", UID:"b1a2af46-1fd9-42a3-a954-118e0fa3b824", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"117259", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.686811       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"system:controller:pod-garbage-collector", UID:"4c2913ad-0dc2-4ce4-9555-79884b51f3ad", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"119", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
I0911 11:49:58.696411       1 globalrolebinding_controller.go:204] Successfully synced key:admin
I0911 11:49:58.696471       1 event.go:278] Event(v1.ObjectReference{Kind:"GlobalRoleBinding", Namespace:"", Name:"admin", UID:"e599dd3d-efa7-49c1-9e29-f734ed7c2fd3", APIVersion:"iam.kubesphere.io/v1alpha2", ResourceVersion:"104977", FieldPath:""}): type: 'Normal' reason: 'Synced' GlobalRoleBinding synced successfully
I0911 11:49:58.700995       1 clusterrolebinding_controller.go:188] Successfully synced key:admin-cluster-admin
I0911 11:49:58.701351       1 event.go:278] Event(v1.ObjectReference{Kind:"ClusterRoleBinding", Namespace:"", Name:"admin-cluster-admin", UID:"8c1bc9f3-7e49-4198-b951-846c594a04ab", APIVersion:"rbac.authorization.k8s.io/v1", ResourceVersion:"107126", FieldPath:""}): type: 'Normal' reason: 'Synced' ClusterRoleBinding synced successfully
E0911 11:50:25.054212       1 user_controller.go:239] Internal error occurred: failed calling webhook "users.iam.kubesphere.io": Post https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=4s: context deadline exceeded
E0911 11:50:25.054314       1 basecontroller.go:132] error syncing 'admin' in user-controller: Internal error occurred: failed calling webhook "users.iam.kubesphere.io": Post https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=4s: context deadline exceeded, requeuing 
[root@k8s-node1 ~]# kubectl delete secret -n cattle-system cattle-webhook-tls
Error from server (NotFound): secrets "cattle-webhook-tls" not found
[root@k8s-node1 ~]# kubectl delete mutatingwebhookconfigurations.admissionregistration.k8s.io --ignore-not-found=true rancher.cattle.io
[root@k8s-node1 ~]# kubectl delete pod -n cattle-system -l app=rancher-webhook
No resources found
[root@k8s-node1 ~]# kubectl delete secret -n cattle-system cattle-webhook-tls
Error from server (NotFound): secrets "cattle-webhook-tls" not found
[root@k8s-node1 ~]# kubectl delete mutatingwebhookconfigurations.admissionregistration.k8s.io --ignore-not-found=true rancher.cattle.io
[root@k8s-node1 ~]# kubectl delete pod -n cattle-system -l app=rancher-webhook
No resources found
[root@k8s-node1 ~]# set ks-controller-manage hostNetwork: true
[root@k8s-node1 ~]# kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io users.iam.kubesphere.io
validatingwebhookconfiguration.admissionregistration.k8s.io "users.iam.kubesphere.io" deleted
[root@k8s-node1 ~]# systemctl  docker  restart  
Unknown operation 'docker'.
[root@k8s-node1 ~]# reboot 

 reboot之后： 

[root@k8s-node1 ~]# kubectl get users
NAME    EMAIL                 STATUS
admin   [e-mail]   Active
[root@k8s-node1 ~]# 
[root@k8s-node1 ~]# 

 重新登录成功