K8s 镜像缓存管理 kube-fledged 认知
写在前面
- 博文内容为K8s 镜像缓存管理 kube-fledged 认知
- 内容涉及：
- kube-fledged 简单介绍
- 部署以及基本使用
- 理解不足小伙伴帮忙指正
不必太纠结于当下，也不必太忧虑未来，当你经历过一些事情的时候，眼前的风景已经和从前不一样了。——村上春树
简单介绍
我们知道 k8s 上的容器调度需要在调度的节点行拉取当前容器的镜像，在一些特殊场景中，
- 需要
快速启动和/或扩展的应用程序。例如，由于数据量激增，执行实时数据处理的应用程序需要快速扩展。 - 镜像比较庞大，涉及多个版本，节点存储有限，需要动态清理不需要的镜像
无服务器函数通常需要在几分之一秒内立即对传入事件和启动容器做出反应。- 在边缘设备上运行的
IoT 应用程序，需要容忍边缘设备和镜像镜像仓库之间的间歇性网络连接。 - 如果需要从
专用仓库中拉取镜像，并且无法授予每个人从此镜像仓库拉取镜像的访问权限，则可以在群集的节点上提供镜像。 - 如果集群管理员或操作员需要对应用程序进行升级，并希望事先验证是否可以成功拉取新镜像。
kube-fledged 是一个 kubernetes operator，用于直接在 Kubernetes 集群的 worker 节点上创建和管理容器镜像缓存。它允许用户定义镜像列表以及这些镜像应缓存到哪些工作节点上（即拉取）。因此，应用程序 Pod 几乎可以立即启动，因为不需要从镜像仓库中提取镜像。
kube-fledged 提供了 CRUD API 来管理镜像缓存的生命周期，并支持多个可配置的参数，可以根据自己的需要自定义功能。
Kubernetes 具有内置的镜像垃圾回收机制。节点中的 kubelet 会定期检查磁盘使用率是否达到特定阈值（可通过标志进行配置）。一旦达到这个阈值，kubelet 会自动删除节点中所有未使用的镜像。
需要在建议的解决方案中实现自动和定期刷新机制。如果镜像缓存中的镜像被 kubelet 的 gc 删除，下一个刷新周期会将已删除的镜像拉入镜像缓存中。这可确保镜像缓存是最新的。
设计流程
https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png
部署 kube-fledged
Helm 方式部署
──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$mkdir kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$cd kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$export KUBEFLEDGED_NAMESPACE=kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE}
namespace/kube-fledged created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/
"kubefledged-charts" has been added to your repositories
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubefledged-charts" chart repository
...Successfully got an update from the "kubescape" chart repository
...Successfully got an update from the "rancher-stable" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "openkruise" chart repository
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "botkube" chart repository
Update Complete. ⎈Happy Helming!⎈
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait
实际部署中发现，由于网络问题，chart 无法下载，所以通过 make deploy-using-yaml 使用 yaml 方式部署
Yaml 文件部署
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$git clone https://github.com/senthilrch/kube-fledged.git
正克隆到 'kube-fledged'...
remote: Enumerating objects: 10613, done.
remote: Counting objects: 100% (1501/1501), done.
remote: Compressing objects: 100% (629/629), done.
remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112
接收对象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done.
处理 delta 中: 100% (4431/4431), done.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$ls
kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$cd kube-fledged/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
第一次部署，发现镜像拉不下来
┌──[root@vms100.liruilongs.github.io]-[~]
└─$kubectl get all -n kube-fledged
NAME READY STATUS RESTARTS AGE
pod/kube-fledged-controller-df69f6565-drrqg 0/1 CrashLoopBackOff 35 (5h59m ago) 21h
pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2 0/1 Init:CrashLoopBackOff 35 (5h58m ago) 21h
pod/kubefledged-controller-55f848cc67-7f4rl 1/1 Running 0 21h
pod/kubefledged-webhook-server-597dbf4ff5-l8fbh 0/1 Init:CrashLoopBackOff 34 (6h ago) 21hNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 21h
service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 21hNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kube-fledged-controller 0/1 1 0 21h
deployment.apps/kube-fledged-webhook-server 0/1 1 0 21h
deployment.apps/kubefledged-controller 0/1 1 0 21h
deployment.apps/kubefledged-webhook-server 0/1 1 0 21hNAME DESIRED CURRENT READY AGE
replicaset.apps/kube-fledged-controller-df69f6565 1 1 0 21h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 0 21h
replicaset.apps/kubefledged-controller-55f848cc67 1 1 0 21h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 0 21h
┌──[root@vms100.liruilongs.github.io]-[~]
└─$
这里我们找一下要拉取的镜像
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat *.yaml | grep image:- image: senthilrch/kubefledged-controller:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0
单独拉取一些，当前使用 ansible 在所有工作节点批量操作
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml
其他相关的镜像都拉取一下
操作完成之后容器状态全部正常
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl -n kube-fledged get all
NAME READY STATUS RESTARTS AGE
pod/kube-fledged-controller-df69f6565-wdb4g 1/1 Running 0 13h
pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp 1/1 Running 0 13h
pod/kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 13h
pod/kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 13hNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 36h
service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 36hNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kube-fledged-controller 1/1 1 1 36h
deployment.apps/kube-fledged-webhook-server 1/1 1 1 36h
deployment.apps/kubefledged-controller 1/1 1 1 36h
deployment.apps/kubefledged-webhook-server 1/1 1 1 36hNAME DESIRED CURRENT READY AGE
replicaset.apps/kube-fledged-controller-df69f6565 1 1 1 36h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 1 36h
replicaset.apps/kubefledged-controller-55f848cc67 1 1 1 36h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 1 36h
验证是否安装成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get pods -n kube-fledged -l app=kubefledged
NAME READY STATUS RESTARTS AGE
kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 16h
kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 16h
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
使用 kubefledged
创建镜像缓存对象
根据 Demo 文件，创建镜像缓存对象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$cd deploy/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- ghcr.io/jitesoft/nginx:1.23.1# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- us.gcr.io/k8s-artifacts-prod/cassandra:v7- us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0nodeSelector:tier: backend# Specifies a list of image pull secrets to pull images from private repositories into the cacheimagePullSecrets:- name: myregistrykey
官方的 Demo 中对应的 镜像拉取不下来，所以换一下
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7
Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
为了测试选择器标签的使用,我们找一个节点的标签单独做镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get nodes --show-labels
同时我们直接从公有仓库拉取镜像，所以不需要 imagePullSecrets 对象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml
修改后的 yaml 文件
- 添加了一个所有节点的 liruilong/my-busybox:latest 镜像缓存
- 添加了一个
kubernetes.io/hostname: vms105.liruilongs.github.io对应标签选择器的liruilong/hikvision-sdk-config-ftp:latest镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
直接创建报错了
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io")
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
解决办法，删除对应的对象，重新创建
我在当前项目的一个 issues 下面找到了解决办法 https://github.com/senthilrch/kube-fledged/issues/76
看起来这是因为 Webhook CA 是硬编码的，但是当 webhook 服务器启动时，会生成一个新的 CA 捆绑包并更新 webhook 配置。当发生另一个部署时，将重新应用原始 CA 捆绑包，并且 Webhook 请求开始失败，直到再次重新启动 Webhook 组件以修补捆绑包init-server
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make remove-kubefledged-and-operator
# Remove kubefledged
kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml
error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2"
ensure CRDs are installed first
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
namespace/kube-fledged created
kubectl apply -f deploy/kubefledged-crd.yaml
customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged
....................
kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch
Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available...
deployment "kubefledged-webhook-server" successfully rolled out
kubectl get pods -n kube-fledged
NAME READY STATUS RESTARTS AGE
kubefledged-controller-55f848cc67-76c4v 1/1 Running 0 112s
kubefledged-webhook-server-597dbf4ff5-56h6z 1/1 Running 0 66s
重新创建缓存对象，创建成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
imagecache.kubefledged.io/imagecache1 created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
NAME AGE
imagecache1 10s
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
查看当前被纳管的镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl get imagecaches imagecache1 -n kube-fledged -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 83,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20169836","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:06:47Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheRefresh","startTime": "2024-03-02T01:05:33Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$
通过 ansible 来验证
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | CHANGED | rc=0 >>
liruilong/hikvision-sdk-config-ftp latest a02cd03b4342 4 months ago 830MB
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
开启自动刷新
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=
imagecache.kubefledged.io/imagecache1 annotated
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
添加镜像缓存
添加一个新的镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 92,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175233","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest","liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:43:32Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:40:34Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
通过 ansible 确认
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.102 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
删除镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 94,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175766","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"message": "Image cache is being updated. Please view the status after some time","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:48:03Z","status": "Processing"}
}
通过 Ansible 确认，可以看到无论是 mastere 上的节点还是 work 的节点，对应的镜像缓存都被清理
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
这里需要注意如果清除所有的镜像缓存，那么需要把 images 下的数组 写成 “”.
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 98,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20176849","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": [""]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:52:16Z","message": "All cached images succesfully deleted from respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:51:47Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$
如果通过下面的方式删除，直接注释调对应的标签
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster#- images:#- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$
那么会报下面的错
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists
You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.
博文部分内容参考
© 文中涉及参考链接内容版权归原作者所有，如有侵权请告知,如果你认可它不要吝啬星星哦 😃
https://github.com/senthilrch/kube-fledged
© 2018-2024 liruilonger@gmail.com, All rights reserved. 保持署名-非商用-相同方式共享(CC BY-NC-SA 4.0)
相关文章：
K8s 镜像缓存管理 kube-fledged 认知
写在前面 博文内容为K8s 镜像缓存管理 kube-fledged 认知内容涉及： kube-fledged 简单介绍部署以及基本使用 理解不足小伙伴帮忙指正 不必太纠结于当下，也不必太忧虑未来，当你经历过一些事情的时候，眼前的风景已经和从前不一样了。…...
ModbusTcp协议
Modbus TCP是一种通信协议，用于工业设备之间的通信。它是Modbus协议家族中的一个成员，最初是为串行通信设计的，但后来扩展到了TCP/IP网络。Modbus TCP/IP是一种公开的标准，由Modbus组织制定，并且被广泛应用于工业自动化…...
常用工具——Gradle
前言 实践是最好的学习方式，技术也如此。 文章目录 前言一、Gradle 简介二、文件结构详解 一、Gradle 简介 Gradle 文件是一个独立于 android 之外的一个东西； 是什么 gradle 就是编译、打包 Android 工程的一个构建工具；build.gradle 文件&…...
OpenHarmony教程指南—Navigation开发 页面切换场景范例
简介 在应用开发时，我们常常遇到，需要在应用内多页面跳转场景时中使用Navigation导航组件做统一的页面跳转管理，它提供了一系列属性方法来设置页面的标题栏、工具栏以及菜单栏的各种展示样式。除此之外还拥有动态加载，navPathSta…...
2024-简单点-picamera2除了文档还有哪里可以学习实例？
picamera2学习例子 去github的picamera2库，找app和examples目录，然后学习...
JavaScript实现点击鼠标弹钢琴的效果
思路： 图片设置宽900px，找到鼠标按下时的x坐标和img距离body的x坐标，两个值相减，然后除100取整，赋值给a，通过判断a的值来确定放出那个音乐。 完整代码： <!DOCTYPE html> <html lan…...
docker-compose Install rustdesk
RustDesk RustDesk 是一款开源的远程支持和远程桌面工具，它旨在为用户提供便捷的远程协助和远程访问功能。 默认情况下，hbbs 监听21115(tcp), 21116(tcp/udp), 21118(tcp)，hbbr 监听21117(tcp), 21119(tcp)。务必在防火墙开启这几个端口， 请注意21116同时要开启TCP和UDP。…...
初学C++
注释 变量 作用：给一段指定的内存空间起名，方便操作这段内容 数据类型 变量名 变量初始值; 常量 用于记录程序中不可更改的数据 宏常量： #define 宏常量 常量值 const修饰的变量： const 数据类型 常量名 常量值; 关键字 …...
数据分析-Pandas数据y轴双坐标设置
数据分析-Pandas数据y轴双坐标设置 数据分析和处理中，难免会遇到各种数据，那么数据呈现怎样的规律呢？不管金融数据，风控数据，营销数据等等，莫不如此。如何通过图示展示数据的规律？ 数据表&…...
Android多线程实现方式及并发与同步，Android面试题汇总
一. 开发背景 想要成为一名优秀的Android开发，你需要一份完备的知识体系，在这里，让我们一起成长为自己所想的那样。 我们的项目需要开发一款智能硬件。它由 Web 后台发送指令到一款桌面端应用程序，再由桌面程序来控制不同的硬件设…...
2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中，敬请期待…
2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中，敬请期待… 武汉唯众智创科技有限公司 2024 年 2 月 联系人：辜渝傧13037102709 题号：试题01 模块三：业务分析与可视化 （一࿰…...
设计MySQL数据表的几个注意点
最近合作搞项目，发现了很多问题。特别的，数据库层面上的问题更为致命。记录一下，希望后面看到博客的同学们注意。 注意：以下观点只用于一般情况下的单体、微服务，不保证适用所有场景。 一、ID问题 ID名称问题 如下图…...
android 键盘遮挡输入框问题回忆
背景 刚开始做Android的时候，有一次遇到输入框位于页面底部，弹出的键盘老是遮挡输入框，这就给人一种感觉----不咋舒服。当时，网上百度了一遍，后面终于解决了，由于当时天天加班，没时间写博客&…...
ZJGSU 1737 链表
题目描述 请根据输入数据构造一个带头结点的单链表，链表结点的数据结构为struct node {int data; struct node *next;}，试设计算法：按递增次序输出单链表中各结点的数据元素，并释放结点所占用的存储空间。 要求：不允…...
Java开发人员不得不收集的代码，java软件开发面试常见问题
前言 今年的金三银四已经过去一大半了，在这其中参与过不少面试，2021都说工作不好找，这也是对开发人员的要求变向的提高了。 之前在Github上收获15Kstar的Java核心神技（这参数，质量多高就不用我多说了吧）非…...
浅谈块存储、文件存储、对象存储
**块存储、文件存储和对象存储各自有其独特的特点和适用场景**。具体来说： 1. **块存储**： - 描述：块存储将存储空间分割成固定大小的块，这些块可以直接映射到主机操作系统。它提供的是原始的存储空间，不带文件系统…...
2024年 Python面试热点
当然，以下是一些Python相关的热点面试题： 请解释一下Python中的装饰器（Decorators）是什么，以及它们的主要用途是什么？ 装饰器是Python中的一个高级功能，它允许用户修改或增强函数、方法或类的行…...
Map集合体系——遍历，HashMap，TreeMap，LikedHashMap
认识Map集合 Map集合体系特点 方法 代码示例 package com.zz.Map;import java.util.*;public class Test {public static void main(String args[]){Map<String, Integer> map new HashMap <>();//经典代码，按照键 无序 不重复 无索引map.put("…...
docker mysql主从复制
新建主服务器容器实例3301 mysql 主 3301 docker run -p 3301:3306 --name mysql-master \ -v /mydata/mysql-master/log:/var/log/mysql \ -v /mydata/mysql-master/data:/var/lib/mysql \ -v /mydata/mysql-master/conf:/etc/mysql \ -v /home/mysql/mysql-files:/var/lib/…...
iOS 自动化测试踩坑(一): 技术方案、环境配置与落地实践
移动端的自动化测试，最常见的是 Android 自动化测试，我个人觉得 Android 的测试优先级会更高，也更开放，更容易测试；而 iOS 相较于 Android 要安全稳定的多，但也是一个必须测试的方向，这个系列文…...
Redis的事务
在 Redis 中，事务（Transaction）是一组命令的集合，可以作为一个单独的操作来执行，保证这组命令要么全部执行成功，要么全部执行失败，具有原子性。在 Redis 中，事务是通过 MULTI、EXEC、…...
4.2 比多数opencv函数效果更好的二值化（python)
在这里之间写代码： import numpy as np import torch import torch.nn as nn import cv2#1.silu激活函数 class SiLU(nn.Module):staticmethoddef forward(x):return x*torch.sigmoid(x)#2.获得轨道的类 def railway_classes3(img,x1,x2,y1,y2):img2 img[x1:x2, y…...
webpack打包一个文件，做了哪些事情
用webpack打包一个文件，在webpack内部做了哪些事情，用代码详细介绍一下 当你使用 Webpack 打包一个文件时，Webpack 内部会进行一系列操作来实现模块加载、代码转换、依赖分析、模块打包等功能。以下是使用 Webpack 打包一个简单 JavaScript …...
设计模式学习笔记 - 设计原则 - 6.KISS原则和YAGNI原则
前言 今天，将两个设计原则：KISS 原则和 YANGI 原则。其中，KISS 原则比较经典，耳熟能详，但 YANGI 你可能没怎么听过，不过它理解起来也不难。 理解这个两个原则的时候，经常会有一个共同的问题&a…...
【Vue3-vite】动态导入路由
route文件结构 router moduleindex.ts 路由定义 // 需要导入的路由如下： const routes [{path: /manage,name: manage,component: () > import(/views/home/index.vue),children: manageRoutes,}]index.ts实现从module中自动导入 // 动态导入 const routeFil…...
C++——string类
前言：哈喽小伙伴们，从这篇文章开始我们将进行若干个C中的重要的类容器的学习。本篇文章将讲解第一个类容器——string。 目录 一.什么是string类 二.string类常见接口 1.string类对象的常见构造 2.string类对象的容量操作 3. string类对象的访问及遍…...
进制转换md5绕过 [安洵杯 2019]easy_web1
打开题目 在查看url的时候得到了一串类似编码的东西，源码那里也是一堆base64，但是转换成图片就是网页上我们看见的那个表情包 ?imgTXpVek5UTTFNbVUzTURabE5qYz0&cmd 我们可以先试把前面的img那串解码了 解码的时候发现长度不够，那我们…...
.kat6.l6st6r勒索病毒的最新威胁：如何恢复您的数据？
导言： 在当今数字化时代，数据安全变得至关重要。然而，随着网络威胁不断增加，勒索病毒已成为企业和个人面临的严重威胁之一。其中，.kat6.l6st6r勒索病毒是最新的变种之一，它能够加密您的数据文件࿰…...
Day 6.有名信号量（信号灯）、网络的相关概念和发端
有名信号量 1.创建： semget int semget(key_t key, int nsems, int semflg); 功能：创建一组信号量 参数：key：IPC对像的名字 nsems：信号量的数量 semflg：IPC_CREAT 返回值：成功返回信号量ID…...
MySQL 常用优化方式
MySQL 常用优化方式 sql 书写顺序与执行顺序SQL设计优化使用索引避免索引失效分析慢查询合理使用子查询和临时表列相关使用 日常SQL优化场景limit语句隐式类型转换嵌套子查询混合排序查询重写 sql 书写顺序与执行顺序 (7) SELECT (8) DISTINCT <select_list> (1) FROM &…...