当前位置： 首页 > news >正文

K8s 镜像缓存管理 kube-fledged 认知

news 2026/1/1 3:05:14

写在前面

  • 博文内容为K8s 镜像缓存管理 kube-fledged 认知
  • 内容涉及：
    • kube-fledged 简单介绍
    • 部署以及基本使用
  • 理解不足小伙伴帮忙指正

不必太纠结于当下，也不必太忧虑未来，当你经历过一些事情的时候，眼前的风景已经和从前不一样了。——村上春树

简单介绍

我们知道 k8s 上的容器调度需要在调度的节点行拉取当前容器的镜像，在一些特殊场景中，

  • 需要快速启动和/或扩展的应用程序。例如，由于数据量激增，执行实时数据处理的应用程序需要快速扩展。
  • 镜像比较庞大，涉及多个版本，节点存储有限，需要动态清理不需要的镜像
  • 无服务器函数通常需要在几分之一秒内立即对传入事件和启动容器做出反应。
  • 在边缘设备上运行的 IoT 应用程序，需要容忍边缘设备和镜像镜像仓库之间的间歇性网络连接。
  • 如果需要从专用仓库中拉取镜像，并且无法授予每个人从此镜像仓库拉取镜像的访问权限，则可以在群集的节点上提供镜像。
  • 如果集群管理员或操作员需要对应用程序进行升级，并希望事先验证是否可以成功拉取新镜像。

kube-fledged 是一个 kubernetes operator，用于直接在 Kubernetes 集群的 worker 节点上创建和管理容器镜像缓存。它允许用户定义镜像列表以及这些镜像应缓存到哪些工作节点上（即拉取）。因此，应用程序 Pod 几乎可以立即启动，因为不需要从镜像仓库中提取镜像。

kube-fledged 提供了 CRUD API 来管理镜像缓存的生命周期，并支持多个可配置的参数，可以根据自己的需要自定义功能。

Kubernetes 具有内置的镜像垃圾回收机制。节点中的 kubelet 会定期检查磁盘使用率是否达到特定阈值（可通过标志进行配置）。一旦达到这个阈值，kubelet 会自动删除节点中所有未使用的镜像。

需要在建议的解决方案中实现自动和定期刷新机制。如果镜像缓存中的镜像被 kubelet 的 gc 删除，下一个刷新周期会将已删除的镜像拉入镜像缓存中。这可确保镜像缓存是最新的。

设计流程

https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png

部署 kube-fledged

Helm 方式部署

──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$mkdir  kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$cd kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$export KUBEFLEDGED_NAMESPACE=kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE}
namespace/kube-fledged created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/
"kubefledged-charts" has been added to your repositories
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubefledged-charts" chart repository
...Successfully got an update from the "kubescape" chart repository
...Successfully got an update from the "rancher-stable" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "openkruise" chart repository
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "botkube" chart repository
Update Complete. ⎈Happy Helming!⎈

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait

实际部署中发现，由于网络问题，chart 无法下载，所以通过 make deploy-using-yaml 使用 yaml 方式部署

Yaml 文件部署

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$git clone https://github.com/senthilrch/kube-fledged.git
正克隆到 'kube-fledged'...
remote: Enumerating objects: 10613, done.
remote: Counting objects: 100% (1501/1501), done.
remote: Compressing objects: 100% (629/629), done.
remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112
接收对象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done.
处理 delta 中: 100% (4431/4431), done.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$ls
kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$cd kube-fledged/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml

第一次部署，发现镜像拉不下来

┌──[root@vms100.liruilongs.github.io]-[~]
└─$kubectl get all -n kube-fledged
NAME                                               READY   STATUS                  RESTARTS         AGE
pod/kube-fledged-controller-df69f6565-drrqg        0/1     CrashLoopBackOff        35 (5h59m ago)   21h
pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2   0/1     Init:CrashLoopBackOff   35 (5h58m ago)   21h
pod/kubefledged-controller-55f848cc67-7f4rl        1/1     Running                 0                21h
pod/kubefledged-webhook-server-597dbf4ff5-l8fbh    0/1     Init:CrashLoopBackOff   34 (6h ago)      21hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   21h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   21hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       0/1     1            0           21h
deployment.apps/kube-fledged-webhook-server   0/1     1            0           21h
deployment.apps/kubefledged-controller        0/1     1            0           21h
deployment.apps/kubefledged-webhook-server    0/1     1            0           21hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         0       21h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         0       21h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         0       21h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         0       21h
┌──[root@vms100.liruilongs.github.io]-[~]
└─$

这里我们找一下要拉取的镜像

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat *.yaml | grep image:- image: senthilrch/kubefledged-controller:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0

单独拉取一些，当前使用 ansible 在所有工作节点批量操作

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml

其他相关的镜像都拉取一下

操作完成之后容器状态全部正常

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl -n kube-fledged get all
NAME                                               READY   STATUS    RESTARTS   AGE
pod/kube-fledged-controller-df69f6565-wdb4g        1/1     Running   0          13h
pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp   1/1     Running   0          13h
pod/kubefledged-controller-55f848cc67-klxlm        1/1     Running   0          13h
pod/kubefledged-webhook-server-597dbf4ff5-ktbsh    1/1     Running   0          13hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   36h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   36hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       1/1     1            1           36h
deployment.apps/kube-fledged-webhook-server   1/1     1            1           36h
deployment.apps/kubefledged-controller        1/1     1            1           36h
deployment.apps/kubefledged-webhook-server    1/1     1            1           36hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         1       36h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         1       36h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         1       36h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         1       36h

验证是否安装成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get pods -n kube-fledged -l app=kubefledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-klxlm       1/1     Running   0          16h
kubefledged-webhook-server-597dbf4ff5-ktbsh   1/1     Running   0          16h
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.

使用 kubefledged

创建镜像缓存对象

根据 Demo 文件，创建镜像缓存对象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$cd deploy/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- ghcr.io/jitesoft/nginx:1.23.1# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- us.gcr.io/k8s-artifacts-prod/cassandra:v7- us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0nodeSelector:tier: backend# Specifies a list of image pull secrets to pull images from private repositories into the cacheimagePullSecrets:- name: myregistrykey

官方的 Demo 中对应的 镜像拉取不下来，所以换一下

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7
Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

为了测试选择器标签的使用,我们找一个节点的标签单独做镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get nodes  --show-labels

同时我们直接从公有仓库拉取镜像，所以不需要 imagePullSecrets 对象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml

修改后的 yaml 文件

  • 添加了一个所有节点的 liruilong/my-busybox:latest 镜像缓存
  • 添加了一个 kubernetes.io/hostname: vms105.liruilongs.github.io 对应标签选择器的 liruilong/hikvision-sdk-config-ftp:latest 镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

直接创建报错了

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io")
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

解决办法，删除对应的对象，重新创建

我在当前项目的一个 issues 下面找到了解决办法 https://github.com/senthilrch/kube-fledged/issues/76

看起来这是因为 Webhook CA 是硬编码的，但是当 webhook 服务器启动时，会生成一个新的 CA 捆绑包并更新 webhook 配置。当发生另一个部署时，将重新应用原始 CA 捆绑包，并且 Webhook 请求开始失败，直到再次重新启动 Webhook 组件以修补捆绑包init-server

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make remove-kubefledged-and-operator
# Remove kubefledged
kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml
error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2"
ensure CRDs are installed first

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
namespace/kube-fledged created
kubectl apply -f deploy/kubefledged-crd.yaml
customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged
....................
kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch
Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available...
deployment "kubefledged-webhook-server" successfully rolled out
kubectl get pods -n kube-fledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-76c4v       1/1     Running   0          112s
kubefledged-webhook-server-597dbf4ff5-56h6z   1/1     Running   0          66s

重新创建缓存对象，创建成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
imagecache.kubefledged.io/imagecache1 created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
NAME          AGE
imagecache1   10s
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

查看当前被纳管的镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl get imagecaches imagecache1 -n kube-fledged -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 83,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20169836","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:06:47Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheRefresh","startTime": "2024-03-02T01:05:33Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$

通过 ansible 来验证

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | CHANGED | rc=0 >>
liruilong/hikvision-sdk-config-ftp                                          latest            a02cd03b4342   4 months ago    830MB
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

开启自动刷新

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=
imagecache.kubefledged.io/imagecache1 annotated
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

添加镜像缓存

添加一个新的镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 92,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175233","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest","liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:43:32Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:40:34Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

通过 ansible 确认

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.102 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest                                      17dbd4002a8c   5 years ago     170MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

删除镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 94,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175766","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"message": "Image cache is being updated. Please view the status after some time","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:48:03Z","status": "Processing"}
}

通过 Ansible 确认，可以看到无论是 mastere 上的节点还是 work 的节点，对应的镜像缓存都被清理

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

这里需要注意如果清除所有的镜像缓存，那么需要把 images 下的数组 写成 “”.

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 98,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20176849","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": [""]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:52:16Z","message": "All cached images succesfully deleted from respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:51:47Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

如果通过下面的方式删除，直接注释调对应的标签

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster#- images:#- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

那么会报下面的错

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists
You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.

博文部分内容参考

© 文中涉及参考链接内容版权归原作者所有，如有侵权请告知,如果你认可它不要吝啬星星哦 😃

https://github.com/senthilrch/kube-fledged

© 2018-2024 liruilonger@gmail.com, All rights reserved. 保持署名-非商用-相同方式共享(CC BY-NC-SA 4.0)

相关文章：

K8s 镜像缓存管理 kube-fledged 认知

写在前面 博文内容为K8s 镜像缓存管理 kube-fledged 认知内容涉及&#xff1a; kube-fledged 简单介绍部署以及基本使用 理解不足小伙伴帮忙指正 不必太纠结于当下&#xff0c;也不必太忧虑未来&#xff0c;当你经历过一些事情的时候&#xff0c;眼前的风景已经和从前不一样了。…...

编程日记 2024/3/7 5:52:45

ModbusTcp协议

Modbus TCP是一种通信协议&#xff0c;用于工业设备之间的通信。它是Modbus协议家族中的一个成员&#xff0c;最初是为串行通信设计的&#xff0c;但后来扩展到了TCP/IP网络。Modbus TCP/IP是一种公开的标准&#xff0c;由Modbus组织制定&#xff0c;并且被广泛应用于工业自动化…...

编程日记 2024/3/7 5:48:42

常用工具——Gradle

前言 实践是最好的学习方式&#xff0c;技术也如此。 文章目录 前言一、Gradle 简介二、文件结构详解 一、Gradle 简介 Gradle 文件是一个独立于 android 之外的一个东西&#xff1b; 是什么 gradle 就是编译、打包 Android 工程的一个构建工具&#xff1b;build.gradle 文件&…...

编程日记 2024/3/7 5:47:41

OpenHarmony教程指南—Navigation开发 页面切换场景范例

简介 在应用开发时&#xff0c;我们常常遇到&#xff0c;需要在应用内多页面跳转场景时中使用Navigation导航组件做统一的页面跳转管理&#xff0c;它提供了一系列属性方法来设置页面的标题栏、工具栏以及菜单栏的各种展示样式。除此之外还拥有动态加载&#xff0c;navPathSta…...

编程日记 2024/3/7 5:46:40

2024-简单点-picamera2除了文档还有哪里可以学习实例？

picamera2学习例子 去github的picamera2库&#xff0c;找app和examples目录&#xff0c;然后学习...

编程日记 2024/3/7 5:42:36

JavaScript实现点击鼠标弹钢琴的效果

思路&#xff1a; 图片设置宽900px&#xff0c;找到鼠标按下时的x坐标和img距离body的x坐标&#xff0c;两个值相减&#xff0c;然后除100取整&#xff0c;赋值给a&#xff0c;通过判断a的值来确定放出那个音乐。 完整代码&#xff1a; <!DOCTYPE html> <html lan…...

编程日记 2024/3/7 5:40:34

docker-compose Install rustdesk

RustDesk RustDesk 是一款开源的远程支持和远程桌面工具，它旨在为用户提供便捷的远程协助和远程访问功能。 默认情况下，hbbs 监听21115(tcp), 21116(tcp/udp), 21118(tcp)，hbbr 监听21117(tcp), 21119(tcp)。务必在防火墙开启这几个端口， 请注意21116同时要开启TCP和UDP。…...

编程日记 2024/3/7 5:38:32

初学C++

注释 变量 作用&#xff1a;给一段指定的内存空间起名&#xff0c;方便操作这段内容 数据类型 变量名 变量初始值; 常量 用于记录程序中不可更改的数据 宏常量&#xff1a; #define 宏常量 常量值 const修饰的变量&#xff1a; const 数据类型 常量名 常量值; 关键字 …...

编程日记 2024/3/7 5:37:31

数据分析-Pandas数据y轴双坐标设置

数据分析-Pandas数据y轴双坐标设置 数据分析和处理中&#xff0c;难免会遇到各种数据&#xff0c;那么数据呈现怎样的规律呢&#xff1f;不管金融数据&#xff0c;风控数据&#xff0c;营销数据等等&#xff0c;莫不如此。如何通过图示展示数据的规律&#xff1f; 数据表&…...

编程日记 2024/3/7 5:34:27

Android多线程实现方式及并发与同步，Android面试题汇总

一. 开发背景 想要成为一名优秀的Android开发&#xff0c;你需要一份完备的知识体系&#xff0c;在这里&#xff0c;让我们一起成长为自己所想的那样。 我们的项目需要开发一款智能硬件。它由 Web 后台发送指令到一款桌面端应用程序&#xff0c;再由桌面程序来控制不同的硬件设…...

编程日记 2024/3/7 5:33:26

2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中，敬请期待…

2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中&#xff0c;敬请期待… 武汉唯众智创科技有限公司 2024 年 2 月 联系人&#xff1a;辜渝傧13037102709 题号&#xff1a;试题01 模块三&#xff1a;业务分析与可视化 &#xff08;一&#xff0…...

编程日记 2024/3/7 5:31:22

设计MySQL数据表的几个注意点

最近合作搞项目&#xff0c;发现了很多问题。特别的&#xff0c;数据库层面上的问题更为致命。记录一下&#xff0c;希望后面看到博客的同学们注意。 注意&#xff1a;以下观点只用于一般情况下的单体、微服务&#xff0c;不保证适用所有场景。 一、ID问题 ID名称问题 如下图…...

编程日记 2024/3/7 5:30:22

android 键盘遮挡输入框问题回忆

背景 刚开始做Android的时候&#xff0c;有一次遇到输入框位于页面底部&#xff0c;弹出的键盘老是遮挡输入框&#xff0c;这就给人一种感觉----不咋舒服。当时&#xff0c;网上百度了一遍&#xff0c;后面终于解决了&#xff0c;由于当时天天加班&#xff0c;没时间写博客&…...

编程日记 2024/3/7 5:29:21

ZJGSU 1737 链表

题目描述 请根据输入数据构造一个带头结点的单链表&#xff0c;链表结点的数据结构为struct node {int data; struct node *next;}&#xff0c;试设计算法&#xff1a;按递增次序输出单链表中各结点的数据元素&#xff0c;并释放结点所占用的存储空间。 要求&#xff1a;不允…...

编程日记 2024/3/7 5:26:18

Java开发人员不得不收集的代码，java软件开发面试常见问题

前言 今年的金三银四已经过去一大半了&#xff0c;在这其中参与过不少面试&#xff0c;2021都说工作不好找&#xff0c;这也是对开发人员的要求变向的提高了。 之前在Github上收获15Kstar的Java核心神技&#xff08;这参数&#xff0c;质量多高就不用我多说了吧&#xff09;非…...

编程日记 2024/3/7 5:25:17

浅谈块存储、文件存储、对象存储

**块存储、文件存储和对象存储各自有其独特的特点和适用场景**。具体来说&#xff1a; 1. **块存储**&#xff1a; - 描述&#xff1a;块存储将存储空间分割成固定大小的块&#xff0c;这些块可以直接映射到主机操作系统。它提供的是原始的存储空间&#xff0c;不带文件系统…...

编程日记 2024/3/7 5:22:14

2024年 Python面试热点

当然&#xff0c;以下是一些Python相关的热点面试题&#xff1a; 请解释一下Python中的装饰器&#xff08;Decorators&#xff09;是什么&#xff0c;以及它们的主要用途是什么&#xff1f; 装饰器是Python中的一个高级功能&#xff0c;它允许用户修改或增强函数、方法或类的行…...

编程日记 2024/3/7 5:18:09

Map集合体系——遍历，HashMap，TreeMap，LikedHashMap

认识Map集合 Map集合体系特点 方法 代码示例 package com.zz.Map;import java.util.*;public class Test {public static void main(String args[]){Map<String, Integer> map new HashMap <>();//经典代码&#xff0c;按照键 无序 不重复 无索引map.put("…...

编程日记 2024/3/7 5:13:05

docker mysql主从复制

新建主服务器容器实例3301 mysql 主 3301 docker run -p 3301:3306 --name mysql-master \ -v /mydata/mysql-master/log:/var/log/mysql \ -v /mydata/mysql-master/data:/var/lib/mysql \ -v /mydata/mysql-master/conf:/etc/mysql \ -v /home/mysql/mysql-files:/var/lib/…...

编程日记 2024/3/7 5:12:04

iOS 自动化测试踩坑(一): 技术方案、环境配置与落地实践

移动端的自动化测试&#xff0c;最常见的是 Android 自动化测试&#xff0c;我个人觉得 Android 的测试优先级会更高&#xff0c;也更开放&#xff0c;更容易测试&#xff1b;而 iOS 相较于 Android 要安全稳定的多&#xff0c;但也是一个必须测试的方向&#xff0c;这个系列文…...

编程日记 2024/3/7 5:11:03

java 实现excel文件转pdf | 无水印 | 无限制

文章目录 目录 文章目录 前言 1.项目远程仓库配置 2.pom文件引入相关依赖 3.代码破解 二、Excel转PDF 1.代码实现 2.Aspose.License.xml 授权文件 总结 前言 java处理excel转pdf一直没找到什么好用的免费jar包工具，自己手写的难度，恐怕高级程序员花费一年的事件，也…...

编程新知 2025/11/5 4:10:42

STM32F4基本定时器使用和原理详解

STM32F4基本定时器使用和原理详解 前言如何确定定时器挂载在哪条时钟线上配置及使用方法参数配置PrescalerCounter ModeCounter Periodauto-reload preloadTrigger Event Selection 中断配置生成的代码及使用方法初始化代码基本定时器触发DCA或者ADC的代码讲解中断代码定时启动…...

编程新知 2025/12/21 1:00:15

推荐 github 项目:GeminiImageApp(图片生成方向，可以做一定的素材)

推荐 github 项目:GeminiImageApp(图片生成方向&#xff0c;可以做一定的素材) 这个项目能干嘛? 使用 gemini 2.0 的 api 和 google 其他的 api 来做衍生处理 简化和优化了文生图和图生图的行为(我的最主要) 并且有一些目标检测和切割(我用不到) 视频和 imagefx 因为没 a…...

编程新知 2025/12/31 17:35:03

七、数据库的完整性

七、数据库的完整性 主要内容 7.1 数据库的完整性概述 7.2 实体完整性 7.3 参照完整性 7.4 用户定义的完整性 7.5 触发器 7.6 SQL Server中数据库完整性的实现 7.7 小结 7.1 数据库的完整性概述 数据库完整性的含义 正确性 指数据的合法性 有效性 指数据是否属于所定…...

编程新知 2025/12/31 5:09:49

FFmpeg：Windows系统小白安装及其使用

一、安装 1.访问官网 Download FFmpeg 2.点击版本目录 3.选择版本点击安装 注意这里选择的是【release buids】&#xff0c;注意左上角标题 例如我安装在目录 F:\FFmpeg 4.解压 5.添加环境变量 把你解压后的bin目录&#xff08;即exe所在文件夹&#xff09;加入系统变量…...

编程新知 2026/1/1 0:15:00

群晖NAS如何在虚拟机创建飞牛NAS

套件中心下载安装Virtual Machine Manager 创建虚拟机 配置虚拟机 飞牛官网下载 https://iso.liveupdate.fnnas.com/x86_64/trim/fnos-0.9.2-863.iso 群晖NAS如何在虚拟机创建飞牛NAS - 个人信息分享...

编程新知 2026/1/1 0:27:43

HTML前端开发：JavaScript 获取元素方法详解

作为前端开发者&#xff0c;高效获取 DOM 元素是必备技能。以下是 JS 中核心的获取元素方法&#xff0c;分为两大系列&#xff1a; 一、getElementBy... 系列 传统方法&#xff0c;直接通过 DOM 接口访问&#xff0c;返回动态集合&#xff08;元素变化会实时更新&#xff09;。…...

编程新知 2025/10/6 18:44:24

基于江科大stm32屏幕驱动，实现OLED多级菜单（动画效果），结构体链表实现(独创源码)

引言 在嵌入式系统中&#xff0c;用户界面的设计往往直接影响到用户体验。本文将以STM32微控制器和OLED显示屏为例&#xff0c;介绍如何实现一个多级菜单系统。该系统支持用户通过按键导航菜单&#xff0c;执行相应操作&#xff0c;并提供平滑的滚动动画效果。 本文设计了一个…...

编程新知 2026/1/1 1:02:27

qt+vs Generated File下的moc_和ui_文件丢失导致 error LNK2001

qt 5.9.7 vs2013 qt add-in 2.3.2 起因是添加一个新的控件类&#xff0c;直接把源文件拖进VS的项目里&#xff0c;然后VS卡住十秒&#xff0c;然后编译就报一堆 error LNK2001 一看项目的Generated Files下的moc_和ui_文件丢失了一部分&#xff0c;导致编译的时候找不到了。因…...

编程新知 2025/12/25 16:21:58

【PX4飞控】mavros gps相关话题分析，经纬度海拔获取方法，卫星数锁定状态获取方法

使用 ROS1-Noetic 和 mavros v1.20.1&#xff0c; 携带经纬度海拔的话题主要有三个&#xff1a; /mavros/global_position/raw/fix/mavros/gpsstatus/gps1/raw/mavros/global_position/global 查看 mavros 源码&#xff0c;来分析他们的发布过程。发现前两个话题都对应了同一…...

编程新知 2025/12/25 3:41:44