当前位置： 首页 > news >正文

K8s 镜像缓存管理 kube-fledged 认知

news 2025/9/19 12:43:59

写在前面

  • 博文内容为K8s 镜像缓存管理 kube-fledged 认知
  • 内容涉及：
    • kube-fledged 简单介绍
    • 部署以及基本使用
  • 理解不足小伙伴帮忙指正

不必太纠结于当下，也不必太忧虑未来，当你经历过一些事情的时候，眼前的风景已经和从前不一样了。——村上春树

简单介绍

我们知道 k8s 上的容器调度需要在调度的节点行拉取当前容器的镜像，在一些特殊场景中，

  • 需要快速启动和/或扩展的应用程序。例如，由于数据量激增，执行实时数据处理的应用程序需要快速扩展。
  • 镜像比较庞大，涉及多个版本，节点存储有限，需要动态清理不需要的镜像
  • 无服务器函数通常需要在几分之一秒内立即对传入事件和启动容器做出反应。
  • 在边缘设备上运行的 IoT 应用程序，需要容忍边缘设备和镜像镜像仓库之间的间歇性网络连接。
  • 如果需要从专用仓库中拉取镜像，并且无法授予每个人从此镜像仓库拉取镜像的访问权限，则可以在群集的节点上提供镜像。
  • 如果集群管理员或操作员需要对应用程序进行升级，并希望事先验证是否可以成功拉取新镜像。

kube-fledged 是一个 kubernetes operator，用于直接在 Kubernetes 集群的 worker 节点上创建和管理容器镜像缓存。它允许用户定义镜像列表以及这些镜像应缓存到哪些工作节点上（即拉取）。因此，应用程序 Pod 几乎可以立即启动，因为不需要从镜像仓库中提取镜像。

kube-fledged 提供了 CRUD API 来管理镜像缓存的生命周期，并支持多个可配置的参数，可以根据自己的需要自定义功能。

Kubernetes 具有内置的镜像垃圾回收机制。节点中的 kubelet 会定期检查磁盘使用率是否达到特定阈值（可通过标志进行配置）。一旦达到这个阈值，kubelet 会自动删除节点中所有未使用的镜像。

需要在建议的解决方案中实现自动和定期刷新机制。如果镜像缓存中的镜像被 kubelet 的 gc 删除，下一个刷新周期会将已删除的镜像拉入镜像缓存中。这可确保镜像缓存是最新的。

设计流程

https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png

部署 kube-fledged

Helm 方式部署

──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$mkdir  kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$cd kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$export KUBEFLEDGED_NAMESPACE=kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE}
namespace/kube-fledged created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/
"kubefledged-charts" has been added to your repositories
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "kubefledged-charts" chart repository
...Successfully got an update from the "kubescape" chart repository
...Successfully got an update from the "rancher-stable" chart repository
...Successfully got an update from the "skm" chart repository
...Successfully got an update from the "openkruise" chart repository
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "botkube" chart repository
Update Complete. ⎈Happy Helming!⎈

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait

实际部署中发现，由于网络问题，chart 无法下载，所以通过 make deploy-using-yaml 使用 yaml 方式部署

Yaml 文件部署

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$git clone https://github.com/senthilrch/kube-fledged.git
正克隆到 'kube-fledged'...
remote: Enumerating objects: 10613, done.
remote: Counting objects: 100% (1501/1501), done.
remote: Compressing objects: 100% (629/629), done.
remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112
接收对象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done.
处理 delta 中: 100% (4431/4431), done.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$ls
kube-fledged
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$cd kube-fledged/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml

第一次部署，发现镜像拉不下来

┌──[root@vms100.liruilongs.github.io]-[~]
└─$kubectl get all -n kube-fledged
NAME                                               READY   STATUS                  RESTARTS         AGE
pod/kube-fledged-controller-df69f6565-drrqg        0/1     CrashLoopBackOff        35 (5h59m ago)   21h
pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2   0/1     Init:CrashLoopBackOff   35 (5h58m ago)   21h
pod/kubefledged-controller-55f848cc67-7f4rl        1/1     Running                 0                21h
pod/kubefledged-webhook-server-597dbf4ff5-l8fbh    0/1     Init:CrashLoopBackOff   34 (6h ago)      21hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   21h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   21hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       0/1     1            0           21h
deployment.apps/kube-fledged-webhook-server   0/1     1            0           21h
deployment.apps/kubefledged-controller        0/1     1            0           21h
deployment.apps/kubefledged-webhook-server    0/1     1            0           21hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         0       21h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         0       21h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         0       21h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         0       21h
┌──[root@vms100.liruilongs.github.io]-[~]
└─$

这里我们找一下要拉取的镜像

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat *.yaml | grep image:- image: senthilrch/kubefledged-controller:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0- image: senthilrch/kubefledged-webhook-server:v0.10.0

单独拉取一些，当前使用 ansible 在所有工作节点批量操作

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml

其他相关的镜像都拉取一下

操作完成之后容器状态全部正常

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl -n kube-fledged get all
NAME                                               READY   STATUS    RESTARTS   AGE
pod/kube-fledged-controller-df69f6565-wdb4g        1/1     Running   0          13h
pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp   1/1     Running   0          13h
pod/kubefledged-controller-55f848cc67-klxlm        1/1     Running   0          13h
pod/kubefledged-webhook-server-597dbf4ff5-ktbsh    1/1     Running   0          13hNAME                                  TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/kube-fledged-webhook-server   ClusterIP   10.100.194.199   <none>        3443/TCP   36h
service/kubefledged-webhook-server    ClusterIP   10.101.191.206   <none>        3443/TCP   36hNAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-fledged-controller       1/1     1            1           36h
deployment.apps/kube-fledged-webhook-server   1/1     1            1           36h
deployment.apps/kubefledged-controller        1/1     1            1           36h
deployment.apps/kubefledged-webhook-server    1/1     1            1           36hNAME                                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-fledged-controller-df69f6565        1         1         1       36h
replicaset.apps/kube-fledged-webhook-server-7bcd589bc4   1         1         1       36h
replicaset.apps/kubefledged-controller-55f848cc67        1         1         1       36h
replicaset.apps/kubefledged-webhook-server-597dbf4ff5    1         1         1       36h

验证是否安装成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get pods -n kube-fledged -l app=kubefledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-klxlm       1/1     Running   0          16h
kubefledged-webhook-server-597dbf4ff5-ktbsh   1/1     Running   0          16h
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.

使用 kubefledged

创建镜像缓存对象

根据 Demo 文件，创建镜像缓存对象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$cd deploy/
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- ghcr.io/jitesoft/nginx:1.23.1# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- us.gcr.io/k8s-artifacts-prod/cassandra:v7- us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0nodeSelector:tier: backend# Specifies a list of image pull secrets to pull images from private repositories into the cacheimagePullSecrets:- name: myregistrykey

官方的 Demo 中对应的 镜像拉取不下来，所以换一下

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7
Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

为了测试选择器标签的使用,我们找一个节点的标签单独做镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get nodes  --show-labels

同时我们直接从公有仓库拉取镜像，所以不需要 imagePullSecrets 对象

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml

修改后的 yaml 文件

  • 添加了一个所有节点的 liruilong/my-busybox:latest 镜像缓存
  • 添加了一个 kubernetes.io/hostname: vms105.liruilongs.github.io 对应标签选择器的 liruilong/hikvision-sdk-config-ftp:latest 镜像缓存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster- images:- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

直接创建报错了

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io")
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
No resources found in kube-fledged namespace.
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

解决办法，删除对应的对象，重新创建

我在当前项目的一个 issues 下面找到了解决办法 https://github.com/senthilrch/kube-fledged/issues/76

看起来这是因为 Webhook CA 是硬编码的，但是当 webhook 服务器启动时，会生成一个新的 CA 捆绑包并更新 webhook 配置。当发生另一个部署时，将重新应用原始 CA 捆绑包，并且 Webhook 请求开始失败，直到再次重新启动 Webhook 组件以修补捆绑包init-server

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make remove-kubefledged-and-operator
# Remove kubefledged
kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml
error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2"
ensure CRDs are installed first

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged]
└─$make deploy-using-yaml
kubectl apply -f deploy/kubefledged-namespace.yaml
namespace/kube-fledged created
kubectl apply -f deploy/kubefledged-crd.yaml
customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged
....................
kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch
Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available...
deployment "kubefledged-webhook-server" successfully rolled out
kubectl get pods -n kube-fledged
NAME                                          READY   STATUS    RESTARTS   AGE
kubefledged-controller-55f848cc67-76c4v       1/1     Running   0          112s
kubefledged-webhook-server-597dbf4ff5-56h6z   1/1     Running   0          66s

重新创建缓存对象，创建成功

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl create -f kubefledged-imagecache.yaml
imagecache.kubefledged.io/imagecache1 created
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl get imagecaches -n kube-fledged
NAME          AGE
imagecache1   10s
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

查看当前被纳管的镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$kubectl get imagecaches imagecache1 -n kube-fledged -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 83,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20169836","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:06:47Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheRefresh","startTime": "2024-03-02T01:05:33Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged]
└─$

通过 ansible 来验证

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | CHANGED | rc=0 >>
liruilong/hikvision-sdk-config-ftp                                          latest            a02cd03b4342   4 months ago    830MB
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

开启自动刷新

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache=
imagecache.kubefledged.io/imagecache1 annotated
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

添加镜像缓存

添加一个新的镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 92,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175233","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/my-busybox:latest","liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:43:32Z","message": "All requested images pulled succesfully to respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:40:34Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

通过 ansible 确认

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.101 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.102 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.100 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest    17dbd4002a8c   5 years ago     170MB
192.168.26.103 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest                                      17dbd4002a8c   5 years ago     170MB
192.168.26.105 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
192.168.26.106 | CHANGED | rc=0 >>
liruilong/jdk1.8_191                                                        latest            17dbd4002a8c   5 years ago     170MB
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

删除镜像缓存

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 94,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20175766","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": ["liruilong/jdk1.8_191:latest"]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"message": "Image cache is being updated. Please view the status after some time","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:48:03Z","status": "Processing"}
}

通过 Ansible 确认，可以看到无论是 mastere 上的节点还是 work 的节点，对应的镜像缓存都被清理

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.102 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.101 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | CHANGED | rc=0 >>
liruilong/my-busybox                                                        latest    497b83a63aad   11 months ago   1.24MB
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

这里需要注意如果清除所有的镜像缓存，那么需要把 images 下的数组 写成 “”.

┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
imagecache.kubefledged.io/imagecache1 edited
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml
192.168.26.102 | FAILED | rc=1 >>
non-zero return code
192.168.26.101 | FAILED | rc=1 >>
non-zero return code
192.168.26.100 | FAILED | rc=1 >>
non-zero return code
192.168.26.105 | FAILED | rc=1 >>
non-zero return code
192.168.26.103 | FAILED | rc=1 >>
non-zero return code
192.168.26.106 | FAILED | rc=1 >>
non-zero return code
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$kubectl get imagecaches.kubefledged.io  -n kube-fledged  imagecache1 -o json
{"apiVersion": "kubefledged.io/v1alpha2","kind": "ImageCache","metadata": {"creationTimestamp": "2024-03-01T15:08:42Z","generation": 98,"labels": {"app": "kubefledged","kubefledged": "imagecache"},"name": "imagecache1","namespace": "kube-fledged","resourceVersion": "20176849","uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72"},"spec": {"cacheSpec": [{"images": [""]},{"images": ["liruilong/hikvision-sdk-config-ftp:latest"],"nodeSelector": {"kubernetes.io/hostname": "vms105.liruilongs.github.io"}}]},"status": {"completionTime": "2024-03-02T01:52:16Z","message": "All cached images succesfully deleted from respective nodes","reason": "ImageCacheUpdate","startTime": "2024-03-02T01:51:47Z","status": "Succeeded"}
}
┌──[root@vms100.liruilongs.github.io]-[~/ansible]
└─$

如果通过下面的方式删除，直接注释调对应的标签

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$cat kubefledged-imagecache.yaml
---
apiVersion: kubefledged.io/v1alpha2
kind: ImageCache
metadata:# Name of the image cache. A cluster can have multiple image cache objectsname: imagecache1namespace: kube-fledged# The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preferencelabels:app: kubefledgedkubefledged: imagecache
spec:# The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled).cacheSpec:# Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster#- images:#- liruilong/my-busybox:latest# Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector- images:- liruilong/hikvision-sdk-config-ftp:latestnodeSelector:kubernetes.io/hostname: vms105.liruilongs.github.io# Specifies a list of image pull secrets to pull images from private repositories into the cache#imagePullSecrets:#- name: myregistrykey
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$

那么会报下面的错

┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$kubectl edit imagecaches imagecache1 -n kube-fledged
error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists
You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.

博文部分内容参考

© 文中涉及参考链接内容版权归原作者所有，如有侵权请告知,如果你认可它不要吝啬星星哦 😃

https://github.com/senthilrch/kube-fledged

© 2018-2024 liruilonger@gmail.com, All rights reserved. 保持署名-非商用-相同方式共享(CC BY-NC-SA 4.0)

相关文章：

K8s 镜像缓存管理 kube-fledged 认知

写在前面 博文内容为K8s 镜像缓存管理 kube-fledged 认知内容涉及&#xff1a; kube-fledged 简单介绍部署以及基本使用 理解不足小伙伴帮忙指正 不必太纠结于当下&#xff0c;也不必太忧虑未来&#xff0c;当你经历过一些事情的时候&#xff0c;眼前的风景已经和从前不一样了。…...

编程日记 2024/3/7 5:52:45

ModbusTcp协议

Modbus TCP是一种通信协议&#xff0c;用于工业设备之间的通信。它是Modbus协议家族中的一个成员&#xff0c;最初是为串行通信设计的&#xff0c;但后来扩展到了TCP/IP网络。Modbus TCP/IP是一种公开的标准&#xff0c;由Modbus组织制定&#xff0c;并且被广泛应用于工业自动化…...

编程日记 2024/3/7 5:48:42

常用工具——Gradle

前言 实践是最好的学习方式&#xff0c;技术也如此。 文章目录 前言一、Gradle 简介二、文件结构详解 一、Gradle 简介 Gradle 文件是一个独立于 android 之外的一个东西&#xff1b; 是什么 gradle 就是编译、打包 Android 工程的一个构建工具&#xff1b;build.gradle 文件&…...

编程日记 2024/3/7 5:47:41

OpenHarmony教程指南—Navigation开发 页面切换场景范例

简介 在应用开发时&#xff0c;我们常常遇到&#xff0c;需要在应用内多页面跳转场景时中使用Navigation导航组件做统一的页面跳转管理&#xff0c;它提供了一系列属性方法来设置页面的标题栏、工具栏以及菜单栏的各种展示样式。除此之外还拥有动态加载&#xff0c;navPathSta…...

编程日记 2024/3/7 5:46:40

2024-简单点-picamera2除了文档还有哪里可以学习实例？

picamera2学习例子 去github的picamera2库&#xff0c;找app和examples目录&#xff0c;然后学习...

编程日记 2024/3/7 5:42:36

JavaScript实现点击鼠标弹钢琴的效果

思路&#xff1a; 图片设置宽900px&#xff0c;找到鼠标按下时的x坐标和img距离body的x坐标&#xff0c;两个值相减&#xff0c;然后除100取整&#xff0c;赋值给a&#xff0c;通过判断a的值来确定放出那个音乐。 完整代码&#xff1a; <!DOCTYPE html> <html lan…...

编程日记 2024/3/7 5:40:34

docker-compose Install rustdesk

RustDesk RustDesk 是一款开源的远程支持和远程桌面工具，它旨在为用户提供便捷的远程协助和远程访问功能。 默认情况下，hbbs 监听21115(tcp), 21116(tcp/udp), 21118(tcp)，hbbr 监听21117(tcp), 21119(tcp)。务必在防火墙开启这几个端口， 请注意21116同时要开启TCP和UDP。…...

编程日记 2024/3/7 5:38:32

初学C++

注释 变量 作用&#xff1a;给一段指定的内存空间起名&#xff0c;方便操作这段内容 数据类型 变量名 变量初始值; 常量 用于记录程序中不可更改的数据 宏常量&#xff1a; #define 宏常量 常量值 const修饰的变量&#xff1a; const 数据类型 常量名 常量值; 关键字 …...

编程日记 2024/3/7 5:37:31

数据分析-Pandas数据y轴双坐标设置

数据分析-Pandas数据y轴双坐标设置 数据分析和处理中&#xff0c;难免会遇到各种数据&#xff0c;那么数据呈现怎样的规律呢&#xff1f;不管金融数据&#xff0c;风控数据&#xff0c;营销数据等等&#xff0c;莫不如此。如何通过图示展示数据的规律&#xff1f; 数据表&…...

编程日记 2024/3/7 5:34:27

Android多线程实现方式及并发与同步，Android面试题汇总

一. 开发背景 想要成为一名优秀的Android开发&#xff0c;你需要一份完备的知识体系&#xff0c;在这里&#xff0c;让我们一起成长为自己所想的那样。 我们的项目需要开发一款智能硬件。它由 Web 后台发送指令到一款桌面端应用程序&#xff0c;再由桌面程序来控制不同的硬件设…...

编程日记 2024/3/7 5:33:26

2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中，敬请期待…

2023年全国职业院校技能大赛中职组大数据应用与服务赛项题库参考答案陆续更新中&#xff0c;敬请期待… 武汉唯众智创科技有限公司 2024 年 2 月 联系人&#xff1a;辜渝傧13037102709 题号&#xff1a;试题01 模块三&#xff1a;业务分析与可视化 &#xff08;一&#xff0…...

编程日记 2024/3/7 5:31:22

设计MySQL数据表的几个注意点

最近合作搞项目&#xff0c;发现了很多问题。特别的&#xff0c;数据库层面上的问题更为致命。记录一下&#xff0c;希望后面看到博客的同学们注意。 注意&#xff1a;以下观点只用于一般情况下的单体、微服务&#xff0c;不保证适用所有场景。 一、ID问题 ID名称问题 如下图…...

编程日记 2024/3/7 5:30:22

android 键盘遮挡输入框问题回忆

背景 刚开始做Android的时候&#xff0c;有一次遇到输入框位于页面底部&#xff0c;弹出的键盘老是遮挡输入框&#xff0c;这就给人一种感觉----不咋舒服。当时&#xff0c;网上百度了一遍&#xff0c;后面终于解决了&#xff0c;由于当时天天加班&#xff0c;没时间写博客&…...

编程日记 2024/3/7 5:29:21

ZJGSU 1737 链表

题目描述 请根据输入数据构造一个带头结点的单链表&#xff0c;链表结点的数据结构为struct node {int data; struct node *next;}&#xff0c;试设计算法&#xff1a;按递增次序输出单链表中各结点的数据元素&#xff0c;并释放结点所占用的存储空间。 要求&#xff1a;不允…...

编程日记 2024/3/7 5:26:18

Java开发人员不得不收集的代码，java软件开发面试常见问题

前言 今年的金三银四已经过去一大半了&#xff0c;在这其中参与过不少面试&#xff0c;2021都说工作不好找&#xff0c;这也是对开发人员的要求变向的提高了。 之前在Github上收获15Kstar的Java核心神技&#xff08;这参数&#xff0c;质量多高就不用我多说了吧&#xff09;非…...

编程日记 2024/3/7 5:25:17

浅谈块存储、文件存储、对象存储

**块存储、文件存储和对象存储各自有其独特的特点和适用场景**。具体来说&#xff1a; 1. **块存储**&#xff1a; - 描述&#xff1a;块存储将存储空间分割成固定大小的块&#xff0c;这些块可以直接映射到主机操作系统。它提供的是原始的存储空间&#xff0c;不带文件系统…...

编程日记 2024/3/7 5:22:14

2024年 Python面试热点

当然&#xff0c;以下是一些Python相关的热点面试题&#xff1a; 请解释一下Python中的装饰器&#xff08;Decorators&#xff09;是什么&#xff0c;以及它们的主要用途是什么&#xff1f; 装饰器是Python中的一个高级功能&#xff0c;它允许用户修改或增强函数、方法或类的行…...

编程日记 2024/3/7 5:18:09

Map集合体系——遍历，HashMap，TreeMap，LikedHashMap

认识Map集合 Map集合体系特点 方法 代码示例 package com.zz.Map;import java.util.*;public class Test {public static void main(String args[]){Map<String, Integer> map new HashMap <>();//经典代码&#xff0c;按照键 无序 不重复 无索引map.put("…...

编程日记 2024/3/7 5:13:05

docker mysql主从复制

新建主服务器容器实例3301 mysql 主 3301 docker run -p 3301:3306 --name mysql-master \ -v /mydata/mysql-master/log:/var/log/mysql \ -v /mydata/mysql-master/data:/var/lib/mysql \ -v /mydata/mysql-master/conf:/etc/mysql \ -v /home/mysql/mysql-files:/var/lib/…...

编程日记 2024/3/7 5:12:04

iOS 自动化测试踩坑(一): 技术方案、环境配置与落地实践

移动端的自动化测试&#xff0c;最常见的是 Android 自动化测试&#xff0c;我个人觉得 Android 的测试优先级会更高&#xff0c;也更开放&#xff0c;更容易测试&#xff1b;而 iOS 相较于 Android 要安全稳定的多&#xff0c;但也是一个必须测试的方向&#xff0c;这个系列文…...

编程日记 2024/3/7 5:11:03

macOS多出来了：Google云端硬盘、YouTube、表格、幻灯片、Gmail、Google文档等应用

文章目录 问题现象问题原因解决办法 问题现象 macOS启动台&#xff08;Launchpad&#xff09;多出来了&#xff1a;Google云端硬盘、YouTube、表格、幻灯片、Gmail、Google文档等应用。 问题原因 很明显&#xff0c;都是Google家的办公全家桶。这些应用并不是通过独立安装的…...

编程新知 2025/8/18 23:22:02

SpringCloudGateway 自定义局部过滤器

场景&#xff1a; 将所有请求转化为同一路径请求&#xff08;方便穿网配置&#xff09;在请求头内标识原来路径&#xff0c;然后在将请求分发给不同服务 AllToOneGatewayFilterFactory import lombok.Getter; import lombok.Setter; import lombok.extern.slf4j.Slf4j; impor…...

编程新知 2025/9/3 11:05:35

css3笔记 （1） 自用

outline: none 用于移除元素获得焦点时默认的轮廓线 broder:0 用于移除边框 font-size&#xff1a;0 用于设置字体不显示 list-style: none 消除<li> 标签默认样式 margin: xx auto 版心居中 width:100% 通栏 vertical-align 作用于行内元素 / 表格单元格&#xff…...

编程新知 2025/9/6 3:33:57

ip子接口配置及删除

配置永久生效的子接口&#xff0c;2个IP 都可以登录你这一台服务器。重启不失效。 永久的 [应用] vi /etc/sysconfig/network-scripts/ifcfg-eth0修改文件内内容 TYPE"Ethernet" BOOTPROTO"none" NAME"eth0" DEVICE"eth0" ONBOOT&q…...

编程新知 2025/7/26 12:16:34

GitFlow 工作模式（详解）

今天再学项目的过程中遇到使用gitflow模式管理代码&#xff0c;因此进行学习并且发布关于gitflow的一些思考 Git与GitFlow模式 我们在写代码的时候通常会进行网上保存&#xff0c;无论是github还是gittee&#xff0c;都是一种基于git去保存代码的形式&#xff0c;这样保存代码…...

编程新知 2025/9/8 20:54:35

Oracle11g安装包

Oracle 11g安装包 适用于windows系统&#xff0c;64位 下载路径 oracle 11g 安装包...

编程新知 2025/9/13 16:14:04

深度学习之模型压缩三驾马车：模型剪枝、模型量化、知识蒸馏

一、引言 在深度学习中&#xff0c;我们训练出的神经网络往往非常庞大&#xff08;比如像 ResNet、YOLOv8、Vision Transformer&#xff09;&#xff0c;虽然精度很高&#xff0c;但“太重”了&#xff0c;运行起来很慢&#xff0c;占用内存大&#xff0c;不适合部署到手机、摄…...

编程新知 2025/8/1 4:58:46

Python 训练营打卡 Day 47

注意力热力图可视化 在day 46代码的基础上&#xff0c;对比不同卷积层热力图可视化的结果 import torch import torch.nn as nn import torch.optim as optim from torchvision import datasets, transforms from torch.utils.data import DataLoader import matplotlib.pypl…...

编程新知 2025/9/1 8:36:19

springboot 日志类切面，接口成功记录日志，失败不记录

springboot 日志类切面&#xff0c;接口成功记录日志&#xff0c;失败不记录 自定义一个注解方法 import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target;/***…...

编程新知 2025/9/9 10:08:07

Python实现简单音频数据压缩与解压算法

Python实现简单音频数据压缩与解压算法 引言 在音频数据处理中&#xff0c;压缩算法是降低存储成本和传输效率的关键技术。Python作为一门灵活且功能强大的编程语言&#xff0c;提供了丰富的库和工具来实现音频数据的压缩与解压。本文将通过一个简单的音频数据压缩与解压算法…...

编程新知 2025/9/18 22:43:13