K8S资源限制之ResourceQuota

ResourceQuota介绍

在K8S中，大部分资源都可以指定到一个名称空间下，因此可以对一个名称空间的计算资源，存储资源，资源数量等维度做资源限制。

如限制pod数量、svc数量，控制器数量，限制PVC请求的存储量

注意事项

• 多个quota资源只要满足其中任意一个都会被限制，quota基于名称空间的粒度进行限制，无法做到一个名称空间下针对某个pod进行限制
• 名称空间使用quota进行了资源量请求限制，创建pod时必须指定pod的资源请求和限制。否则quota会拒绝创建

ResourceQuota配置案例

1 基于资源请求量进行限制

1.首先创建ResourceQuota资源，声明该名称空间下的CPU资源使用上线，pod数量…

cat > 01-compute-resources.yaml <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:name: compute-resourcesnamespace: kube-public
spec:# 定义硬性配置hard:# 配置名称空间请求cpu的相关参数，请求的总 CPU 核心数为 1，使用的最大限制为 2 核心requests.cpu: "1"limits.cpu: "2"requests.memory: 2Gilimits.memory: 3Gi# 定义GPU相关的参数# requests.nvidia.com/gpu: 4
EOF 

运行后查看配额

[root@master231~]# kubectl -n kube-public get quota
NAME                AGE    REQUEST                                     LIMIT
compute-resources   106s   requests.cpu: 0/1, requests.memory: 0/2Gi   limits.cpu: 0/2, limits.memory: 0/3Gi

2.创建第1个pod，让其用掉名称空间下的一部分资源

cat > 02-pods.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:name: pods-nginxnamespace: kube-public
spec:containers:- name: webimage: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1resources:requests:cpu: 0.5memory: 1Gilimits:cpu: 1memory: 2Gi
EOF

[root@master231~]# kubectl -n kube-public get quota
NAME                AGE    REQUEST                                          LIMIT
compute-resources   9m8s   requests.cpu: 500m/1, requests.memory: 1Gi/2Gi   limits.cpu: 1/2, limits.memory: 2Gi/3Gi

3.创建第2个pod，这时名称空间的资源已经不足了

cat > 03-pods.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:name: pods-alpinenamespace: kube-public
spec:containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v2resources:requests:cpu: 1.5memory: 2Gilimits:cpu: 2memory: 4Gi
EOF

kubectl apply -f 07-pods2.yaml 运行时直接提示拒绝创建

Error from server (Forbidden): error when creating “07-pods2.yaml”: pods “pods-alpine” is forbidden: exceeded quota: compute-resources, requested: limits.cpu=2,limits.memory=4Gi,requests.cpu=1500m,requests.memory=2Gi, used: limits.cpu=1,limits.memory=2Gi,requests.cpu=500m,requests.memory=1Gi, limited: limits.cpu=2,limits.memory=3Gi,requests.cpu=1,requests.memory=2Gi

2 基于对象数量进行限制

1.编写quota限制并应用，限制对应资源数量。上一步 基于资源请求量进行限制 的quota和pod留着不要删除。

cat > 01-object-counts.yaml <<EOF
apiVersion: v1
kind: ResourceQuota
metadata:name: object-countsnamespace: kube-public
spec:hard:pods: "10"count/deployments.apps: "3"count/services: "3"
EOF

再次查看quota限制

[root@master231~]# kubectl -n kube-public get quota
NAME                AGE     REQUEST                                                        LIMIT
compute-resources   46s     requests.cpu: 500m/1, requests.memory: 1Gi/2Gi                 limits.cpu: 1/2, limits.memory: 2Gi/3Gi
object-counts       5m57s   count/deployments.apps: 0/3, count/services: 0/3, pods: 1/10

2.创建pod，这次使用控制器指定pod数量。但是并不指定资源期望和限制

cat > 02-deploy-xiuxian.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:name: deloy-xiuxiannamespace: kube-public 
spec:replicas: 3selector:matchLabels:apps: xiuxiantemplate:metadata:labels:apps: xiuxianversion: v1spec:containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v2
EOF

pod虽然可以运用，但是没有创建出来

[root@master231~]# kubectl apply -f 02-deploy-xiuxian.yaml 
deployment.apps/deloy-xiuxian create[root@master231~]# kubectl -n kube-public get pods
NAME         READY   STATUS    RESTARTS   AGE
pods-nginx   1/1     Running   0          4m14s[root@master231~]# kubectl -n kube-public get deployments.apps 
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
deloy-xiuxian   0/3     0            0           3m10s

3.查看名称空间的事件信息：kubectl -n kube-public get events，看到以下信息

compute-resources: must specify limits.cpu,limits.memory,requests.cpu,requests.memory。

这是因为 Deployment 的 Pod 配置没有明确指定 requests.cpu、requests.memory、limits.cpu 和 limits.memory，而 ResourceQuota (compute-resources) 要求必须设置这些字段

4.移除compute-resources quota，再重建02-deploy-xiuxian.yaml，可以看到pod正常创建出来了

[root@master231~]# kubectl -n kube-public get quota
NAME                AGE   REQUEST                                                        LIMIT
compute-resources   26m   requests.cpu: 500m/1, requests.memory: 1Gi/2Gi                 limits.cpu: 1/2, limits.memory: 2Gi/3Gi
object-counts       31m   count/deployments.apps: 1/3, count/services: 0/3, pods: 1/10   
[root@master231~]# kubectl -n kube-public delete quota compute-resources
resourcequota "compute-resources" deleted
[root@master231~]# kubectl -n kube-public get quota
NAME            AGE   REQUEST                                                        LIMIT
object-counts   31m   count/deployments.apps: 1/3, count/services: 0/3, pods: 1/10

[root@master231~]# kubectl -n kube-public get pods
NAME                             READY   STATUS    RESTARTS   AGE
deloy-xiuxian-8676cbd54f-9lxl4   1/1     Running   0          6s
deloy-xiuxian-8676cbd54f-bpldg   1/1     Running   0          6s
deloy-xiuxian-8676cbd54f-xgbt8   1/1     Running   0          6s
pods-nginx                       1/1     Running   0          28m
[root@master231~]# kubectl -n kube-public get quota
NAME            AGE   REQUEST                                                        LIMIT
object-counts   34m   count/deployments.apps: 1/3, count/services: 0/3, pods: 4/10

5.继续在kube-public名称空间下创建pod，这次指定数量为10个。但是已经超出了10个，所以多出的4个是不会创建出来的。

[root@master231~]# kubectl -n kube-public get pods
NAME                                  READY   STATUS    RESTARTS   AGE
deloy-xiuxian-10pod-5b85959cd-2n2dk   1/1     Running   0          7s
deloy-xiuxian-10pod-5b85959cd-gpjmn   1/1     Running   0          7s
deloy-xiuxian-10pod-5b85959cd-jpbr2   1/1     Running   0          7s
deloy-xiuxian-10pod-5b85959cd-mfnpj   1/1     Running   0          7s
deloy-xiuxian-10pod-5b85959cd-rdnrl   1/1     Running   0          7s
deloy-xiuxian-10pod-5b85959cd-sf2km   1/1     Running   0          7s
deloy-xiuxian-8676cbd54f-9lxl4        1/1     Running   0          13m
deloy-xiuxian-8676cbd54f-bpldg        1/1     Running   0          13m
deloy-xiuxian-8676cbd54f-xgbt8        1/1     Running   0          13m
pods-nginx                            1/1     Running   0          41m
[root@master231~]# kubectl -n kube-public get deployments.apps 
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
deloy-xiuxian         3/3     3            3           13m
deloy-xiuxian-10pod   6/10    6            6           11s
[root@master231~]# kubectl -n kube-public get deployments.apps deloy-xiuxian-10pod 
NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
deloy-xiuxian-10pod   6/10    6            6           18s