k8s单master多node环境搭建-k8s版本低于1.24，容器运行时为docker

1.环境规划

服务器为最小化部署

集群角色	IP	主机名	安装组件
控制节点	192.168.2.217	master1	apiserver、controller-manager、scheduler、kubelet、etcd、docker、kube-proxy、keepalived、nginx、calico
工作节点1	192.168.2.219	node1	kubelet、docker、kube-proxy、calico、coredns
工作节点2	192.168.2.239	node2	kubelet、docker、kube-proxy、calico、coredns

k8s环境规划：
podSubnet（pod网段）10.244.0.0/16
serviceSubnet（service网段）: 10.96.0.0/12

kubeadm和二进制安装k8s适用场景分析
kubeadm是官方提供的开源工具，是一个开源项目，用于快速搭建kubernetes集群，目前是比较方便和推荐使用的。kubeadm init 以及 kubeadm join 这两个命令可以快速创建 kubernetes 集群。Kubeadm初始化k8s，所有的组件都是以pod形式运行的，具备故障自恢复能力。
kubeadm是工具，可以快速搭建集群，也就是相当于用程序脚本帮我们装好了集群，属于自动部署，简化部署操作，自动部署屏蔽了很多细节，使得对各个模块感知很少，如果对k8s架构组件理解不深的话，遇到问题比较难排查。
kubeadm适合需要经常部署k8s，或者对自动化要求比较高的场景下使用。
二进制：在官网下载相关组件的二进制包，如果手动安装，对kubernetes理解也会更全面。 
Kubeadm和二进制都适合生产环境，在生产环境运行都很稳定，具体如何选择，可以根据实际项目进行评估。

2.初始化服务器

1）配置主机名

# 三个设备分别执行
[root@localhost ~]# hostnamectl set-hostname master1 && bash
[root@localhost ~]# hostnamectl set-hostname node1 && bash
[root@localhost ~]# hostnamectl set-hostname node2 && bash# && bash 当前会话立即生效

2）设置IP为静态IP

# 三个设备都需要修改-注意IP地址不要重复
[root@master1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-ens192
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"	# 由dhcp修改为static
IPADDR="192.168.2.217"	# 设置IP地址
NETMASK="255.255.255.0"	# 设置子网掩码
GATEWAY="192.168.2.1"	# 设置网关：由ip route或netstat -rn等命令查询
DNS1="192.168.2.1"	# 设置DNS
DNS2="8.8.8.8"	# 设置DNS
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens32"
UUID="71ffc482-d255-4de0-a06c-0a5c036e8e96"
DEVICE="ens32"
ONBOOT="yes"
# node1(219)和node2(239)节点配置与上述配置除IP外，其余相同
# 重启网络
[root@master1 ~]# service network restart

3）关闭selinux

# 三个设备都需要修改
# 临时关闭
[root@master1 ~]# setenforce 0
# 永久关闭
[root@master1 ~]# vi /etc/selinux/config
# 将SELINUX设置为disabled
SELINUX=disabled
# 此参数修改完成后，需要重启服务器才能生效
# 检查修改状态
[root@master1 ~]# getenforce
显示为Disabled则说明修改成功

重启后检查结果如下（重启命令：reboot）

4）配置主机hosts文件

# 相互之间通过主机名互相访问，三个设备都需要修改
# 修改三台机器的/etc/hosts文件，增加如下三行
192.168.2.217 master1
192.168.2.219 node1
192.168.2.239 node2
# 修改后显示为
[root@master1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.217 master1
192.168.2.219 node1
192.168.2.239 node2

5）配置三台主机之间免密登录

# 三台设备都生成证书，并将SSH信息复制到其他节点（可选）
[root@master1 ~]# ssh-keygen	#一路回车，无需输入密码
[root@master1 ~]# ssh-copy-id master1	# 复制ssh信息到master1节点
[root@master1 ~]# ssh-copy-id node1	# 复制ssh信息到node1节点
[root@master1 ~]# ssh-copy-id node2	# 复制ssh信息到node2节点

6）关闭交换分区swap，提升性能

# 三台设备都运行
# 临时关闭
[root@master1 ~]# swapoff -a
# 永久关闭
[root@master1 ~]# vi /etc/fstab
# /dev/mapper/centos-swap swap                    swap    defaults        0 0
# 关闭后需要重启服务器，否则在初始化k8s时候会报错

7）修改内核参数

# 三台设备都运行
# 加载模块
[root@master1~]# modprobe br_netfilter
# 修改内核参数（创建docker.conf文件并写入内核参数）
[root@master1 ~]# cat > /etc/sysctl.d/docker.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
# 使参数生效
[root@master1~]# sysctl -p /etc/sysctl.d/docker.conf

8）关闭firewalld防火墙

# 三台设备都运行
[root@master1~]# systemctl stop firewalld && systemctl disable firewalld
# 或 [root@master1 ~]# systemctl disable firewalld --now

9）配置阿里云repo源

# 三台设备都运行
# 默认yum源无法使用，需要更换成国内yum源，演示为阿里云yum源# 备份基础repo源
[root@master1 ~]# mkdir /root/repo.bak
[root@master1 ~]# cd /etc/yum.repos.d/
[root@master1 ~]# mv * /root/repo.bak/
# 把CentOS-Base.repo和epel.repo文件上传到各个主机的/etc/yum.repos.d/目录下（具体内容参考下述）# 修改后安装依赖进行测试
# 安装rzsz
[root@master1 ~]# yum install lrzsz -y
# 安装scp
[root@master1 ~]# yum install openssh-clients# 配置国内阿里云docker的repo源
[root@master1 ~]# yum install yum-utils -y
[root@master1 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

CentOS-Base.repo文件内容

# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the 
# remarked out baseurl= line instead.
#
#[base]
name=CentOS-$releasever - Base - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/os/$basearch/http://mirrors.aliyuncs.com/centos/$releasever/os/$basearch/http://mirrors.cloud.aliyuncs.com/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#released updates 
[updates]
name=CentOS-$releasever - Updates - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/updates/$basearch/http://mirrors.aliyuncs.com/centos/$releasever/updates/$basearch/http://mirrors.cloud.aliyuncs.com/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/extras/$basearch/http://mirrors.aliyuncs.com/centos/$releasever/extras/$basearch/http://mirrors.cloud.aliyuncs.com/centos/$releasever/extras/$basearch/
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/centosplus/$basearch/http://mirrors.aliyuncs.com/centos/$releasever/centosplus/$basearch/http://mirrors.cloud.aliyuncs.com/centos/$releasever/centosplus/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib - mirrors.aliyun.com
failovermethod=priority
baseurl=http://mirrors.aliyun.com/centos/$releasever/contrib/$basearch/http://mirrors.aliyuncs.com/centos/$releasever/contrib/$basearch/http://mirrors.cloud.aliyuncs.com/centos/$releasever/contrib/$basearch/
gpgcheck=1
enabled=0
gpgkey=http://mirrors.aliyun.com/centos/RPM-GPG-KEY-CentOS-7

epel.repo文件内容

[epel]
name=Extra Packages for Enterprise Linux 7 - $basearch
#baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7[epel-debuginfo]
name=Extra Packages for Enterprise Linux 7 - $basearch - Debug
#baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch/debug
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
gpgcheck=1[epel-source]
name=Extra Packages for Enterprise Linux 7 - $basearch - Source
#baseurl=http://download.fedoraproject.org/pub/epel/7/SRPMS
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
gpgcheck=1

安装测试

操作完成后，目录下只存在3个repo

10）配置安装k8s组件需要的阿里云repo源

# 三台设备都运行
[root@master1 ~]# vi /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0

11）配置时间同步

# 三台设备都运行
# 安装ntp服务
[root@docker ~]# yum -y install ntp ntpdate
# 同步时间（若存在本地时间服务器，可将cn.pool.ntp.org换成时间服务器IP）
[root@docker ~]# ntpdate cn.pool.ntp.org
# 编写计划任务
[root@docker ~]# crontab -e
输入：
* */1 * * * /usr/sbin/ntpdate cn.pool.ntp.org
# 可使用crontab -l命令查看
# 重启crond服务使配置生效
[root@docker ~]# systemctl restart crond

时间同步

制作定时任务及查看

12）安装基础软件包

# 三台设备都运行
[root@master1 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2 wget net-tools nfs-utils lrzsz gcc gcc-c++ make cmake libxml2-devel openssl-devel curl curl-devel unzip sudo ntp libaio-devel wget vim ncurses-devel autoconf automake zlib-devel  python-devel epel-release openssh-server socat  ipvsadm conntrack ntpdate telnet ipvsadm vim

3.安装docker服务

# 三台设备都运行
1）安装docker-ce指定版本
[root@docker ~]# yum install -y docker-ce-20.10.6 docker-ce-cli-20.10.6 containerd.io
# 启动docker服务并配置为自启动
[root@docker ~]# systemctl start docker && systemctl enable docker && systemctl status docker
# running 表示运行状态
2）配置docker镜像加速器和驱动
[root@master1 ~]# vim  /etc/docker/daemon.json
{"registry-mirrors":["https://axcmsqgw.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com","http://qtid6917.mirror.aliyuncs.com", "https://rncxm540.mirror.aliyuncs.com"],"exec-opts": ["native.cgroupdriver=systemd"]
}
3）重新加载配置，且重启docker
[root@master1 ~]# systemctl daemon-reload  && systemctl restart docker
4）安装初始化k8s需要的软件包
[root@master1 ~]# yum install -y kubelet-1.20.6 kubeadm-1.20.6 kubectl-1.20.6
# 配置开机自启动（先不用启动）
[root@master1 ~]# systemctl enable kubelet
注：每个软件包的作用
Kubeadm:  kubeadm是一个工具，用来初始化k8s集群的
kubelet:   安装在集群所有节点上，用于启动Pod的
kubectl:   通过kubectl可以部署和管理应用，查看各种资源，创建、删除和更新各种组件

4.kubeadm初始化k8s

1）镜像下载并打包

# 需要的镜像为以下（建议master1节点拉取）
# 若全部节点拉起，则无需再将此镜像打包传输至工作节点
[root@master1 scripts]# docker pull registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_kube-controllers_v3.18.0
[root@master1 scripts]# docker pull registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_cni_v3.18.0
[root@master1 scripts]# docker pull registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_node_v3.18.0
[root@master1 scripts]# docker pull registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_pod2daemon-flexvol_v3.18.0
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/coredns:1.7.0
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/pause:3.2
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/kube-controller-manager:v1.20.6
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/kube-scheduler:v1.20.6
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/etcd:3.4.13-0
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/kube-proxy:v1.20.6
[root@master1 scripts]# docker pull registry.aliyuncs.com/google_containers/kube-apiserver:v1.20.6# 将下载的镜像打包到k8simages-1.20.6.tar.gz文件中，node节点就无需对镜像一个个进行下载
[root@master1 scripts]# docker save -o k8simages-1.20.6.tar.gz registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_kube-controllers_v3.18.0 registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_cni_v3.18.0 registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_node_v3.18.0 registry.cn-hangzhou.aliyuncs.com/zhangxiaoye/app:calico_pod2daemon-flexvol_v3.18.0 registry.aliyuncs.com/google_containers/coredns:1.7.0 registry.aliyuncs.com/google_containers/pause:3.2 registry.aliyuncs.com/google_containers/kube-controller-manager:v1.20.6 registry.aliyuncs.com/google_containers/kube-scheduler:v1.20.6 registry.aliyuncs.com/google_containers/etcd:3.4.13-0 registry.aliyuncs.com/google_containers/kube-proxy:v1.20.6 registry.aliyuncs.com/google_containers/kube-apiserver:v1.20.6
# 工作节点解压镜像
[root@node1 scripts]# docker load -i k8simages-1.20.6.tar.gz
[root@node2 scripts]# docker load -i k8simages-1.20.6.tar.gz

master1节点拉取镜像并打包传输至node节点

node节点解压镜像

2）kubeadm初始化master节点

# 仅master1节点执行即可
1.生成配置文件
[root@master1 ~]# kubeadm config print init-defaults > kubeadm.yaml2.修改配置文件
[root@master1 ~]# vim kubeadm.yaml
# advertiseAddress: 192.168.2.217	master节点IP
# name: master1		master节点主机名
# imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers 阿里云镜像仓库
# kubernetesVersion: v1.20.6	k8s版本
# serviceSubnet: 10.96.0.0/12		service网段
# podSubnet: 10.244.0.0/16		（新增）pod网段
# criSocket: /var/run/dockershim.sock	容器运行池
# 在scheduler: {}下一行，新增以下配置
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

kubeadm.yaml文件内容如下：

[root@master1 ~]# cat kubeadm.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:- system:bootstrappers:kubeadm:default-node-tokentoken: abcdef.0123456789abcdefttl: 24h0m0susages:- signing- authentication
kind: InitConfiguration
localAPIEndpoint:advertiseAddress: 192.168.2.217bindPort: 6443
nodeRegistration:criSocket: /var/run/dockershim.sockname: master1taints:- effect: NoSchedulekey: node-role.kubernetes.io/master
---
apiServer:timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:type: CoreDNS
etcd:local:dataDir: /var/lib/etcd
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.20.6
networking:dnsDomain: cluster.localserviceSubnet: 10.96.0.0/12podSubnet: 10.244.0.0/16
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd

3.基于kubeadm.yaml文件初始化k8s
[root@master1 scripts]# kubeadm init --config=kubeadm.yaml --ignore-preflight-errors=SystemVerification
# mster1安装完成后，需要执行以下命令
[root@master1 ~]# mkdir -p $HOME/.kube
[root@master1 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master1 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config# 配置文件查看
#直接查看
[root@master1 ~]# cat /root/.kube/config
# 使用kubectl查看
[root@master1 ~]# kubectl config view

注：
1.若第一次初始化失败，再次初始化的时候，提醒端口冲突，则重启kubeadm即可
或者检查swap分区是否关闭
[root@master1 scripts]# kubeadm reset
2.若k8s版本不对，需将原始版本卸载之后，再通过命令指定版本安装
[root@master1 ~]# yum remove -y kubelet kubeadm kubectl	# 卸载
[root@master1 ~]# yum install -y kubelet-1.20.6 kubeadm-1.20.6 kubectl-1.20.6 # 安装

初始化完成成输出如下：

设置环境变量

3）查询k8s集群状态

[root@master1 scripts]# kubectl get nodes
# 其他节点查看集群状态，需要将master节点/root/.kube/目录下的config文件拷贝到对应服务器/root/.kube/目录下
[root@master1 ~]# kubectl get node
NAME      STATUS     ROLES                  AGE   VERSION
master1   NotReady   control-plane,master   59m   v1.20.6
# 此时集群状态还是NotReady状态，因为没有安装网络插件# 查询kube-system命名空间下的所有pod
[root@master1 ~]#  kubectl get pods -n kube-systemNAME                              READY   STATUS    RESTARTS   AGE
coredns-54d67798b7-6n2h7          0/1     Pending   0          61m
coredns-54d67798b7-dv4dj          0/1     Pending   0          61m
etcd-master1                      1/1     Running   0          61m
kube-apiserver-master1            1/1     Running   0          61m
kube-controller-manager-master1   1/1     Running   0          61m
kube-proxy-4v46j                  1/1     Running   0          61m
kube-scheduler-master1            1/1     Running   0          61m
You have new mail in /var/spool/mail/root
[root@master1 ~]# 
[root@master1 ~]# kubectl get pods -n kube-system -owide
NAME                              READY   STATUS    RESTARTS   AGE   IP              NODE      NOMINATED NODE   READINESS GATES
coredns-54d67798b7-6n2h7          0/1     Pending   0          61m   <none>          <none>    <none>           <none>
coredns-54d67798b7-dv4dj          0/1     Pending   0          61m   <none>          <none>    <none>           <none>
etcd-master1                      1/1     Running   0          61m   192.168.2.217   master1   <none>           <none>
kube-apiserver-master1            1/1     Running   0          61m   192.168.2.217   master1   <none>           <none>
kube-controller-manager-master1   1/1     Running   0          61m   192.168.2.217   master1   <none>           <none>
kube-proxy-4v46j                  1/1     Running   0          61m   192.168.2.217   master1   <none>           <none>
kube-scheduler-master1            1/1     Running   0          61m   192.168.2.217   master1   <none>           <none>

4）安装网络插件-Calico

# 仅master节点执行即可
下载地址：https://docs.projectcalico.org/manifests/calico.yaml
# 将calico.yaml上传至服务器，使用以下命令安装calico
[root@master1 ~]# kubectl apply -f calico.yaml
# 查看nodes状态，显示为ready即表示成功
[root@master1 ~]# kubectl get nodes
NAME      STATUS   ROLES                  AGE   VERSION
master1   Ready    control-plane,master   66m   v1.20.6
[root@master1 ~]# kubectl get pods -n kube-system
NAME                                       READY   STATUS              RESTARTS   AGE
calico-kube-controllers-6949477b58-pjtf2   0/1     ContainerCreating   0          2m11s
calico-node-q5k5j                          1/1     Running             0          2m10s
coredns-54d67798b7-6n2h7                   1/1     Running             0          67m
coredns-54d67798b7-dv4dj                   1/1     Running             0          67m
etcd-master1                               1/1     Running             0          67m
kube-apiserver-master1                     1/1     Running             0          67m
kube-controller-manager-master1            1/1     Running             0          67m
kube-proxy-4v46j                           1/1     Running             0          67m
kube-scheduler-master1                     1/1     Running             0          67m

k8s集群节点状态已经被更新

calico.yaml内容如下：

---
# Source: calico/templates/calico-config.yaml
# This ConfigMap is used to configure a self-hosted Calico installation.
kind: ConfigMap
apiVersion: v1
metadata:name: calico-confignamespace: kube-system
data:# Typha is disabled.typha_service_name: "none"# Configure the backend to use.calico_backend: "bird"# Configure the MTU to use for workload interfaces and tunnels.# By default, MTU is auto-detected, and explicitly setting this field should not be required.# You can override auto-detection by providing a non-zero value.veth_mtu: "0"# The CNI network configuration to install on each node. The special# values in this config will be automatically populated.cni_network_config: |-{"name": "k8s-pod-network","cniVersion": "0.3.1","plugins": [{"type": "calico","log_level": "info","log_file_path": "/var/log/calico/cni/cni.log","datastore_type": "kubernetes","nodename": "__KUBERNETES_NODE_NAME__","mtu": __CNI_MTU__,"ipam": {"type": "calico-ipam"},"policy": {"type": "k8s"},"kubernetes": {"kubeconfig": "__KUBECONFIG_FILEPATH__"}},{"type": "portmap","snat": true,"capabilities": {"portMappings": true}},{"type": "bandwidth","capabilities": {"bandwidth": true}}]}---
# Source: calico/templates/kdd-crds.yamlapiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: bgpconfigurations.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: BGPConfigurationlistKind: BGPConfigurationListplural: bgpconfigurationssingular: bgpconfigurationscope: Clusterversions:- name: v1schema:openAPIV3Schema:description: BGPConfiguration contains the configuration for any BGP routing.properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: BGPConfigurationSpec contains the values of the BGP configuration.properties:asNumber:description: 'ASNumber is the default AS number used by a node. [Default:64512]'format: int32type: integercommunities:description: Communities is a list of BGP community values and theirarbitrary names for tagging routes.items:description: Community contains standard or large community valueand its name.properties:name:description: Name given to community value.type: stringvalue:description: Value must be of format `aa:nn` or `aa:nn:mm`.For standard community use `aa:nn` format, where `aa` and`nn` are 16 bit number. For large community use `aa:nn:mm`format, where `aa`, `nn` and `mm` are 32 bit number. Where,`aa` is an AS Number, `nn` and `mm` are per-AS identifier.pattern: ^(\d+):(\d+)$|^(\d+):(\d+):(\d+)$type: stringtype: objecttype: arraylistenPort:description: ListenPort is the port where BGP protocol should listen.Defaults to 179maximum: 65535minimum: 1type: integerlogSeverityScreen:description: 'LogSeverityScreen is the log severity above which logsare sent to the stdout. [Default: INFO]'type: stringnodeToNodeMeshEnabled:description: 'NodeToNodeMeshEnabled sets whether full node to nodeBGP mesh is enabled. [Default: true]'type: booleanprefixAdvertisements:description: PrefixAdvertisements contains per-prefix advertisementconfiguration.items:description: PrefixAdvertisement configures advertisement propertiesfor the specified CIDR.properties:cidr:description: CIDR for which properties should be advertised.type: stringcommunities:description: Communities can be list of either community namesalready defined in `Specs.Communities` or community valueof format `aa:nn` or `aa:nn:mm`. For standard community use`aa:nn` format, where `aa` and `nn` are 16 bit number. Forlarge community use `aa:nn:mm` format, where `aa`, `nn` and`mm` are 32 bit number. Where,`aa` is an AS Number, `nn` and`mm` are per-AS identifier.items:type: stringtype: arraytype: objecttype: arrayserviceClusterIPs:description: ServiceClusterIPs are the CIDR blocks from which servicecluster IPs are allocated. If specified, Calico will advertise theseblocks, as well as any cluster IPs within them.items:description: ServiceClusterIPBlock represents a single allowed ClusterIPCIDR block.properties:cidr:type: stringtype: objecttype: arrayserviceExternalIPs:description: ServiceExternalIPs are the CIDR blocks for KubernetesService External IPs. Kubernetes Service ExternalIPs will only beadvertised if they are within one of these blocks.items:description: ServiceExternalIPBlock represents a single allowedExternal IP CIDR block.properties:cidr:type: stringtype: objecttype: arrayserviceLoadBalancerIPs:description: ServiceLoadBalancerIPs are the CIDR blocks for KubernetesService LoadBalancer IPs. Kubernetes Service status.LoadBalancer.IngressIPs will only be advertised if they are within one of these blocks.items:description: ServiceLoadBalancerIPBlock represents a single allowedLoadBalancer IP CIDR block.properties:cidr:type: stringtype: objecttype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: bgppeers.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: BGPPeerlistKind: BGPPeerListplural: bgppeerssingular: bgppeerscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: BGPPeerSpec contains the specification for a BGPPeer resource.properties:asNumber:description: The AS Number of the peer.format: int32type: integerkeepOriginalNextHop:description: Option to keep the original nexthop field when routesare sent to a BGP Peer. Setting "true" configures the selected BGPPeers node to use the "next hop keep;" instead of "next hop self;"(default)in the specific branch of the Node on "bird.cfg".type: booleannode:description: The node name identifying the Calico node instance thatis targeted by this peer. If this is not set, and no nodeSelectoris specified, then this BGP peer selects all nodes in the cluster.type: stringnodeSelector:description: Selector for the nodes that should have this peering.  Whenthis is set, the Node field must be empty.type: stringpassword:description: Optional BGP password for the peerings generated by thisBGPPeer resource.properties:secretKeyRef:description: Selects a key of a secret in the node pod's namespace.properties:key:description: The key of the secret to select from.  Must bea valid secret key.type: stringname:description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#namesTODO: Add other useful fields. apiVersion, kind, uid?'type: stringoptional:description: Specify whether the Secret or its key must bedefinedtype: booleanrequired:- keytype: objecttype: objectpeerIP:description: The IP address of the peer followed by an optional portnumber to peer with. If port number is given, format should be `[<IPv6>]:port`or `<IPv4>:<port>` for IPv4. If optional port number is not set,and this peer IP and ASNumber belongs to a calico/node with ListenPortset in BGPConfiguration, then we use that port to peer.type: stringpeerSelector:description: Selector for the remote nodes to peer with.  When thisis set, the PeerIP and ASNumber fields must be empty.  For eachpeering between the local node and selected remote nodes, we configurean IPv4 peering if both ends have NodeBGPSpec.IPv4Address specified,and an IPv6 peering if both ends have NodeBGPSpec.IPv6Address specified.  Theremote AS number comes from the remote node's NodeBGPSpec.ASNumber,or the global default if that is not set.type: stringsourceAddress:description: Specifies whether and how to configure a source addressfor the peerings generated by this BGPPeer resource.  Default value"UseNodeIP" means to configure the node IP as the source address.  "None"means not to configure a source address.type: stringtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: blockaffinities.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: BlockAffinitylistKind: BlockAffinityListplural: blockaffinitiessingular: blockaffinityscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: BlockAffinitySpec contains the specification for a BlockAffinityresource.properties:cidr:type: stringdeleted:description: Deleted indicates that this block affinity is being deleted.This field is a string for compatibility with older releases thatmistakenly treat this field as a string.type: stringnode:type: stringstate:type: stringrequired:- cidr- deleted- node- statetype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: clusterinformations.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: ClusterInformationlistKind: ClusterInformationListplural: clusterinformationssingular: clusterinformationscope: Clusterversions:- name: v1schema:openAPIV3Schema:description: ClusterInformation contains the cluster specific information.properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: ClusterInformationSpec contains the values of describingthe cluster.properties:calicoVersion:description: CalicoVersion is the version of Calico that the clusteris runningtype: stringclusterGUID:description: ClusterGUID is the GUID of the clustertype: stringclusterType:description: ClusterType describes the type of the clustertype: stringdatastoreReady:description: DatastoreReady is used during significant datastore migrationsto signal to components such as Felix that it should wait beforeaccessing the datastore.type: booleanvariant:description: Variant declares which variant of Calico should be active.type: stringtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: felixconfigurations.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: FelixConfigurationlistKind: FelixConfigurationListplural: felixconfigurationssingular: felixconfigurationscope: Clusterversions:- name: v1schema:openAPIV3Schema:description: Felix Configuration contains the configuration for Felix.properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: FelixConfigurationSpec contains the values of the Felix configuration.properties:allowIPIPPacketsFromWorkloads:description: 'AllowIPIPPacketsFromWorkloads controls whether Felixwill add a rule to drop IPIP encapsulated traffic from workloads[Default: false]'type: booleanallowVXLANPacketsFromWorkloads:description: 'AllowVXLANPacketsFromWorkloads controls whether Felixwill add a rule to drop VXLAN encapsulated traffic from workloads[Default: false]'type: booleanawsSrcDstCheck:description: 'Set source-destination-check on AWS EC2 instances. Acceptedvalue must be one of "DoNothing", "Enabled" or "Disabled". [Default:DoNothing]'enum:- DoNothing- Enable- Disabletype: stringbpfConnectTimeLoadBalancingEnabled:description: 'BPFConnectTimeLoadBalancingEnabled when in BPF mode,controls whether Felix installs the connection-time load balancer.  Theconnect-time load balancer is required for the host to be able toreach Kubernetes services and it improves the performance of pod-to-serviceconnections.  The only reason to disable it is for debugging purposes.  [Default:true]'type: booleanbpfDataIfacePattern:description: BPFDataIfacePattern is a regular expression that controlswhich interfaces Felix should attach BPF programs to in order tocatch traffic to/from the network.  This needs to match the interfacesthat Calico workload traffic flows over as well as any interfacesthat handle incoming traffic to nodeports and services from outsidethe cluster.  It should not match the workload interfaces (usuallynamed cali...).type: stringbpfDisableUnprivileged:description: 'BPFDisableUnprivileged, if enabled, Felix sets the kernel.unprivileged_bpf_disabledsysctl to disable unprivileged use of BPF.  This ensures that unprivilegedusers cannot access Calico''s BPF maps and cannot insert their ownBPF programs to interfere with Calico''s. [Default: true]'type: booleanbpfEnabled:description: 'BPFEnabled, if enabled Felix will use the BPF dataplane.[Default: false]'type: booleanbpfExternalServiceMode:description: 'BPFExternalServiceMode in BPF mode, controls how connectionsfrom outside the cluster to services (node ports and cluster IPs)are forwarded to remote workloads.  If set to "Tunnel" then bothrequest and response traffic is tunneled to the remote node.  Ifset to "DSR", the request traffic is tunneled but the response trafficis sent directly from the remote node.  In "DSR" mode, the remotenode appears to use the IP of the ingress node; this requires apermissive L2 network.  [Default: Tunnel]'type: stringbpfKubeProxyEndpointSlicesEnabled:description: BPFKubeProxyEndpointSlicesEnabled in BPF mode, controlswhether Felix's embedded kube-proxy accepts EndpointSlices or not.type: booleanbpfKubeProxyIptablesCleanupEnabled:description: 'BPFKubeProxyIptablesCleanupEnabled, if enabled in BPFmode, Felix will proactively clean up the upstream Kubernetes kube-proxy''siptables chains.  Should only be enabled if kube-proxy is not running.  [Default:true]'type: booleanbpfKubeProxyMinSyncPeriod:description: 'BPFKubeProxyMinSyncPeriod, in BPF mode, controls theminimum time between updates to the dataplane for Felix''s embeddedkube-proxy.  Lower values give reduced set-up latency.  Higher valuesreduce Felix CPU usage by batching up more work.  [Default: 1s]'type: stringbpfLogLevel:description: 'BPFLogLevel controls the log level of the BPF programswhen in BPF dataplane mode.  One of "Off", "Info", or "Debug".  Thelogs are emitted to the BPF trace pipe, accessible with the command`tc exec bpf debug`. [Default: Off].'type: stringchainInsertMode:description: 'ChainInsertMode controls whether Felix hooks the kernel''stop-level iptables chains by inserting a rule at the top of thechain or by appending a rule at the bottom. insert is the safe defaultsince it prevents Calico''s rules from being bypassed. If you switchto append mode, be sure that the other rules in the chains signalacceptance by falling through to the Calico rules, otherwise theCalico policy will be bypassed. [Default: insert]'type: stringdataplaneDriver:type: stringdebugDisableLogDropping:type: booleandebugMemoryProfilePath:type: stringdebugSimulateCalcGraphHangAfter:type: stringdebugSimulateDataplaneHangAfter:type: stringdefaultEndpointToHostAction:description: 'DefaultEndpointToHostAction controls what happens totraffic that goes from a workload endpoint to the host itself (afterthe traffic hits the endpoint egress policy). By default Calicoblocks traffic from workload endpoints to the host itself with aniptables "DROP" action. If you want to allow some or all trafficfrom endpoint to host, set this parameter to RETURN or ACCEPT. UseRETURN if you have your own rules in the iptables "INPUT" chain;Calico will insert its rules at the top of that chain, then "RETURN"packets to the "INPUT" chain once it has completed processing workloadendpoint egress policy. Use ACCEPT to unconditionally accept packetsfrom workloads after processing workload endpoint egress policy.[Default: Drop]'type: stringdeviceRouteProtocol:description: This defines the route protocol added to programmed deviceroutes, by default this will be RTPROT_BOOT when left blank.type: integerdeviceRouteSourceAddress:description: This is the source address to use on programmed deviceroutes. By default the source address is left blank, leaving thekernel to choose the source address used.type: stringdisableConntrackInvalidCheck:type: booleanendpointReportingDelay:type: stringendpointReportingEnabled:type: booleanexternalNodesList:description: ExternalNodesCIDRList is a list of CIDR's of external-non-calico-nodeswhich may source tunnel traffic and have the tunneled traffic beaccepted at calico nodes.items:type: stringtype: arrayfailsafeInboundHostPorts:description: 'FailsafeInboundHostPorts is a comma-delimited list ofUDP/TCP ports that Felix will allow incoming traffic to host endpointson irrespective of the security policy. This is useful to avoidaccidentally cutting off a host with incorrect configuration. Eachport should be specified as tcp:<port-number> or udp:<port-number>.For back-compatibility, if the protocol is not specified, it defaultsto "tcp". To disable all inbound host ports, use the value none.The default value allows ssh access and DHCP. [Default: tcp:22,udp:68, tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666, tcp:6667]'items:description: ProtoPort is combination of protocol and port, bothmust be specified.properties:port:type: integerprotocol:type: stringrequired:- port- protocoltype: objecttype: arrayfailsafeOutboundHostPorts:description: 'FailsafeOutboundHostPorts is a comma-delimited listof UDP/TCP ports that Felix will allow outgoing traffic from hostendpoints to irrespective of the security policy. This is usefulto avoid accidentally cutting off a host with incorrect configuration.Each port should be specified as tcp:<port-number> or udp:<port-number>.For back-compatibility, if the protocol is not specified, it defaultsto "tcp". To disable all outbound host ports, use the value none.The default value opens etcd''s standard ports to ensure that Felixdoes not get cut off from etcd as well as allowing DHCP and DNS.[Default: tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666, tcp:6667,udp:53, udp:67]'items:description: ProtoPort is combination of protocol and port, bothmust be specified.properties:port:type: integerprotocol:type: stringrequired:- port- protocoltype: objecttype: arrayfeatureDetectOverride:description: FeatureDetectOverride is used to override the featuredetection. Values are specified in a comma separated list with nospaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock="."true" or "false" will force the feature, empty or omitted valuesare auto-detected.type: stringgenericXDPEnabled:description: 'GenericXDPEnabled enables Generic XDP so network cardsthat don''t support XDP offload or driver modes can use XDP. Thisis not recommended since it doesn''t provide better performancethan iptables. [Default: false]'type: booleanhealthEnabled:type: booleanhealthHost:type: stringhealthPort:type: integerinterfaceExclude:description: 'InterfaceExclude is a comma-separated list of interfacesthat Felix should exclude when monitoring for host endpoints. Thedefault value ensures that Felix ignores Kubernetes'' IPVS dummyinterface, which is used internally by kube-proxy. If you want toexclude multiple interface names using a single value, the listsupports regular expressions. For regular expressions you must wrapthe value with ''/''. For example having values ''/^kube/,veth1''will exclude all interfaces that begin with ''kube'' and also theinterface ''veth1''. [Default: kube-ipvs0]'type: stringinterfacePrefix:description: 'InterfacePrefix is the interface name prefix that identifiesworkload endpoints and so distinguishes them from host endpointinterfaces. Note: in environments other than bare metal, the orchestratorsconfigure this appropriately. For example our Kubernetes and Dockerintegrations set the ''cali'' value, and our OpenStack integrationsets the ''tap'' value. [Default: cali]'type: stringinterfaceRefreshInterval:description: InterfaceRefreshInterval is the period at which Felixrescans local interfaces to verify their state. The rescan can bedisabled by setting the interval to 0.type: stringipipEnabled:type: booleanipipMTU:description: 'IPIPMTU is the MTU to set on the tunnel device. SeeConfiguring MTU [Default: 1440]'type: integeripsetsRefreshInterval:description: 'IpsetsRefreshInterval is the period at which Felix re-checksall iptables state to ensure that no other process has accidentallybroken Calico''s rules. Set to 0 to disable iptables refresh. [Default:90s]'type: stringiptablesBackend:description: IptablesBackend specifies which backend of iptables willbe used. The default is legacy.type: stringiptablesFilterAllowAction:type: stringiptablesLockFilePath:description: 'IptablesLockFilePath is the location of the iptableslock file. You may need to change this if the lock file is not inits standard location (for example if you have mapped it into Felix''scontainer at a different path). [Default: /run/xtables.lock]'type: stringiptablesLockProbeInterval:description: 'IptablesLockProbeInterval is the time that Felix willwait between attempts to acquire the iptables lock if it is notavailable. Lower values make Felix more responsive when the lockis contended, but use more CPU. [Default: 50ms]'type: stringiptablesLockTimeout:description: 'IptablesLockTimeout is the time that Felix will waitfor the iptables lock, or 0, to disable. To use this feature, Felixmust share the iptables lock file with all other processes thatalso take the lock. When running Felix inside a container, thisrequires the /run directory of the host to be mounted into the calico/nodeor calico/felix container. [Default: 0s disabled]'type: stringiptablesMangleAllowAction:type: stringiptablesMarkMask:description: 'IptablesMarkMask is the mask that Felix selects itsIPTables Mark bits from. Should be a 32 bit hexadecimal number withat least 8 bits set, none of which clash with any other mark bitsin use on the system. [Default: 0xff000000]'format: int32type: integeriptablesNATOutgoingInterfaceFilter:type: stringiptablesPostWriteCheckInterval:description: 'IptablesPostWriteCheckInterval is the period after Felixhas done a write to the dataplane that it schedules an extra readback in order to check the write was not clobbered by another process.This should only occur if another application on the system doesn''trespect the iptables lock. [Default: 1s]'type: stringiptablesRefreshInterval:description: 'IptablesRefreshInterval is the period at which Felixre-checks the IP sets in the dataplane to ensure that no other processhas accidentally broken Calico''s rules. Set to 0 to disable IPsets refresh. Note: the default for this value is lower than theother refresh intervals as a workaround for a Linux kernel bug thatwas fixed in kernel version 4.11. If you are using v4.11 or greateryou may want to set this to, a higher value to reduce Felix CPUusage. [Default: 10s]'type: stringipv6Support:type: booleankubeNodePortRanges:description: 'KubeNodePortRanges holds list of port ranges used forservice node ports. Only used if felix detects kube-proxy runningin ipvs mode. Felix uses these ranges to separate host and workloadtraffic. [Default: 30000:32767].'items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraylogFilePath:description: 'LogFilePath is the full path to the Felix log. Set tonone to disable file logging. [Default: /var/log/calico/felix.log]'type: stringlogPrefix:description: 'LogPrefix is the log prefix that Felix uses when renderingLOG rules. [Default: calico-packet]'type: stringlogSeverityFile:description: 'LogSeverityFile is the log severity above which logsare sent to the log file. [Default: Info]'type: stringlogSeverityScreen:description: 'LogSeverityScreen is the log severity above which logsare sent to the stdout. [Default: Info]'type: stringlogSeveritySys:description: 'LogSeveritySys is the log severity above which logsare sent to the syslog. Set to None for no logging to syslog. [Default:Info]'type: stringmaxIpsetSize:type: integermetadataAddr:description: 'MetadataAddr is the IP address or domain name of theserver that can answer VM queries for cloud-init metadata. In OpenStack,this corresponds to the machine running nova-api (or in Ubuntu,nova-api-metadata). A value of none (case insensitive) means thatFelix should not set up any NAT rule for the metadata path. [Default:127.0.0.1]'type: stringmetadataPort:description: 'MetadataPort is the port of the metadata server. This,combined with global.MetadataAddr (if not ''None''), is used toset up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort.In most cases this should not need to be changed [Default: 8775].'type: integermtuIfacePattern:description: MTUIfacePattern is a regular expression that controlswhich interfaces Felix should scan in order to calculate the host'sMTU. This should not match workload interfaces (usually named cali...).type: stringnatOutgoingAddress:description: NATOutgoingAddress specifies an address to use when performingsource NAT for traffic in a natOutgoing pool that is leaving thenetwork. By default the address used is an address on the interfacethe traffic is leaving on (ie it uses the iptables MASQUERADE target)type: stringnatPortRange:anyOf:- type: integer- type: stringdescription: NATPortRange specifies the range of ports that is usedfor port mapping when doing outgoing NAT. When unset the defaultbehavior of the network stack is used.pattern: ^.*x-kubernetes-int-or-string: truenetlinkTimeout:type: stringopenstackRegion:description: 'OpenstackRegion is the name of the region that a particularFelix belongs to. In a multi-region Calico/OpenStack deployment,this must be configured somehow for each Felix (here in the datamodel,or in felix.cfg or the environment on each compute node), and mustmatch the [calico] openstack_region value configured in neutron.confon each node. [Default: Empty]'type: stringpolicySyncPathPrefix:description: 'PolicySyncPathPrefix is used to by Felix to communicatepolicy changes to external services, like Application layer policy.[Default: Empty]'type: stringprometheusGoMetricsEnabled:description: 'PrometheusGoMetricsEnabled disables Go runtime metricscollection, which the Prometheus client does by default, when setto false. This reduces the number of metrics reported, reducingPrometheus load. [Default: true]'type: booleanprometheusMetricsEnabled:description: 'PrometheusMetricsEnabled enables the Prometheus metricsserver in Felix if set to true. [Default: false]'type: booleanprometheusMetricsHost:description: 'PrometheusMetricsHost is the host that the Prometheusmetrics server should bind to. [Default: empty]'type: stringprometheusMetricsPort:description: 'PrometheusMetricsPort is the TCP port that the Prometheusmetrics server should bind to. [Default: 9091]'type: integerprometheusProcessMetricsEnabled:description: 'PrometheusProcessMetricsEnabled disables process metricscollection, which the Prometheus client does by default, when setto false. This reduces the number of metrics reported, reducingPrometheus load. [Default: true]'type: booleanremoveExternalRoutes:description: Whether or not to remove device routes that have notbeen programmed by Felix. Disabling this will allow external applicationsto also add device routes. This is enabled by default which meanswe will remove externally added routes.type: booleanreportingInterval:description: 'ReportingInterval is the interval at which Felix reportsits status into the datastore or 0 to disable. Must be non-zeroin OpenStack deployments. [Default: 30s]'type: stringreportingTTL:description: 'ReportingTTL is the time-to-live setting for process-widestatus reports. [Default: 90s]'type: stringrouteRefreshInterval:description: 'RouteRefreshInterval is the period at which Felix re-checksthe routes in the dataplane to ensure that no other process hasaccidentally broken Calico''s rules. Set to 0 to disable route refresh.[Default: 90s]'type: stringrouteSource:description: 'RouteSource configures where Felix gets its routinginformation. - WorkloadIPs: use workload endpoints to constructroutes. - CalicoIPAM: the default - use IPAM data to construct routes.'type: stringrouteTableRange:description: Calico programs additional Linux route tables for variouspurposes.  RouteTableRange specifies the indices of the route tablesthat Calico should use.properties:max:type: integermin:type: integerrequired:- max- mintype: objectserviceLoopPrevention:description: 'When service IP advertisement is enabled, prevent routingloops to service IPs that are not in use, by dropping or rejectingpackets that do not get DNAT''d by kube-proxy. Unless set to "Disabled",in which case such routing loops continue to be allowed. [Default:Drop]'type: stringsidecarAccelerationEnabled:description: 'SidecarAccelerationEnabled enables experimental sidecaracceleration [Default: false]'type: booleanusageReportingEnabled:description: 'UsageReportingEnabled reports anonymous Calico versionnumber and cluster size to projectcalico.org. Logs warnings returnedby the usage server. For example, if a significant security vulnerabilityhas been discovered in the version of Calico being used. [Default:true]'type: booleanusageReportingInitialDelay:description: 'UsageReportingInitialDelay controls the minimum delaybefore Felix makes a report. [Default: 300s]'type: stringusageReportingInterval:description: 'UsageReportingInterval controls the interval at whichFelix makes reports. [Default: 86400s]'type: stringuseInternalDataplaneDriver:type: booleanvxlanEnabled:type: booleanvxlanMTU:description: 'VXLANMTU is the MTU to set on the tunnel device. SeeConfiguring MTU [Default: 1440]'type: integervxlanPort:type: integervxlanVNI:type: integerwireguardEnabled:description: 'WireguardEnabled controls whether Wireguard is enabled.[Default: false]'type: booleanwireguardInterfaceName:description: 'WireguardInterfaceName specifies the name to use forthe Wireguard interface. [Default: wg.calico]'type: stringwireguardListeningPort:description: 'WireguardListeningPort controls the listening port usedby Wireguard. [Default: 51820]'type: integerwireguardMTU:description: 'WireguardMTU controls the MTU on the Wireguard interface.See Configuring MTU [Default: 1420]'type: integerwireguardRoutingRulePriority:description: 'WireguardRoutingRulePriority controls the priority valueto use for the Wireguard routing rule. [Default: 99]'type: integerxdpEnabled:description: 'XDPEnabled enables XDP acceleration for suitable untrackedincoming deny rules. [Default: true]'type: booleanxdpRefreshInterval:description: 'XDPRefreshInterval is the period at which Felix re-checksall XDP state to ensure that no other process has accidentally brokenCalico''s BPF maps or attached programs. Set to 0 to disable XDPrefresh. [Default: 90s]'type: stringtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: globalnetworkpolicies.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: GlobalNetworkPolicylistKind: GlobalNetworkPolicyListplural: globalnetworkpoliciessingular: globalnetworkpolicyscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:properties:applyOnForward:description: ApplyOnForward indicates to apply the rules in this policyon forward traffic.type: booleandoNotTrack:description: DoNotTrack indicates whether packets matched by the rulesin this policy should go through the data plane's connection tracking,such as Linux conntrack.  If True, the rules in this policy areapplied before any data plane connection tracking, and packets allowedby this policy are marked as not to be tracked.type: booleanegress:description: The ordered set of egress rules.  Each rule containsa set of packet match criteria and a corresponding action to apply.items:description: "A Rule encapsulates a set of match criteria and anaction.  Both selector-based security Policy and security Profilesreference rules - separated out as a list of rules for both ingressand egress packet matching. \n Each positive match criteria hasa negated version, prefixed with \"Not\". All the match criteriawithin a rule must be satisfied for a packet to match. A singlerule can contain the positive and negative version of a matchand both must be satisfied for the rule to match."properties:action:type: stringdestination:description: Destination contains the match criteria that applyto destination entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objecthttp:description: HTTP contains match criteria that apply to HTTPrequests.properties:methods:description: Methods is an optional field that restrictsthe rule to apply only to HTTP requests that use one ofthe listed HTTP Methods (e.g. GET, PUT, etc.) Multiplemethods are OR'd together.items:type: stringtype: arraypaths:description: 'Paths is an optional field that restrictsthe rule to apply to HTTP requests that use one of thelisted HTTP Paths. Multiple paths are OR''d together.e.g: - exact: /foo - prefix: /bar NOTE: Each entry mayONLY specify either a `exact` or a `prefix` match. Thevalidator will check for it.'items:description: 'HTTPPath specifies an HTTP path to match.It may be either of the form: exact: <path>: which matchesthe path exactly or prefix: <path-prefix>: which matchesthe path prefix'properties:exact:type: stringprefix:type: stringtype: objecttype: arraytype: objecticmp:description: ICMP is an optional field that restricts the ruleto apply to a specific type and code of ICMP traffic.  Thisshould only be specified if the Protocol field is set to "ICMP"or "ICMPv6".properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectipVersion:description: IPVersion is an optional field that restricts therule to only match a specific IP version.type: integermetadata:description: Metadata contains additional information for thisruleproperties:annotations:additionalProperties:type: stringdescription: Annotations is a set of key value pairs thatgive extra information about the ruletype: objecttype: objectnotICMP:description: NotICMP is the negated version of the ICMP field.properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectnotProtocol:anyOf:- type: integer- type: stringdescription: NotProtocol is the negated version of the Protocolfield.pattern: ^.*x-kubernetes-int-or-string: trueprotocol:anyOf:- type: integer- type: stringdescription: "Protocol is an optional field that restricts therule to only apply to traffic of a specific IP protocol. Requiredif any of the EntityRules contain Ports (because ports onlyapply to certain protocols). \n Must be one of these stringvalues: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",\"UDPLite\" or an integer in the range 1-255."pattern: ^.*x-kubernetes-int-or-string: truesource:description: Source contains the match criteria that apply tosource entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objectrequired:- actiontype: objecttype: arrayingress:description: The ordered set of ingress rules.  Each rule containsa set of packet match criteria and a corresponding action to apply.items:description: "A Rule encapsulates a set of match criteria and anaction.  Both selector-based security Policy and security Profilesreference rules - separated out as a list of rules for both ingressand egress packet matching. \n Each positive match criteria hasa negated version, prefixed with \"Not\". All the match criteriawithin a rule must be satisfied for a packet to match. A singlerule can contain the positive and negative version of a matchand both must be satisfied for the rule to match."properties:action:type: stringdestination:description: Destination contains the match criteria that applyto destination entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objecthttp:description: HTTP contains match criteria that apply to HTTPrequests.properties:methods:description: Methods is an optional field that restrictsthe rule to apply only to HTTP requests that use one ofthe listed HTTP Methods (e.g. GET, PUT, etc.) Multiplemethods are OR'd together.items:type: stringtype: arraypaths:description: 'Paths is an optional field that restrictsthe rule to apply to HTTP requests that use one of thelisted HTTP Paths. Multiple paths are OR''d together.e.g: - exact: /foo - prefix: /bar NOTE: Each entry mayONLY specify either a `exact` or a `prefix` match. Thevalidator will check for it.'items:description: 'HTTPPath specifies an HTTP path to match.It may be either of the form: exact: <path>: which matchesthe path exactly or prefix: <path-prefix>: which matchesthe path prefix'properties:exact:type: stringprefix:type: stringtype: objecttype: arraytype: objecticmp:description: ICMP is an optional field that restricts the ruleto apply to a specific type and code of ICMP traffic.  Thisshould only be specified if the Protocol field is set to "ICMP"or "ICMPv6".properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectipVersion:description: IPVersion is an optional field that restricts therule to only match a specific IP version.type: integermetadata:description: Metadata contains additional information for thisruleproperties:annotations:additionalProperties:type: stringdescription: Annotations is a set of key value pairs thatgive extra information about the ruletype: objecttype: objectnotICMP:description: NotICMP is the negated version of the ICMP field.properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectnotProtocol:anyOf:- type: integer- type: stringdescription: NotProtocol is the negated version of the Protocolfield.pattern: ^.*x-kubernetes-int-or-string: trueprotocol:anyOf:- type: integer- type: stringdescription: "Protocol is an optional field that restricts therule to only apply to traffic of a specific IP protocol. Requiredif any of the EntityRules contain Ports (because ports onlyapply to certain protocols). \n Must be one of these stringvalues: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",\"UDPLite\" or an integer in the range 1-255."pattern: ^.*x-kubernetes-int-or-string: truesource:description: Source contains the match criteria that apply tosource entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objectrequired:- actiontype: objecttype: arraynamespaceSelector:description: NamespaceSelector is an optional field for an expressionused to select a pod based on namespaces.type: stringorder:description: Order is an optional field that specifies the order inwhich the policy is applied. Policies with higher "order" are appliedafter those with lower order.  If the order is omitted, it may beconsidered to be "infinite" - i.e. the policy will be applied last.  Policieswith identical order will be applied in alphanumerical order basedon the Policy "Name".type: numberpreDNAT:description: PreDNAT indicates to apply the rules in this policy beforeany DNAT.type: booleanselector:description: "The selector is an expression used to pick pick outthe endpoints that the policy should be applied to. \n Selectorexpressions follow this syntax: \n \tlabel == \"string_literal\"\ ->  comparison, e.g. my_label == \"foo bar\" \tlabel != \"string_literal\"\  ->  not equal; also matches if label is not present \tlabel in{ \"a\", \"b\", \"c\", ... }  ->  true if the value of label X isone of \"a\", \"b\", \"c\" \tlabel not in { \"a\", \"b\", \"c\",... }  ->  true if the value of label X is not one of \"a\", \"b\",\"c\" \thas(label_name)  -> True if that label is present \t! expr-> negation of expr \texpr && expr  -> Short-circuit and \texpr|| expr  -> Short-circuit or \t( expr ) -> parens for grouping \tall()or the empty selector -> matches all endpoints. \n Label names areallowed to contain alphanumerics, -, _ and /. String literals aremore permissive but they do not support escape characters. \n Examples(with made-up labels): \n \ttype == \"webserver\" && deployment== \"prod\" \ttype in {\"frontend\", \"backend\"} \tdeployment !=\"dev\" \t! has(label_name)"type: stringserviceAccountSelector:description: ServiceAccountSelector is an optional field for an expressionused to select a pod based on service accounts.type: stringtypes:description: "Types indicates whether this policy applies to ingress,or to egress, or to both.  When not explicitly specified (and sothe value on creation is empty or nil), Calico defaults Types accordingto what Ingress and Egress rules are present in the policy.  Thedefault is: \n - [ PolicyTypeIngress ], if there are no Egress rules(including the case where there are   also no Ingress rules) \n- [ PolicyTypeEgress ], if there are Egress rules but no Ingressrules \n - [ PolicyTypeIngress, PolicyTypeEgress ], if there areboth Ingress and Egress rules. \n When the policy is read back again,Types will always be one of these values, never empty or nil."items:description: PolicyType enumerates the possible values of the PolicySpecTypes field.type: stringtype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: globalnetworksets.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: GlobalNetworkSetlistKind: GlobalNetworkSetListplural: globalnetworksetssingular: globalnetworksetscope: Clusterversions:- name: v1schema:openAPIV3Schema:description: GlobalNetworkSet contains a set of arbitrary IP sub-networks/CIDRsthat share labels to allow rules to refer to them via selectors.  The labelsof GlobalNetworkSet are not namespaced.properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: GlobalNetworkSetSpec contains the specification for a NetworkSetresource.properties:nets:description: The list of IP networks that belong to this set.items:type: stringtype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: hostendpoints.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: HostEndpointlistKind: HostEndpointListplural: hostendpointssingular: hostendpointscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: HostEndpointSpec contains the specification for a HostEndpointresource.properties:expectedIPs:description: "The expected IP addresses (IPv4 and IPv6) of the endpoint.If \"InterfaceName\" is not present, Calico will look for an interfacematching any of the IPs in the list and apply policy to that. Note:\tWhen using the selector match criteria in an ingress or egresssecurity Policy \tor Profile, Calico converts the selector intoa set of IP addresses. For host \tendpoints, the ExpectedIPs fieldis used for that purpose. (If only the interface \tname is specified,Calico does not learn the IPs of the interface for use in match\tcriteria.)"items:type: stringtype: arrayinterfaceName:description: "Either \"*\", or the name of a specific Linux interfaceto apply policy to; or empty.  \"*\" indicates that this HostEndpointgoverns all traffic to, from or through the default network namespaceof the host named by the \"Node\" field; entering and leaving thatnamespace via any interface, including those from/to non-host-networkedlocal workloads. \n If InterfaceName is not \"*\", this HostEndpointonly governs traffic that enters or leaves the host through thespecific interface named by InterfaceName, or - when InterfaceNameis empty - through the specific interface that has one of the IPsin ExpectedIPs. Therefore, when InterfaceName is empty, at leastone expected IP must be specified.  Only external interfaces (suchas \"eth0\") are supported here; it isn't possible for a HostEndpointto protect traffic through a specific local workload interface.\n Note: Only some kinds of policy are implemented for \"*\" HostEndpoints;initially just pre-DNAT policy.  Please check Calico documentationfor the latest position."type: stringnode:description: The node name identifying the Calico node instance.type: stringports:description: Ports contains the endpoint's named ports, which maybe referenced in security policy rules.items:properties:name:type: stringport:type: integerprotocol:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truerequired:- name- port- protocoltype: objecttype: arrayprofiles:description: A list of identifiers of security Profile objects thatapply to this endpoint. Each profile is applied in the order thatthey appear in this list.  Profile rules are applied after the selector-basedsecurity policy.items:type: stringtype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: ipamblocks.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: IPAMBlocklistKind: IPAMBlockListplural: ipamblockssingular: ipamblockscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: IPAMBlockSpec contains the specification for an IPAMBlockresource.properties:affinity:type: stringallocations:items:type: integer# TODO: This nullable is manually added in. We should update controller-gen# to handle []*int properly itself.nullable: truetype: arrayattributes:items:properties:handle_id:type: stringsecondary:additionalProperties:type: stringtype: objecttype: objecttype: arraycidr:type: stringdeleted:type: booleanstrictAffinity:type: booleanunallocated:items:type: integertype: arrayrequired:- allocations- attributes- cidr- strictAffinity- unallocatedtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: ipamconfigs.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: IPAMConfiglistKind: IPAMConfigListplural: ipamconfigssingular: ipamconfigscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: IPAMConfigSpec contains the specification for an IPAMConfigresource.properties:autoAllocateBlocks:type: booleanmaxBlocksPerHost:description: MaxBlocksPerHost, if non-zero, is the max number of blocksthat can be affine to each host.type: integerstrictAffinity:type: booleanrequired:- autoAllocateBlocks- strictAffinitytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: ipamhandles.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: IPAMHandlelistKind: IPAMHandleListplural: ipamhandlessingular: ipamhandlescope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: IPAMHandleSpec contains the specification for an IPAMHandleresource.properties:block:additionalProperties:type: integertype: objectdeleted:type: booleanhandleID:type: stringrequired:- block- handleIDtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: ippools.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: IPPoollistKind: IPPoolListplural: ippoolssingular: ippoolscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: IPPoolSpec contains the specification for an IPPool resource.properties:blockSize:description: The block size to use for IP address assignments fromthis pool. Defaults to 26 for IPv4 and 112 for IPv6.type: integercidr:description: The pool CIDR.type: stringdisabled:description: When disabled is true, Calico IPAM will not assign addressesfrom this pool.type: booleanipip:description: 'Deprecated: this field is only used for APIv1 backwardscompatibility. Setting this field is not allowed, this field isfor internal use only.'properties:enabled:description: When enabled is true, ipip tunneling will be usedto deliver packets to destinations within this pool.type: booleanmode:description: The IPIP mode.  This can be one of "always" or "cross-subnet".  Amode of "always" will also use IPIP tunneling for routing todestination IP addresses within this pool.  A mode of "cross-subnet"will only use IPIP tunneling when the destination node is ona different subnet to the originating node.  The default value(if not specified) is "always".type: stringtype: objectipipMode:description: Contains configuration for IPIP tunneling for this pool.If not specified, then this is defaulted to "Never" (i.e. IPIP tunnelingis disabled).type: stringnat-outgoing:description: 'Deprecated: this field is only used for APIv1 backwardscompatibility. Setting this field is not allowed, this field isfor internal use only.'type: booleannatOutgoing:description: When nat-outgoing is true, packets sent from Calico networkedcontainers in this pool to destinations outside of this pool willbe masqueraded.type: booleannodeSelector:description: Allows IPPool to allocate for a specific node by labelselector.type: stringvxlanMode:description: Contains configuration for VXLAN tunneling for this pool.If not specified, then this is defaulted to "Never" (i.e. VXLANtunneling is disabled).type: stringrequired:- cidrtype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: kubecontrollersconfigurations.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: KubeControllersConfigurationlistKind: KubeControllersConfigurationListplural: kubecontrollersconfigurationssingular: kubecontrollersconfigurationscope: Clusterversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: KubeControllersConfigurationSpec contains the values of theKubernetes controllers configuration.properties:controllers:description: Controllers enables and configures individual Kubernetescontrollersproperties:namespace:description: Namespace enables and configures the namespace controller.Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to perform reconciliationwith the Calico datastore. [Default: 5m]'type: stringtype: objectnode:description: Node enables and configures the node controller.Enabled by default, set to nil to disable.properties:hostEndpoint:description: HostEndpoint controls syncing nodes to host endpoints.Disabled by default, set to nil to disable.properties:autoCreate:description: 'AutoCreate enables automatic creation ofhost endpoints for every node. [Default: Disabled]'type: stringtype: objectreconcilerPeriod:description: 'ReconcilerPeriod is the period to perform reconciliationwith the Calico datastore. [Default: 5m]'type: stringsyncLabels:description: 'SyncLabels controls whether to copy Kubernetesnode labels to Calico nodes. [Default: Enabled]'type: stringtype: objectpolicy:description: Policy enables and configures the policy controller.Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to perform reconciliationwith the Calico datastore. [Default: 5m]'type: stringtype: objectserviceAccount:description: ServiceAccount enables and configures the serviceaccount controller. Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to perform reconciliationwith the Calico datastore. [Default: 5m]'type: stringtype: objectworkloadEndpoint:description: WorkloadEndpoint enables and configures the workloadendpoint controller. Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to perform reconciliationwith the Calico datastore. [Default: 5m]'type: stringtype: objecttype: objectetcdV3CompactionPeriod:description: 'EtcdV3CompactionPeriod is the period between etcdv3compaction requests. Set to 0 to disable. [Default: 10m]'type: stringhealthChecks:description: 'HealthChecks enables or disables support for healthchecks [Default: Enabled]'type: stringlogSeverityScreen:description: 'LogSeverityScreen is the log severity above which logsare sent to the stdout. [Default: Info]'type: stringprometheusMetricsPort:description: 'PrometheusMetricsPort is the TCP port that the Prometheusmetrics server should bind to. Set to 0 to disable. [Default: 9094]'type: integerrequired:- controllerstype: objectstatus:description: KubeControllersConfigurationStatus represents the statusof the configuration. It's useful for admins to be able to see the actualconfig that was applied, which can be modified by environment variableson the kube-controllers process.properties:environmentVars:additionalProperties:type: stringdescription: EnvironmentVars contains the environment variables onthe kube-controllers that influenced the RunningConfig.type: objectrunningConfig:description: RunningConfig contains the effective config that is runningin the kube-controllers pod, after merging the API resource withany environment variables.properties:controllers:description: Controllers enables and configures individual Kubernetescontrollersproperties:namespace:description: Namespace enables and configures the namespacecontroller. Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to performreconciliation with the Calico datastore. [Default:5m]'type: stringtype: objectnode:description: Node enables and configures the node controller.Enabled by default, set to nil to disable.properties:hostEndpoint:description: HostEndpoint controls syncing nodes to hostendpoints. Disabled by default, set to nil to disable.properties:autoCreate:description: 'AutoCreate enables automatic creationof host endpoints for every node. [Default: Disabled]'type: stringtype: objectreconcilerPeriod:description: 'ReconcilerPeriod is the period to performreconciliation with the Calico datastore. [Default:5m]'type: stringsyncLabels:description: 'SyncLabels controls whether to copy Kubernetesnode labels to Calico nodes. [Default: Enabled]'type: stringtype: objectpolicy:description: Policy enables and configures the policy controller.Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to performreconciliation with the Calico datastore. [Default:5m]'type: stringtype: objectserviceAccount:description: ServiceAccount enables and configures the serviceaccount controller. Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to performreconciliation with the Calico datastore. [Default:5m]'type: stringtype: objectworkloadEndpoint:description: WorkloadEndpoint enables and configures the workloadendpoint controller. Enabled by default, set to nil to disable.properties:reconcilerPeriod:description: 'ReconcilerPeriod is the period to performreconciliation with the Calico datastore. [Default:5m]'type: stringtype: objecttype: objectetcdV3CompactionPeriod:description: 'EtcdV3CompactionPeriod is the period between etcdv3compaction requests. Set to 0 to disable. [Default: 10m]'type: stringhealthChecks:description: 'HealthChecks enables or disables support for healthchecks [Default: Enabled]'type: stringlogSeverityScreen:description: 'LogSeverityScreen is the log severity above whichlogs are sent to the stdout. [Default: Info]'type: stringprometheusMetricsPort:description: 'PrometheusMetricsPort is the TCP port that the Prometheusmetrics server should bind to. Set to 0 to disable. [Default:9094]'type: integerrequired:- controllerstype: objecttype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: networkpolicies.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: NetworkPolicylistKind: NetworkPolicyListplural: networkpoliciessingular: networkpolicyscope: Namespacedversions:- name: v1schema:openAPIV3Schema:properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:properties:egress:description: The ordered set of egress rules.  Each rule containsa set of packet match criteria and a corresponding action to apply.items:description: "A Rule encapsulates a set of match criteria and anaction.  Both selector-based security Policy and security Profilesreference rules - separated out as a list of rules for both ingressand egress packet matching. \n Each positive match criteria hasa negated version, prefixed with \"Not\". All the match criteriawithin a rule must be satisfied for a packet to match. A singlerule can contain the positive and negative version of a matchand both must be satisfied for the rule to match."properties:action:type: stringdestination:description: Destination contains the match criteria that applyto destination entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objecthttp:description: HTTP contains match criteria that apply to HTTPrequests.properties:methods:description: Methods is an optional field that restrictsthe rule to apply only to HTTP requests that use one ofthe listed HTTP Methods (e.g. GET, PUT, etc.) Multiplemethods are OR'd together.items:type: stringtype: arraypaths:description: 'Paths is an optional field that restrictsthe rule to apply to HTTP requests that use one of thelisted HTTP Paths. Multiple paths are OR''d together.e.g: - exact: /foo - prefix: /bar NOTE: Each entry mayONLY specify either a `exact` or a `prefix` match. Thevalidator will check for it.'items:description: 'HTTPPath specifies an HTTP path to match.It may be either of the form: exact: <path>: which matchesthe path exactly or prefix: <path-prefix>: which matchesthe path prefix'properties:exact:type: stringprefix:type: stringtype: objecttype: arraytype: objecticmp:description: ICMP is an optional field that restricts the ruleto apply to a specific type and code of ICMP traffic.  Thisshould only be specified if the Protocol field is set to "ICMP"or "ICMPv6".properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectipVersion:description: IPVersion is an optional field that restricts therule to only match a specific IP version.type: integermetadata:description: Metadata contains additional information for thisruleproperties:annotations:additionalProperties:type: stringdescription: Annotations is a set of key value pairs thatgive extra information about the ruletype: objecttype: objectnotICMP:description: NotICMP is the negated version of the ICMP field.properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectnotProtocol:anyOf:- type: integer- type: stringdescription: NotProtocol is the negated version of the Protocolfield.pattern: ^.*x-kubernetes-int-or-string: trueprotocol:anyOf:- type: integer- type: stringdescription: "Protocol is an optional field that restricts therule to only apply to traffic of a specific IP protocol. Requiredif any of the EntityRules contain Ports (because ports onlyapply to certain protocols). \n Must be one of these stringvalues: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",\"UDPLite\" or an integer in the range 1-255."pattern: ^.*x-kubernetes-int-or-string: truesource:description: Source contains the match criteria that apply tosource entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objectrequired:- actiontype: objecttype: arrayingress:description: The ordered set of ingress rules.  Each rule containsa set of packet match criteria and a corresponding action to apply.items:description: "A Rule encapsulates a set of match criteria and anaction.  Both selector-based security Policy and security Profilesreference rules - separated out as a list of rules for both ingressand egress packet matching. \n Each positive match criteria hasa negated version, prefixed with \"Not\". All the match criteriawithin a rule must be satisfied for a packet to match. A singlerule can contain the positive and negative version of a matchand both must be satisfied for the rule to match."properties:action:type: stringdestination:description: Destination contains the match criteria that applyto destination entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objecthttp:description: HTTP contains match criteria that apply to HTTPrequests.properties:methods:description: Methods is an optional field that restrictsthe rule to apply only to HTTP requests that use one ofthe listed HTTP Methods (e.g. GET, PUT, etc.) Multiplemethods are OR'd together.items:type: stringtype: arraypaths:description: 'Paths is an optional field that restrictsthe rule to apply to HTTP requests that use one of thelisted HTTP Paths. Multiple paths are OR''d together.e.g: - exact: /foo - prefix: /bar NOTE: Each entry mayONLY specify either a `exact` or a `prefix` match. Thevalidator will check for it.'items:description: 'HTTPPath specifies an HTTP path to match.It may be either of the form: exact: <path>: which matchesthe path exactly or prefix: <path-prefix>: which matchesthe path prefix'properties:exact:type: stringprefix:type: stringtype: objecttype: arraytype: objecticmp:description: ICMP is an optional field that restricts the ruleto apply to a specific type and code of ICMP traffic.  Thisshould only be specified if the Protocol field is set to "ICMP"or "ICMPv6".properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectipVersion:description: IPVersion is an optional field that restricts therule to only match a specific IP version.type: integermetadata:description: Metadata contains additional information for thisruleproperties:annotations:additionalProperties:type: stringdescription: Annotations is a set of key value pairs thatgive extra information about the ruletype: objecttype: objectnotICMP:description: NotICMP is the negated version of the ICMP field.properties:code:description: Match on a specific ICMP code.  If specified,the Type value must also be specified. This is a technicallimitation imposed by the kernel's iptables firewall,which Calico uses to enforce the rule.type: integertype:description: Match on a specific ICMP type.  For examplea value of 8 refers to ICMP Echo Request (i.e. pings).type: integertype: objectnotProtocol:anyOf:- type: integer- type: stringdescription: NotProtocol is the negated version of the Protocolfield.pattern: ^.*x-kubernetes-int-or-string: trueprotocol:anyOf:- type: integer- type: stringdescription: "Protocol is an optional field that restricts therule to only apply to traffic of a specific IP protocol. Requiredif any of the EntityRules contain Ports (because ports onlyapply to certain protocols). \n Must be one of these stringvalues: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",\"UDPLite\" or an integer in the range 1-255."pattern: ^.*x-kubernetes-int-or-string: truesource:description: Source contains the match criteria that apply tosource entity.properties:namespaceSelector:description: "NamespaceSelector is an optional field thatcontains a selector expression. Only traffic that originatesfrom (or terminates at) endpoints within the selectednamespaces will be matched. When both NamespaceSelectorand Selector are defined on the same rule, then only workloadendpoints that are matched by both selectors will be selectedby the rule. \n For NetworkPolicy, an empty NamespaceSelectorimplies that the Selector is limited to selecting onlyworkload endpoints in the same namespace as the NetworkPolicy.\n For NetworkPolicy, `global()` NamespaceSelector impliesthat the Selector is limited to selecting only GlobalNetworkSetor HostEndpoint. \n For GlobalNetworkPolicy, an emptyNamespaceSelector implies the Selector applies to workloadendpoints across all namespaces."type: stringnets:description: Nets is an optional field that restricts therule to only apply to traffic that originates from (orterminates at) IP addresses in any of the given subnets.items:type: stringtype: arraynotNets:description: NotNets is the negated version of the Netsfield.items:type: stringtype: arraynotPorts:description: NotPorts is the negated version of the Portsfield. Since only some protocols have ports, if any portsare specified it requires the Protocol match in the Ruleto be set to "TCP" or "UDP".items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arraynotSelector:description: NotSelector is the negated version of the Selectorfield.  See Selector field for subtleties with negatedselectors.type: stringports:description: "Ports is an optional field that restrictsthe rule to only apply to traffic that has a source (destination)port that matches one of these ranges/values. This valueis a list of integers or strings that represent rangesof ports. \n Since only some protocols have ports, ifany ports are specified it requires the Protocol matchin the Rule to be set to \"TCP\" or \"UDP\"."items:anyOf:- type: integer- type: stringpattern: ^.*x-kubernetes-int-or-string: truetype: arrayselector:description: "Selector is an optional field that containsa selector expression (see Policy for sample syntax).\ Only traffic that originates from (terminates at) endpointsmatching the selector will be matched. \n Note that: inaddition to the negated version of the Selector (see NotSelectorbelow), the selector expression syntax itself supportsnegation.  The two types of negation are subtly different.One negates the set of matched endpoints, the other negatesthe whole match: \n \tSelector = \"!has(my_label)\" matchespackets that are from other Calico-controlled \tendpointsthat do not have the label \"my_label\". \n \tNotSelector= \"has(my_label)\" matches packets that are not fromCalico-controlled \tendpoints that do have the label \"my_label\".\n The effect is that the latter will accept packets fromnon-Calico sources whereas the former is limited to packetsfrom Calico-controlled endpoints."type: stringserviceAccounts:description: ServiceAccounts is an optional field that restrictsthe rule to only apply to traffic that originates from(or terminates at) a pod running as a matching serviceaccount.properties:names:description: Names is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount whose name is in the list.items:type: stringtype: arrayselector:description: Selector is an optional field that restrictsthe rule to only apply to traffic that originatesfrom (or terminates at) a pod running as a serviceaccount that matches the given label selector. Ifboth Names and Selector are specified then they areAND'ed.type: stringtype: objecttype: objectrequired:- actiontype: objecttype: arrayorder:description: Order is an optional field that specifies the order inwhich the policy is applied. Policies with higher "order" are appliedafter those with lower order.  If the order is omitted, it may beconsidered to be "infinite" - i.e. the policy will be applied last.  Policieswith identical order will be applied in alphanumerical order basedon the Policy "Name".type: numberselector:description: "The selector is an expression used to pick pick outthe endpoints that the policy should be applied to. \n Selectorexpressions follow this syntax: \n \tlabel == \"string_literal\"\ ->  comparison, e.g. my_label == \"foo bar\" \tlabel != \"string_literal\"\  ->  not equal; also matches if label is not present \tlabel in{ \"a\", \"b\", \"c\", ... }  ->  true if the value of label X isone of \"a\", \"b\", \"c\" \tlabel not in { \"a\", \"b\", \"c\",... }  ->  true if the value of label X is not one of \"a\", \"b\",\"c\" \thas(label_name)  -> True if that label is present \t! expr-> negation of expr \texpr && expr  -> Short-circuit and \texpr|| expr  -> Short-circuit or \t( expr ) -> parens for grouping \tall()or the empty selector -> matches all endpoints. \n Label names areallowed to contain alphanumerics, -, _ and /. String literals aremore permissive but they do not support escape characters. \n Examples(with made-up labels): \n \ttype == \"webserver\" && deployment== \"prod\" \ttype in {\"frontend\", \"backend\"} \tdeployment !=\"dev\" \t! has(label_name)"type: stringserviceAccountSelector:description: ServiceAccountSelector is an optional field for an expressionused to select a pod based on service accounts.type: stringtypes:description: "Types indicates whether this policy applies to ingress,or to egress, or to both.  When not explicitly specified (and sothe value on creation is empty or nil), Calico defaults Types accordingto what Ingress and Egress are present in the policy.  The defaultis: \n - [ PolicyTypeIngress ], if there are no Egress rules (includingthe case where there are   also no Ingress rules) \n - [ PolicyTypeEgress], if there are Egress rules but no Ingress rules \n - [ PolicyTypeIngress,PolicyTypeEgress ], if there are both Ingress and Egress rules.\n When the policy is read back again, Types will always be oneof these values, never empty or nil."items:description: PolicyType enumerates the possible values of the PolicySpecTypes field.type: stringtype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:name: networksets.crd.projectcalico.org
spec:group: crd.projectcalico.orgnames:kind: NetworkSetlistKind: NetworkSetListplural: networksetssingular: networksetscope: Namespacedversions:- name: v1schema:openAPIV3Schema:description: NetworkSet is the Namespaced-equivalent of the GlobalNetworkSet.properties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: NetworkSetSpec contains the specification for a NetworkSetresource.properties:nets:description: The list of IP networks that belong to this set.items:type: stringtype: arraytype: objecttype: objectserved: truestorage: true
status:acceptedNames:kind: ""plural: ""conditions: []storedVersions: []---
---
# Source: calico/templates/calico-kube-controllers-rbac.yaml# Include a clusterrole for the kube-controllers component,
# and bind it to the calico-kube-controllers serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: calico-kube-controllers
rules:# Nodes are watched to monitor for deletions.- apiGroups: [""]resources:- nodesverbs:- watch- list- get# Pods are queried to check for existence.- apiGroups: [""]resources:- podsverbs:- get# IPAM resources are manipulated when nodes are deleted.- apiGroups: ["crd.projectcalico.org"]resources:- ippoolsverbs:- list- apiGroups: ["crd.projectcalico.org"]resources:- blockaffinities- ipamblocks- ipamhandlesverbs:- get- list- create- update- delete- watch# kube-controllers manages hostendpoints.- apiGroups: ["crd.projectcalico.org"]resources:- hostendpointsverbs:- get- list- create- update- delete# Needs access to update clusterinformations.- apiGroups: ["crd.projectcalico.org"]resources:- clusterinformationsverbs:- get- create- update# KubeControllersConfiguration is where it gets its config- apiGroups: ["crd.projectcalico.org"]resources:- kubecontrollersconfigurationsverbs:# read its own config- get# create a default if none exists- create# update status- update# watch for changes- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: calico-kube-controllers
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: calico-kube-controllers
subjects:
- kind: ServiceAccountname: calico-kube-controllersnamespace: kube-system
------
# Source: calico/templates/calico-node-rbac.yaml
# Include a clusterrole for the calico-node DaemonSet,
# and bind it to the calico-node serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: calico-node
rules:# The CNI plugin needs to get pods, nodes, and namespaces.- apiGroups: [""]resources:- pods- nodes- namespacesverbs:- get- apiGroups: [""]resources:- endpoints- servicesverbs:# Used to discover service IPs for advertisement.- watch- list# Used to discover Typhas.- get# Pod CIDR auto-detection on kubeadm needs access to config maps.- apiGroups: [""]resources:- configmapsverbs:- get- apiGroups: [""]resources:- nodes/statusverbs:# Needed for clearing NodeNetworkUnavailable flag.- patch# Calico stores some configuration information in node annotations.- update# Watch for changes to Kubernetes NetworkPolicies.- apiGroups: ["networking.k8s.io"]resources:- networkpoliciesverbs:- watch- list# Used by Calico for policy information.- apiGroups: [""]resources:- pods- namespaces- serviceaccountsverbs:- list- watch# The CNI plugin patches pods/status.- apiGroups: [""]resources:- pods/statusverbs:- patch# Calico monitors various CRDs for config.- apiGroups: ["crd.projectcalico.org"]resources:- globalfelixconfigs- felixconfigurations- bgppeers- globalbgpconfigs- bgpconfigurations- ippools- ipamblocks- globalnetworkpolicies- globalnetworksets- networkpolicies- networksets- clusterinformations- hostendpoints- blockaffinitiesverbs:- get- list- watch# Calico must create and update some CRDs on startup.- apiGroups: ["crd.projectcalico.org"]resources:- ippools- felixconfigurations- clusterinformationsverbs:- create- update# Calico stores some configuration information on the node.- apiGroups: [""]resources:- nodesverbs:- get- list- watch# These permissions are only required for upgrade from v2.6, and can# be removed after upgrade or on fresh installations.- apiGroups: ["crd.projectcalico.org"]resources:- bgpconfigurations- bgppeersverbs:- create- update# These permissions are required for Calico CNI to perform IPAM allocations.- apiGroups: ["crd.projectcalico.org"]resources:- blockaffinities- ipamblocks- ipamhandlesverbs:- get- list- create- update- delete- apiGroups: ["crd.projectcalico.org"]resources:- ipamconfigsverbs:- get# Block affinities must also be watchable by confd for route aggregation.- apiGroups: ["crd.projectcalico.org"]resources:- blockaffinitiesverbs:- watch# The Calico IPAM migration needs to get daemonsets. These permissions can be# removed if not upgrading from an installation using host-local IPAM.- apiGroups: ["apps"]resources:- daemonsetsverbs:- get---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: calico-node
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: calico-node
subjects:
- kind: ServiceAccountname: calico-nodenamespace: kube-system---
# Source: calico/templates/calico-node.yaml
# This manifest installs the calico-node container, as well
# as the CNI plugins and network config on
# each master and worker node in a Kubernetes cluster.
kind: DaemonSet
apiVersion: apps/v1
metadata:name: calico-nodenamespace: kube-systemlabels:k8s-app: calico-node
spec:selector:matchLabels:k8s-app: calico-nodeupdateStrategy:type: RollingUpdaterollingUpdate:maxUnavailable: 1template:metadata:labels:k8s-app: calico-nodespec:nodeSelector:kubernetes.io/os: linuxhostNetwork: truetolerations:# Make sure calico-node gets scheduled on all nodes.- effect: NoScheduleoperator: Exists# Mark the pod as a critical add-on for rescheduling.- key: CriticalAddonsOnlyoperator: Exists- effect: NoExecuteoperator: ExistsserviceAccountName: calico-node# Minimize downtime during a rolling upgrade or deletion; tell Kubernetes to do a "force# deletion": https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.terminationGracePeriodSeconds: 0priorityClassName: system-node-criticalinitContainers:# This container performs upgrade from host-local IPAM to calico-ipam.# It can be deleted if this is a fresh installation, or if you have already# upgraded to use calico-ipam.- name: upgrade-ipamimage: docker.io/calico/cni:v3.18.0command: ["/opt/cni/bin/calico-ipam", "-upgrade"]envFrom:- configMapRef:# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.name: kubernetes-services-endpointoptional: trueenv:- name: KUBERNETES_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: CALICO_NETWORKING_BACKENDvalueFrom:configMapKeyRef:name: calico-configkey: calico_backendvolumeMounts:- mountPath: /var/lib/cni/networksname: host-local-net-dir- mountPath: /host/opt/cni/binname: cni-bin-dirsecurityContext:privileged: true# This container installs the CNI binaries# and CNI network config file on each node.- name: install-cniimage: docker.io/calico/cni:v3.18.0command: ["/opt/cni/bin/install"]envFrom:- configMapRef:# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.name: kubernetes-services-endpointoptional: trueenv:# Name of the CNI config file to create.- name: CNI_CONF_NAMEvalue: "10-calico.conflist"# The CNI network config to install on each node.- name: CNI_NETWORK_CONFIGvalueFrom:configMapKeyRef:name: calico-configkey: cni_network_config# Set the hostname based on the k8s node name.- name: KUBERNETES_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName# CNI MTU Config variable- name: CNI_MTUvalueFrom:configMapKeyRef:name: calico-configkey: veth_mtu# Prevents the container from sleeping forever.- name: SLEEPvalue: "false"volumeMounts:- mountPath: /host/opt/cni/binname: cni-bin-dir- mountPath: /host/etc/cni/net.dname: cni-net-dirsecurityContext:privileged: true# Adds a Flex Volume Driver that creates a per-pod Unix Domain Socket to allow Dikastes# to communicate with Felix over the Policy Sync API.- name: flexvol-driverimage: docker.io/calico/pod2daemon-flexvol:v3.18.0volumeMounts:- name: flexvol-driver-hostmountPath: /host/driversecurityContext:privileged: truecontainers:# Runs calico-node container on each Kubernetes node. This# container programs network policy and routes on each# host.- name: calico-nodeimage: docker.io/calico/node:v3.18.0envFrom:- configMapRef:# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.name: kubernetes-services-endpointoptional: trueenv:# Use Kubernetes API as the backing datastore.- name: DATASTORE_TYPEvalue: "kubernetes"# Wait for the datastore.- name: WAIT_FOR_DATASTOREvalue: "true"# Set based on the k8s node name.- name: NODENAMEvalueFrom:fieldRef:fieldPath: spec.nodeName# Choose the backend to use.- name: CALICO_NETWORKING_BACKENDvalueFrom:configMapKeyRef:name: calico-configkey: calico_backend# Cluster type to identify the deployment type- name: CLUSTER_TYPEvalue: "k8s,bgp"# Auto-detect the BGP IP address.- name: IPvalue: "autodetect"# Enable IPIP- name: CALICO_IPV4POOL_IPIPvalue: "Always"# Enable or Disable VXLAN on the default IP pool.- name: CALICO_IPV4POOL_VXLANvalue: "Never"# Set MTU for tunnel device used if ipip is enabled- name: FELIX_IPINIPMTUvalueFrom:configMapKeyRef:name: calico-configkey: veth_mtu# Set MTU for the VXLAN tunnel device.- name: FELIX_VXLANMTUvalueFrom:configMapKeyRef:name: calico-configkey: veth_mtu# Set MTU for the Wireguard tunnel device.- name: FELIX_WIREGUARDMTUvalueFrom:configMapKeyRef:name: calico-configkey: veth_mtu# The default IPv4 pool to create on startup if none exists. Pod IPs will be# chosen from this range. Changing this value after installation will have# no effect. This should fall within `--cluster-cidr`.# - name: CALICO_IPV4POOL_CIDR#   value: "192.168.0.0/16"# Disable file logging so `kubectl logs` works.- name: CALICO_DISABLE_FILE_LOGGINGvalue: "true"# Set Felix endpoint to host default action to ACCEPT.- name: FELIX_DEFAULTENDPOINTTOHOSTACTIONvalue: "ACCEPT"# Disable IPv6 on Kubernetes.- name: FELIX_IPV6SUPPORTvalue: "false"# Set Felix logging to "info"- name: FELIX_LOGSEVERITYSCREENvalue: "info"- name: FELIX_HEALTHENABLEDvalue: "true"securityContext:privileged: trueresources:requests:cpu: 250mlivenessProbe:exec:command:- /bin/calico-node- -felix-live- -bird-liveperiodSeconds: 10initialDelaySeconds: 10failureThreshold: 6readinessProbe:exec:command:- /bin/calico-node- -felix-ready- -bird-readyperiodSeconds: 10volumeMounts:- mountPath: /lib/modulesname: lib-modulesreadOnly: true- mountPath: /run/xtables.lockname: xtables-lockreadOnly: false- mountPath: /var/run/caliconame: var-run-calicoreadOnly: false- mountPath: /var/lib/caliconame: var-lib-calicoreadOnly: false- name: policysyncmountPath: /var/run/nodeagent# For eBPF mode, we need to be able to mount the BPF filesystem at /sys/fs/bpf so we mount in the# parent directory.- name: sysfsmountPath: /sys/fs/# Bidirectional means that, if we mount the BPF filesystem at /sys/fs/bpf it will propagate to the host.# If the host is known to mount that filesystem already then Bidirectional can be omitted.mountPropagation: Bidirectional- name: cni-log-dirmountPath: /var/log/calico/cnireadOnly: truevolumes:# Used by calico-node.- name: lib-moduleshostPath:path: /lib/modules- name: var-run-calicohostPath:path: /var/run/calico- name: var-lib-calicohostPath:path: /var/lib/calico- name: xtables-lockhostPath:path: /run/xtables.locktype: FileOrCreate- name: sysfshostPath:path: /sys/fs/type: DirectoryOrCreate# Used to install CNI.- name: cni-bin-dirhostPath:path: /opt/cni/bin- name: cni-net-dirhostPath:path: /etc/cni/net.d# Used to access CNI logs.- name: cni-log-dirhostPath:path: /var/log/calico/cni# Mount in the directory for host-local IPAM allocations. This is# used when upgrading from host-local to calico-ipam, and can be removed# if not using the upgrade-ipam init container.- name: host-local-net-dirhostPath:path: /var/lib/cni/networks# Used to create per-pod Unix Domain Sockets- name: policysynchostPath:type: DirectoryOrCreatepath: /var/run/nodeagent# Used to install Flex Volume Driver- name: flexvol-driver-hosthostPath:type: DirectoryOrCreatepath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds
---apiVersion: v1
kind: ServiceAccount
metadata:name: calico-nodenamespace: kube-system---
# Source: calico/templates/calico-kube-controllers.yaml
# See https://github.com/projectcalico/kube-controllers
apiVersion: apps/v1
kind: Deployment
metadata:name: calico-kube-controllersnamespace: kube-systemlabels:k8s-app: calico-kube-controllers
spec:# The controllers can only have a single active instance.replicas: 1selector:matchLabels:k8s-app: calico-kube-controllersstrategy:type: Recreatetemplate:metadata:name: calico-kube-controllersnamespace: kube-systemlabels:k8s-app: calico-kube-controllersspec:nodeSelector:kubernetes.io/os: linuxtolerations:# Mark the pod as a critical add-on for rescheduling.- key: CriticalAddonsOnlyoperator: Exists- key: node-role.kubernetes.io/mastereffect: NoScheduleserviceAccountName: calico-kube-controllerspriorityClassName: system-cluster-criticalcontainers:- name: calico-kube-controllersimage: docker.io/calico/kube-controllers:v3.18.0env:# Choose which controllers to run.- name: ENABLED_CONTROLLERSvalue: node- name: DATASTORE_TYPEvalue: kubernetesreadinessProbe:exec:command:- /usr/bin/check-status- -r---apiVersion: v1
kind: ServiceAccount
metadata:name: calico-kube-controllersnamespace: kube-system---# This manifest creates a Pod Disruption Budget for Controller to allow K8s Cluster Autoscaler to evictapiVersion: policy/v1
kind: PodDisruptionBudget
metadata:name: calico-kube-controllersnamespace: kube-systemlabels:k8s-app: calico-kube-controllers
spec:maxUnavailable: 1selector:matchLabels:k8s-app: calico-kube-controllers---
# Source: calico/templates/calico-etcd-secrets.yaml---
# Source: calico/templates/calico-typha.yaml---
# Source: calico/templates/configure-canal.yaml

5）将工作节点添加进集群

在master1节点上执行命令，生成token
[root@master1 ~]# kubeadm token create --print-join-command
kubeadm join 192.168.2.217:6443 --token c1dy8g.nqobfc39ycv3uygo     --discovery-token-ca-cert-hash sha256:fd0885ebdcc8dac61639f1e429cefa1cd7e5fcc4ab22dc3e7934247b1344af02
# 生成的token有效期为24小时
# 每次添加工作节点的时候，都需要执行该命令在工作节点上，执行生成的命令并添加参数（生成两个，在两个工作节点上分别执行）
# --ignore-preflight-errors=SystemVerification
[root@node1 ~]# kubeadm join 192.168.2.217:6443 --token c1dy8g.nqobfc39ycv3uygo     --discovery-token-ca-cert-hash sha256:fd0885ebdcc8dac61639f1e429cefa1cd7e5fcc4ab22dc3e7934247b1344af02 --ignore-preflight-errors=SystemVerification[root@node2 ~]# kubeadm join 192.168.2.217:6443 --token alhlos.h5x5d9y2ddix4dtw     --discovery-token-ca-cert-hash sha256:fd0885ebdcc8dac61639f1e429cefa1cd7e5fcc4ab22dc3e7934247b1344af02 --ignore-preflight-errors=SystemVerification
# 注意：
# 若已经执行过kubeadm join 172.16.32.128:6443 --token xxx命令，再次执行的时候，将无法添加，需要通过命令kubeadm reset重置k8s将工作节点添加进集群后，查询k8s集群状态
[root@master1 ~]# kubectl get nodes
NAME      STATUS   ROLES                  AGE   VERSION
master1   Ready    control-plane,master   77m   v1.20.6
node1     Ready    <none>                 66s   v1.20.6
node2     Ready    <none>                 61s   v1.20.6
# 工作节点的ROLES默认显示为none，若想显示为其他，可以通过在master节点指定命令，对工作节点生成标签
# 打标签
[root@master1 ~]# kubectl label node node1 node-role.kubernetes.io/node1=node1
# 删除标签
[root@master1 ~]# kubectl label node node1 node-role.kubernetes.io/node1-

将node节点加入集群
集群状态查看

更换标签后查看

至此，k8s 1.20.6版本单master多node集群已经搭建完毕。

5. 测试

1）测试k8s创建pod是否可以正常访问网络

# 将busybox-1-28.tar.gz上传至三台服务器
解压busybox-1-28.tar.gz，使每个节点上都有busybox:1.28镜像
[root@master1 ~]# docker load -i busybox-1-28.tar.gz
# master1节点上调用镜像运行pod
[root@master1 ~]# kubectl run busybox --image busybox:1.28 --image-pull-policy=IfNotPresent --restart=Never --rm -it busybox -- sh# 新开一个窗口，查询pod运行在哪个node节点上
[root@master1 ~]# kubectl get pods -o wide
NAME      READY   STATUS    RESTARTS   AGE     IP             NODE    NOMINATED NODE   READINESS GATES
busybox   1/1     Running   0          3m43s   10.244.104.5   node2   <none>           <none># 切换进busybox的pod中
# 测试网络是否正常
/ # ping www.baidu.com
PING www.baidu.com (183.2.172.185): 56 data bytes
64 bytes from 183.2.172.185: seq=0 ttl=127 time=40.756 ms
64 bytes from 183.2.172.185: seq=1 ttl=127 time=29.152 ms
# 能够正常解析，说明calico插件安装正常

测试网络

2）测试coredns解析是否正常

# 切换进busybox的pod中
[root@master1 ~]# kubectl run busybox --image busybox:1.28 --image-pull-policy=IfNotPresent --restart=Never --rm -it busybox -- sh
# 测试coredns解析是否正常
/ # nslookup kubernetes.default.svc.cluster.local
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.localName:      kubernetes.default.svc.cluster.local
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
# 显示为初始化k8s时，kubeadm.yaml配置文件中的service网段

6.kubectl命令

[root@master1 ~]# kubectl --help
kubectl controls the Kubernetes cluster manager.Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/Basic Commands (Beginner):create        Create a resource from a file or from stdin.expose        Take a replication controller, service, deployment or pod and expose it as a new
Kubernetes Servicerun           Run a particular image on the clusterset           Set specific features on objectsBasic Commands (Intermediate):explain       Documentation of resourcesget           Display one or many resourcesedit          Edit a resource on the serverdelete        Delete resources by filenames, stdin, resources and names, or by resources and label
selectorDeploy Commands:rollout       Manage the rollout of a resourcescale         Set a new size for a Deployment, ReplicaSet or Replication Controllerautoscale     Auto-scale a Deployment, ReplicaSet, or ReplicationControllerCluster Management Commands:certificate   Modify certificate resources.cluster-info  Display cluster infotop           Display Resource (CPU/Memory/Storage) usage.cordon        Mark node as unschedulableuncordon      Mark node as schedulabledrain         Drain node in preparation for maintenancetaint         Update the taints on one or more nodesTroubleshooting and Debugging Commands:describe      Show details of a specific resource or group of resourceslogs          Print the logs for a container in a podattach        Attach to a running containerexec          Execute a command in a containerport-forward  Forward one or more local ports to a podproxy         Run a proxy to the Kubernetes API servercp            Copy files and directories to and from containers.auth          Inspect authorizationdebug         Create debugging sessions for troubleshooting workloads and nodesAdvanced Commands:diff          Diff live version against would-be applied versionapply         Apply a configuration to a resource by filename or stdinpatch         Update field(s) of a resourcereplace       Replace a resource by filename or stdinwait          Experimental: Wait for a specific condition on one or many resources.kustomize     Build a kustomization target from a directory or a remote url.Settings Commands:label         Update the labels on a resourceannotate      Update the annotations on a resourcecompletion    Output shell completion code for the specified shell (bash or zsh)Other Commands:api-resources Print the supported API resources on the serverapi-versions  Print the supported API versions on the server, in the form of "group/version"config        Modify kubeconfig filesplugin        Provides utilities for interacting with plugins.version       Print the client and server version informationUsage:kubectl [flags] [options]