最新版Ceph( Reef版本)块存储简单对接k8s

当前ceph

你的ceph集群上执行

1.创建名为k8s-rbd 的存储池
ceph osd pool create k8s-rbd  64 642.初始化
rbd pool init k8s-rbd3 创建k8s访问块设备的认证用户
ceph auth get-or-create client.kubernetes mon 'profile rbd' osd 'profile rbd pool=k8s-rbd'

部署 ceph-rbd-csi

cat <<EOF > csi-config-map.yaml
---
apiVersion: v1
kind: ConfigMap
data:config.json: |-[{"clusterID": "76e8c1f6-0baa-11ef-ae7a-bc24115bcbdc","monitors": ["192.168.0.26:6789","192.168.0.27:6789","192.168.0.28:6789"]}]
metadata:name: ceph-csi-config
EOF###说明###
"clusterID": "76e8c1f6-0baa-11ef-ae7a-bc24115bcbdc"  通过 ceph mon dump 中的 fsid 获取

创建名为“ceph-csi-encryption-kms-config”的ConfigMap，因没有使用KMS，配置内容为空（不能省略，否则后面容器启动会报错

 cat <<EOF > csi-kms-config-map.yaml
---
apiVersion: v1
kind: ConfigMap
data:config.json: |-{}
metadata:name: ceph-csi-encryption-kms-config
EOF

最新版本的Ceph - CSI还需要另一个ConfigMap对象来定义Ceph配置，以添加到CSI容器内的Ceph .conf文件中:

 cat <<EOF > ceph-config-map.yaml
---
apiVersion: v1
kind: ConfigMap
data:ceph.conf: |[global]auth_cluster_required = cephxauth_service_required = cephxauth_client_required = cephxkeyring: |
metadata:name: ceph-config
EOF

创建 secret对象，存储访问ceph的 key

cat <<EOF > csi-rbd-secret.yaml
---
apiVersion: v1
kind: Secret
metadata:name: csi-rbd-secretnamespace: default
stringData:userID: kubernetesuserKey: AQAnEjpmMbBeOhAA8SDvxVDYCGcOWtiQjhS+sg==
EOF###说明###
userID 为上面在 ceph 集群中创建的 kubernetes
userKey 通过下面获取，这里无需通过 base64 加密，切记！# ceph auth get client.kubernetes
[client.kubernetes]key = AQAnEjpmMbBeOhAA8SDvxVDYCGcOWtiQjhS+sg==caps mon = "profile rbd"caps osd = "profile rbd pool=k8s-rbd"

再部署官网提供的yaml文件

$ kubectl apply -f https://raw.githubusercontent.com/ceph/ceph-csi/master/deploy/rbd/kubernetes/csi-provisioner-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/ceph/ceph-csi/master/deploy/rbd/kubernetes/csi-nodeplugin-rbac.yaml$ wget https://raw.githubusercontent.com/ceph/ceph-csi/master/deploy/rbd/kubernetes/csi-rbdplugin-provisioner.yaml
$ kubectl apply -f csi-rbdplugin-provisioner.yaml$ wget https://raw.githubusercontent.com/ceph/ceph-csi/master/deploy/rbd/kubernetes/csi-rbdplugin.yaml
$ kubectl apply -f csi-rbdplugin.yaml

创建SC

 cat <<EOF > csi-rbd-sc.yaml
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: csi-rbd-sc
provisioner: rbd.csi.ceph.com
parameters:clusterID: 76e8c1f6-0baa-11ef-ae7a-bc24115bcbdcpool: k8s-rbd <<<<<<<<<<<<<<<<<---------------（注意你的池）imageFeatures: layeringcsi.storage.k8s.io/provisioner-secret-name: csi-rbd-secretcsi.storage.k8s.io/provisioner-secret-namespace: defaultcsi.storage.k8s.io/controller-expand-secret-name: csi-rbd-secretcsi.storage.k8s.io/controller-expand-secret-namespace: defaultcsi.storage.k8s.io/node-stage-secret-name: csi-rbd-secretcsi.storage.k8s.io/node-stage-secret-namespace: default
reclaimPolicy: Delete
allowVolumeExpansion: true
mountOptions:- discard
EOF

####创建一个pvc 试一试

cat <<EOF > raw-block-pvc.yaml
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: raw-block-pvc
spec:accessModes:- ReadWriteOncevolumeMode: Blockresources:requests:storage: 1GistorageClassName: csi-rbd-sc
EOF

helm在部署一个redis