K8s-MySQL主从集群

K8s-MySQL主从集群

引言

该案例代码均可从https://github.com/WeiXiao-Hyy/k8s_example 获取，欢迎Star！

需求

1. 一个“主从复制”的MySQL集群
2. 有一个主节点Master
3. 有多个从节点Slave
4. 从节点需要能水平扩展
5. 所以写操作只能在主节点上执行
6. 读操作可以在所有节点上执行

xtrabackup 案例

xtrabackup原理和实战案例参考xtrabackup docker实战

MySQL conf文件配置

由于statefulset生成的pod命名为mysql-0,mysql-1依次递增，则可以取末尾的idx作为MySQL service-id.cnf文件中的mysqld属性。

spec:initContainers:- name: init-mysqlimage: mysql:5.7command:- bash- "-c"- |set -ex# Generate mysql server-id from pod ordinal index.[[ `hostname` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}echo [mysqld] > /mnt/conf.d/server-id.cnf# Add an offset to avoid reserved server-id=0 value.echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf# Copy appropriate conf.d files from config-map to emptyDir.if [[ $ordinal -eq 0 ]]; thencp /mnt/config-map/master.cnf /mnt/conf.d/elsecp /mnt/config-map/slave.cnf /mnt/conf.d/fivolumeMounts:- name: confmountPath: /mnt/conf.d- name: config-mapmountPath: /mnt/config-map

Headless Service创建

# Headless service for stable DNS entries of StatefulSet members.
apiVersion: v1
kind: Service
metadata:name: mysqllabels:app: mysql
spec:ports:- name: mysqlport: 3306clusterIP: Noneselector:app: mysql
---
# Client service for connecting to any MySQL instance for reads.
# For writes, you must instead connect to the master: mysql-0.mysql.
apiVersion: v1
kind: Service
metadata:name: mysql-readlabels:app: mysql
spec:ports:- name: mysqlport: 3306selector:app: mysql

创建HeadLess-Service，并且保证主节点的DNS为mysql-0.mysql，其他从节点为mysql-read。

从节点复制同步数据

spec:initContainers:- name: clone-mysqlimage: gcr.io/google-samples/xtrabackup:1.0command:- bash- "-c"- |set -ex# Skip the clone if data already exists.[[ -d /var/lib/mysql/mysql ]] && exit 0# Skip the clone on master (ordinal index 0).[[ `hostname` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}[[ $ordinal -eq 0 ]] && exit 0# Clone data from previous peer.ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql# Prepare the backup.xtrabackup --prepare --target-dir=/var/lib/mysqlvolumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.d

流程图如下：

从节点启动如何恢复数据

前提

1. xtrabackup_salve_info文件存在则说明该节点为从节点
2. xtrabackup_binlog_info 文件存在则说明该节点为主节点
3. 从节点恢复数据启动主要需要获取两个字段，MASTER_LOG_FILE和MASTER_LOG_POS

- name: xtrabackupimage: gcr.io/google-samples/xtrabackup:1.0ports:- name: xtrabackupcontainerPort: 3307command:- bash- "-c"- |set -excd /var/lib/mysql# Determine binlog position of cloned data, if any.if [[ -f xtrabackup_slave_info ]]; then# XtraBackup already generated a partial "CHANGE MASTER TO" query# because we're cloning from an existing slave.mv xtrabackup_slave_info change_master_to.sql.in# Ignore xtrabackup_binlog_info in this case (it's useless).rm -f xtrabackup_binlog_infoelif [[ -f xtrabackup_binlog_info ]]; then# We're cloning directly from master. Parse binlog position.[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1rm xtrabackup_binlog_infoecho "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.infi# Check if we need to complete a clone by starting replication.if [[ -f change_master_to.sql.in ]]; thenecho "Waiting for mysqld to be ready (accepting connections)"until mysql -h 127.0.0.1 -uroot -p$MYSQL_ROOT_PASSWORD -e "SELECT 1"; do sleep 1; doneecho "Initializing replication from clone position"# In case of container restart, attempt this at-most-once.mv change_master_to.sql.in change_master_to.sql.origmysql -h 127.0.0.1 -uroot -p$MYSQL_ROOT_PASSWORD <<EOF$(<change_master_to.sql.orig),MASTER_HOST='mysql-0.mysql',MASTER_USER='root',MASTER_PASSWORD=$MYSQL_ROOT_PASSWORD,MASTER_CONNECT_RETRY=10;START SLAVE;EOFfi# Start a server to send backups when requested by peers.exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 100mmemory: 100Mi

流程图如下

等待MySQL服务启动操作：until mysql -h 127.0.0.1 -uroot -pPASSWORD -e “Select 1”; do sleep 1; done

结果测试

K8s执行效果

MySQL客户端执行效果

主节点mysql-0.mysql写入；从节点mysql-read读取。