基于helm的方式在k8s集群中部署gitlab - 备份恢复（二）

接上一篇 基于helm的方式在k8s集群中部署gitlab - 部署（一），本篇重点介绍在k8s集群中备份gitlab的数据，并在虚拟机上部署相同版本的gitlab，然后将备份的数据进行还原恢复

1. 备份

由于使用的是minio对象存储，然后gitlab的ingress 使用的是nodeport的方式，而默认的minio的配置是域名配置，由于gitlab在14.9以后使用的tootlbox来进行备份的，因此需要更改toolbox引入minio的configmap配置文件。

但是每次upgrade后会覆盖cm文件，因为后面依然需要修改cm，或者使用外置minio。

# 查看tootbox的cm
kubectl get cm -n jihulab
# 编辑tootbox的cm
kubectl edit cm -n jihulab gitlab-toolbox

 ......if [ ! -f "/${secret_dir}/objectstorage/.s3cfg" ]; thencat <<EOF > "/${secret_dir}/.s3cfg"[default]access_key = $(cat /init-secrets/minio/accesskey)secret_key = $(cat /init-secrets/minio/secretkey)bucket_location = us-east-1host_base = minio.bdeet.top:31501  #修改为nodeport的端口host_bucket = minio.bdeet.top:31501/%(bucket)   #修改为nodeport的端口default_mime_type = binary/octet-streamenable_multipart = Truemultipart_max_chunks = 10000multipart_chunk_size_mb = 128recursive = Truerecv_chunk = 65536send_chunk = 65536server_side_encryption = Falsesignature_v2 = Truesocket_timeout = 300use_mime_magic = Falseverbosity = WARNINGwebsite_endpoint = https://minio.bdeet.top:31501  #修改为nodeport的端口EOF
...
...

然后delete掉toolbox的pod，执行备份

# 删除pod
kubectl delete pod -n jihulab gitlab-toolbox-7b796575d8-gplhc
# 备份
kubectl exec -it gitlab-toolbox-7b796575d8-7q8mh -n jihulab -- backup-utility

minio上备份的gitlab数据

2. 恢复到虚拟机上的gitlab

此处跳过安装gitlab到虚拟机上的操作，默认已经安装相同版本的gitlab服务

2.1 将minio上的备份文件（gitlab-backups）下载下来

2.2 将文件放在相同版本gitlab实例的backup目录下，然后执行restore命令

参考gitlab恢复

cd /var/opt/gitlab/backups
sudo gitlab-backup restore

2.3 配置gitlab实例的对象存储

编辑gitlab.rb文件

...
...
gitlab_rails['object_store']['enabled'] = true
gitlab_rails['object_store']['proxy_download'] = true
gitlab_rails['object_store']['connection'] = {'provider' => 'AWS','region' => 'us-east-1','path_style' => 'true','host' => 'minio.bdeet.top:30476','endpoint' => 'https://minio.bdeet.top:30476','aws_access_key_id' => 'NHsiBL6v589G4h1JTn2Kj2sFAV5SxyVLslmoDSWdepqzRs6yYMic3QuKQvTPIXvW','aws_secret_access_key' => 'ye3ySpmaaxCVADAhGz1MbhyBwWnGXW8iJEelVidvq1PZS1fYv6SoQjuTIvZHgHIj'
}gitlab_rails['object_store']['objects']['artifacts']['bucket'] = 'gitlab-artifacts'
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = 'gitlab-mr-diffs'
gitlab_rails['object_store']['objects']['lfs']['bucket'] = 'gitlab-lfs'
gitlab_rails['object_store']['objects']['uploads']['bucket'] = 'gitlab-uploads'
gitlab_rails['object_store']['objects']['packages']['bucket'] = 'gitlab-packages'
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = 'gitlab-dependency-proxy'
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = 'gitlab-terraform-state'
gitlab_rails['object_store']['objects']['ci_secure_files']['bucket'] = 'gitlab-ci-secure-files'
gitlab_rails['object_store']['objects']['pages']['bucket'] = 'gitlab-pages'
gitlab_rails['object_store']['objects']['backups']['bucket'] = 'gitlab-backups'
gitlab_rails['object_store']['objects']['backups']['tmpBucket'] = 'tmp'

gitlab-ctl reconfigure

2.4 gitlab 界面查看并测试

k8s上的项目

之前的文件可以看见，后面上传的文件也可以上传成功。

3. 最终形态的values文件

...
...
## 域名配置hosts:	domain: bdeet.top	hostSuffix:	externalIP:	ssh:	gitlab:	name: kube.bdeet.top	https: true	minio:	name: minio.bdeet.top	https: true	registry:	name: registry.bdeet.top	https: true
...
...
## ldap集成ldap:	preventSignin: false	servers:	main:	label: 'LDAP'	host: '129.226.208.223'	port: 389	uid: 'uid'	bind_dn: 'cn=ldap,dc=wkx,dc=cn'	base: 'dc=wkx,dc=cn'	password:	secret: ldap-admin	key: password	encryption: 'plain'
...
...
## 配置邮箱       smtp:	enabled: true	address: smtp.gmail.com	port: 587	user_name: "[e-mail]"	## https://docs.gitlab.com/charts/installation/secrets#smtp-password	password:	secret: "smtp-gitlab"	key: password	# domain:	authentication: "login"	starttls_auto: true	openssl_verify_mode: "peer"	pool: false	## https://docs.gitlab.com/charts/charts/globals#outgoing-email	## Email persona used in email sent by GitLab	email:	from: "[e-mail]"	display_name: "GitLab Administrator"	reply_to: "[e-mail]"	subject_suffix: "GitLab"	smime:	enabled: false	secretName: ""	keyName: "tls.key"	certName: "tls.crt"
...
...
...
...
nginx-ingress:enabled: true......service:externalTrafficPolicy: "Local"type: "NodePort" #ingress的svc修改为nodeport......
...
...runner:registrationToken:secret: gitlab-gitlab-runner-secret  # gitlab-runner的secret
...
...
gitlab-runner:install: truegitlabUrl: https://kube.bdeet.top  #修改gitlab的域名rbac:create: truerunners:privileged: true #开启特权locked: falseconfig: |[[runners]][runners.kubernetes]image = "ubuntu:18.04"{{- if .Values.global.minio.enabled }}[runners.cache]Type = "s3"Path = "gitlab-runner"Shared = true[runners.cache.s3]#ServerAddress = {{ include "gitlab-runner.cache-tpl.s3ServerAddress" . }}ServerAddress = "https://minio.bdeet.top:31501" #接入对象存储BucketName = "runner-cache"BucketLocation = "us-east-1"Insecure = false
...
...