kubernetes基于helm部署gitlab

kubernetes基于helm部署gitlab

这篇博文介绍如何在 Kubernetes 中使用helm部署 GitLab。

先决条件

• 已运行的 Kubernetes 集群
• 负载均衡器，为ingress-nginx控制器提供EXTERNAL-IP，本示例使用metallb
• 默认存储类，为gitlab pods提供持久化存储，本示例使用nfs-csi

root@ubuntu:~# kubectl -n metallb-system get pods 
NAME                                  READY   STATUS    RESTARTS     AGE
metallb-controller-7d644d8b89-4nhjv   1/1     Running   1 (9h ago)   9h
metallb-speaker-9kwmm                 1/1     Running   1 (9h ago)   9hroot@ubuntu:~# kubectl get sc
NAME                PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-csi (default)   nfs.csi.k8s.io   Delete          Immediate           false                  2d8h

项目地址：https://gitlab.com/gitlab-org/charts/gitlab/-/tree/master/charts/gitlab

官方文档：https://docs.gitlab.com/charts/installation/deployment.html

部署gitlab

添加gitlab helm 仓库

helm repo add gitlab https://charts.gitlab.io

使用helm部署gitlab

helm upgrade --install gitlab gitlab/gitlab \--namespace=gitlab \--create-namespace \--timeout 600s \--set global.edition=ce \--set gitlab-runner.install=false \--set global.hosts.domain=example.com \--set certmanager-issuer.email=[e-mail]

查看运行的pods

root@ubuntu:~# kubectl -n gitlab get pods 
NAME                                               READY   STATUS      RESTARTS     AGE
gitlab-certmanager-569476dc86-pm87k                1/1     Running     0            8h
gitlab-certmanager-cainjector-7cf54459-lnswm       1/1     Running     0            8h
gitlab-certmanager-webhook-69745947df-vq6hw        1/1     Running     0            8h
gitlab-gitaly-0                                    1/1     Running     0            8h
gitlab-gitlab-exporter-b944648cb-w6chf             1/1     Running     0            8h
gitlab-gitlab-shell-6884cccc58-grtjx               1/1     Running     0            8h
gitlab-gitlab-shell-6884cccc58-hhb9k               1/1     Running     0            8h
gitlab-issuer-1-ffvlr                              0/1     Completed   0            8h
gitlab-kas-5cb77566d8-742wx                        1/1     Running     3 (8h ago)   8h
gitlab-kas-5cb77566d8-f6sds                        1/1     Running     3 (8h ago)   8h
gitlab-migrations-1-2j4hg                          0/1     Completed   0            8h
gitlab-minio-8486f8f98b-2ntfs                      1/1     Running     0            8h
gitlab-minio-create-buckets-1-vt2qn                0/1     Completed   0            8h
gitlab-nginx-ingress-controller-56cfd4bf78-wt8vz   1/1     Running     0            8h
gitlab-nginx-ingress-controller-56cfd4bf78-wxtb9   1/1     Running     0            8h
gitlab-postgresql-0                                2/2     Running     0            8h
gitlab-prometheus-server-c4478546-k9c8p            2/2     Running     0            8h
gitlab-redis-master-0                              2/2     Running     0            8h
gitlab-registry-cdb66cfb9-4lcdc                    1/1     Running     0            8h
gitlab-registry-cdb66cfb9-5zpjm                    1/1     Running     0            8h
gitlab-sidekiq-all-in-1-v2-587cc9c486-247f5        1/1     Running     0            8h
gitlab-toolbox-7c576d4dbc-nvttv                    1/1     Running     0            8h
gitlab-webservice-default-756f4bf9b9-tz8wj         2/2     Running     0            8h
gitlab-webservice-default-756f4bf9b9-vjjs4         2/2     Running     0            8h

查看service，确认gitlab-nginx-ingress-controller service是否分配EXTERNAL-IP

root@ubuntu:~# kubectl -n gitlab get svc
NAME                                      TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)                                   AGE
gitlab-certmanager                        ClusterIP      10.96.1.63    <none>           9402/TCP                                  8h
gitlab-certmanager-webhook                ClusterIP      10.96.1.169   <none>           443/TCP                                   8h
gitlab-gitaly                             ClusterIP      None          <none>           8075/TCP,9236/TCP                         8h
gitlab-gitlab-exporter                    ClusterIP      10.96.0.87    <none>           9168/TCP                                  8h
gitlab-gitlab-shell                       ClusterIP      10.96.2.166   <none>           22/TCP                                    8h
gitlab-kas                                ClusterIP      10.96.2.118   <none>           8150/TCP,8153/TCP,8154/TCP,8151/TCP       8h
gitlab-minio-svc                          ClusterIP      10.96.1.87    <none>           9000/TCP                                  8h
gitlab-nginx-ingress-controller           LoadBalancer   10.96.1.193   192.168.72.210   80:30972/TCP,443:32046/TCP,22:31666/TCP   8h
gitlab-nginx-ingress-controller-metrics   ClusterIP      10.96.2.50    <none>           10254/TCP                                 8h
gitlab-postgresql                         ClusterIP      10.96.3.121   <none>           5432/TCP                                  8h
gitlab-postgresql-hl                      ClusterIP      None          <none>           5432/TCP                                  8h
gitlab-postgresql-metrics                 ClusterIP      10.96.2.119   <none>           9187/TCP                                  8h
gitlab-prometheus-server                  ClusterIP      10.96.3.239   <none>           80/TCP                                    8h
gitlab-redis-headless                     ClusterIP      None          <none>           6379/TCP                                  8h
gitlab-redis-master                       ClusterIP      10.96.2.77    <none>           6379/TCP                                  8h
gitlab-redis-metrics                      ClusterIP      10.96.2.45    <none>           9121/TCP                                  8h
gitlab-registry                           ClusterIP      10.96.0.229   <none>           5000/TCP                                  8h
gitlab-webservice-default                 ClusterIP      10.96.2.225   <none>           8080/TCP,8181/TCP,8083/TCP                8h

查看ingress

root@ubuntu:~# kubectl -n gitlab get ingress
NAME                        CLASS          HOSTS                  ADDRESS          PORTS     AGE
gitlab-kas                  gitlab-nginx   kas.example.com        192.168.72.210   80, 443   8h
gitlab-minio                gitlab-nginx   minio.example.com      192.168.72.210   80, 443   8h
gitlab-registry             gitlab-nginx   registry.example.com   192.168.72.210   80, 443   8h
gitlab-webservice-default   gitlab-nginx   gitlab.example.com     192.168.72.210   80, 443   8h

查看pv卷

root@ubuntu:~# kubectl -n gitlab get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                     STORAGECLASS   REASON   AGE
pvc-1f514f02-e926-4947-b4db-5a67873e33e9   10Gi       RWO            Delete           Bound    gitlab/gitlab-minio                       nfs-csi                 8h
pvc-6d355a39-45bc-4b84-b6e9-5db5f123efe2   8Gi        RWO            Delete           Bound    tomcat/tomcat                             nfs-csi                 2d7h
pvc-83b7c3a4-fa4d-4747-bd44-0704952d6006   8Gi        RWO            Delete           Bound    gitlab/data-gitlab-postgresql-0           nfs-csi                 9h
pvc-d1d77751-0760-4609-be83-e45ab6d7c14f   50Gi       RWO            Delete           Bound    gitlab/repo-data-gitlab-gitaly-0          nfs-csi                 9h
pvc-df21b231-263a-4056-bf0c-e226ceee6cb0   8Gi        RWO            Delete           Bound    gitlab/gitlab-prometheus-server           nfs-csi                 8h
pvc-f3b53564-15e1-4613-bbf9-f0a7791d5041   8Gi        RWO            Delete           Bound    gitlab/redis-data-gitlab-redis-master-0   nfs-csi                 9h

访问gitlab

获取gitlab UI root用户的登陆密码

root@ubuntu:~# kubectl -n gitlab get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo
bvTyB0UUwXA3VhVywKOIzdD29KVJV64LB2Td0pyzAJUYe8pcTTOFSYla1SVpXeIx

获取gitlab UI 登陆的URL地址，如果设置 global.hosts.domain=example.com，那么访问地址为

https://gitlab.example.com

配置本地域名解析，其中192.168.72.210为上文gitlab-nginx-ingress-controller service的EXTERNAL-IP

gitlab.example.com 192.168.72.210

登录gitlab后界面如下：

启用gitlab-runner

1、配置gitlab url在pod中的域名解析

gitlab-runner pod启用时需要从pod内部使用 gitlab 外部域名https://gitlab.example.com注册到gitlab，修改coredns配置，提供本地域名解析。

root@ubuntu:~# kubectl -n kube-system edit cm coredns
apiVersion: v1
data:Corefile: |.:53 {errorshosts {192.168.72.210 gitlab.example.comfallthrough}......}

重启coredns pods

kubectl -n kube-system rollout restart deployment.apps/coredns

2、导出gitlab自签名证书

kubectl -n gitlab get secret gitlab-gitlab-tls --template='{{ index .data "tls.crt" }}' | base64 -d > gitlab.crt

3、基于自签名证书创建secrets

kubectl -n gitlab create secret generic gitlab-runner-certs \--from-file=gitlab.example.com.crt=gitlab.crt \--from-file=registry.example.com.crt=gitlab.crt \--from-file=minio.example.com.crt=gitlab.crt

4、更新已安装的gitlab实例，启用gitlab-runner并指定secrets

helm upgrade --install gitlab gitlab/gitlab \--namespace=gitlab \--set gitlab-runner.install=true \--set gitlab-runner.certsSecretName=gitlab-runner-certs \--reuse-values

5、查看gitalb-runner pods

root@ubuntu:~# kubectl -n gitlab get pods -l app=gitlab-gitlab-runner
NAME                                    READY   STATUS    RESTARTS   AGE
gitlab-gitlab-runner-6c8cd68548-v6qpd   1/1     Running   0          5m12s

6、登陆UI查看注册的runner

选择Your work–> Admin Area

选择CI/CD–>Runners，确认存在一个Online状态的Runner