Day01-02-gitlab

1. 什么是gitlab

• 私有代码仓库
• 精细化权限配置，让系统更安全
• 控制用户/用户组是否可以提交到主分支 (PR Push Request)
• 它使用Ruby语言写成。后来，一些部分用Go语言重写

2. Gitlab vs Github/Gitee

	Gitlab	Github/Gitee
共同点	存放代码,git访问	存放代码,git访问
不同点	精确化控制权限 全面安全措施 定时备份 升级 迁移	使用权限,用户管理比较弱,无法做到精细化权限.

3. Gitlab 应用场景

• 开源免费,搭建简单,维护成本低,符合中小公司口味 gogs.io
• 权限管理,实现代码对部分人可见,安全性高
• 离线同步

#gogs 
https://gogs.io/docs/installation/install_from_binary
https://blog.mynook.info/post/host-your-own-git-server-using-gogs/

4. 架构

链接: https://www.processon.com/view/link/61032dde1efad46a20af53f3

• 超详细说明(了解)
  链接: https://docs.gitlab.com/ee/development/architecture.html#components

5. Gitlab 快速上手指南

• 版本选择

gitlab-ce 15.9.x

• ce community edition 社区版

• ee enterprice edition 企业版

1. 安装Gitlab依赖包
2. 下载,安装
3. 配置,访问域名
4. 初始化,启动
5. 访问

5.0 安装要求

• Installation requirements | GitLab 链接: https://docs.gitlab.com/ee/install/requirements.html

#系统要求
Ubuntu (16.04/18.04/20.04)
Debian (9/10)
CentOS (7/8)
openSUSE Leap (15.2)
SUSE Linux Enterprise Server (12 SP2/12 SP5)
Red Hat Enterprise Linux (please use the CentOS packages and instructions)#硬件要求
#磁盘存放代码,比代码空间大  7.2k以上的磁盘 sas ssd  ##CPU 
4 核心 500用户
8 核心 1000用户
## 内存
4 G内存 500用户
8 G内存 1000用户

• 极高并发超过1000用户参考的架构 链接: https://docs.gitlab.com/ee/administration/reference_architectures/index.html

5.1 安装Gitlab组件

yum localinstall -y gitlab-ce-15.9.3-ce.0.el7.x86_64.rpm 
# 安装成功提示

GitLab was unable to detect a valid hostname for your instance.
Please configure a URL for your GitLab instance by setting `external_url`  
#修改gitlab配置文件设置 gitlab external_url (gitlab.oldboylinux.cn server_name)configuration in /etc/gitlab/gitlab.rb file.  #配置文件的路径和名字
Then, you can start your GitLab instance by running the following command:
sudo gitlab-ctl reconfigure      #初始化(重新配置)

5.3 配置访问url

备份原始配置文件
修改配置文件内容为:
[root@devops-oldboy-gitlab ~]# cat /etc/gitlab/gitlab.rb
external_url 'http://gitlab.oldboylinux.cn'
gitlab_rails['smtp_enable'] = false
gitlab_rails['registry_enabled'] = false
registry['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
node_exporter['enable'] = false
redis_exporter['enable'] = false
postgres_exporter['enable'] = false
gitlab_exporter['enable'] = false
prometheus_monitoring['enable'] = false

5.6 初始化

• 修改配置文件需要初始化 gitlab

#使用gitlab.rb文件生成子配置文件
gitlab-ctl reconfigure 
#重启所有gitlab相关文件.
gitlab-ctl restart
Infra Phase complete, 423/1081 resources updated in03 minutes 27 secondsNotes:
Default admin account has been configured with following details:
Username: root
Password: You didn't opt-in to print initial root password to STDOUT.
Password stored to /etc/gitlab/initial_root_password. This file will be cleaned up in first reconfigure run after 24 hours.NOTE: Because these credentials might be present in your log files in plain text, it is highly recommended to reset the password following https://docs.gitlab.com/ee/security/reset_user_password.html#reset-your-root-password.[2024-05-15T15:02:27+08:00] WARN: This release of Cinc Client became end of life (EOL) on May 1st 2023. Please update to a supported release to receive new features, bug fixes, and security updates.
gitlab Reconfigured!

gitlab-ctl status

gitlab-ctl restart

5.8 登录与查看

• gitlab.oldboylinux.cn

5.9 汉化

• 进入账号里面的:prefenences(属性)

• 选择Localization 中的 Language,选择中文，然后保存

5.10 设置密码

5.11 目录结构

ll /var/opt/gitlab/ /var/log/gitlab/ -d配置文件目录 /var/opt/gitlab/  
日志目录     /var/log/gitlab/

5.12 删除

• 删除rpm
• /var/opt/gitlab /opt/gitlab/ 2个目录删除.
• 备份内核参数配置 /opt/gitlab/embedded/etc/*-gitlab-*.conf
• 安装完成,覆盖回去上面几个conf.

5.13 500 vs 502

• 需要等待,启动较慢.检查资源是否够用.
• 500错误.

gitlab-rake db:migrate
gitlab-ctl reconfigure
gitlab-ctl restart

5.14 重置密码

gitlab-rake "gitlab:password:reset"

6. Gitlab用户与组

6.1 基础关系

链接: https://www.processon.com/view/link/61032dde1efad46a20af53f3

6.2 应用案例

案例创建用户组:dev

• dev开发组

• 添加用户oldboy

• 创建1个项目live_app项目

• 项目与dev组关联

• dev开发组

• 添加用户oldboy

• 关联用户与用户组.
  进入群组中,选择邀请成员.权限注意要设置为owner权限.

• 创建项目

• 登录上面的lidao996用户,然后配置密钥认证(流程类似于码云)

• 推送代码

Git 全局设置
git config --global user.name "lidao996"
git config --global user.email "[e-mail]"# 重命名已有的仓库
git remote rename origin old-origin
#添加新的仓库地址
git remote add origin [e-mail]:dev/lidao_app.git
#推送所有分支
git push -u origin --all
#推送所有标签
git push -u origin --tags

• 设置汉化

• 配置秘钥

• 复制保存

• 推送报错

[root@devops-gitlab live-lidao-app]# git push -u origin --all
The authenticity of host 'gitlab.oldboylinux.cn (172.16.1.71)' can't be established.
ECDSA key fingerprint is SHA256:FugASp0AiEBJGhKbpj9MDlJIVgot3PfDqKa9TNaLdj4.
ECDSA key fingerprint is MD5:5c:57:46:28:61:1e:7a:86:71:c0:35:4d:e3:55:7d:cf.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'gitlab.oldboylinux.cn,172.16.1.71' (ECDSA) to the list of known hosts.
Counting objects: 18, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (18/18), 1.54 KiB | 0 bytes/s, done.
Total 18 (delta 0), reused 0 (delta 0)
remote: GitLab: 
remote: A default branch (e.g. main) does not yet exist for dev/lidao_app
remote: Ask a project Owner or Maintainer to create a default branch:
remote: 
remote:   http://gitlab.oldboylinux.cn/dev/lidao_app/-/project_members
remote: 
To [e-mail]:dev/lidao_app.git! [remote rejected] master -> master (pre-receive hook declined)! [remote rejected] shopping -> shopping (pre-receive hook declined)
error: 无法推送一些引用到 '[e-mail]:dev/lidao_app.git'

• 修改权限，权限要是owner权限.

• 检查推送结果

• 流程小结

今日内容

• CI,CD,DevSecOps,DevOps描述清楚.

• Git命令

  • 上传代码(git add / git commit /git push -u origin master/ git push -u origin Վʔtags)

  • 下载代码(git clone http/git)

• 公共代码仓库:Gitee.com 配置密钥访问.

• 私有代码仓库:Gitlab配置密钥访问.

7. Gitlab 备份与恢复

• 目标:
  • 备份Gitlab相关的配置文件
  • 备份Gitlab中的程序代码(代码仓库(项目),用户,用户组,密钥.)

7.0 备份Gitlab服务的配置

• /etc/gitlab目录下所有内容即可

7.1 备份Gitlab里面的内容

1）配置

#修改之后的gitlab.rb
[root@devops-gitlab gitlab]# cat gitlab.rb
external_url 'http://gitlab.oldboylinux.cn'
gitlab_rails['manage_backup_path'] = true
gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
gitlab_rails['backup_archive_permissions'] = 0644
gitlab_rails['backup_keep_time'] = 604800
gitlab_rails['smtp_enable'] = false
gitlab_rails['registry_enabled'] = false
registry['enable'] = false
prometheus['enable'] = false
alertmanager['enable'] = false
node_exporter['enable'] = false
redis_exporter['enable'] = false
postgres_exporter['enable'] = false
gitlab_exporter['enable'] = false
prometheus_monitoring['enable'] = false#然后执行
gitlab-ctl reconfigure 
gitlab-ctl restartok: run: gitlab-kas: (pid 2274) 0s

#配置文件 的backup部分
backup_
### Backup Settings
###! Docs: https://docs.gitlab.com/omnibus/settings/backups.htmlgitlab_rails['manage_backup_path'] = true    #是否可以指定备份目录
gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"     #备份目录
# gitlab_rails['backup_gitaly_backup_path'] = "/opt/gitlab/embedded/bin/gitaly-backup"###! Docs: https://docs.gitlab.com/ee/raketasks/backup_restore.html#backup-archive-permissions
gitlab_rails['backup_archive_permissions'] = 0644    #备份的压缩包权限# gitlab_rails['backup_pg_schema'] = 'public'###! The duration in seconds to keep backups before they are allowed to be deleted
gitlab_rails['backup_keep_time'] = 604800     #备份保留多久 7天
#gitlab备份 需要手动执行#配置生效
gitlab-ctl reconfigure 
gitlab-ctl  restart[root@devops-gitlab gitlab]#  ll /var/opt/gitlab/backups -d
drwx------ 2 git root 6 2024-05-15 15:00 /var/opt/gitlab/backups

2）手动备份

#GitLab 版本≥12.2
gitlab-backup create#GitLab 版本≤12.1:
gitlab-rake gitlab:backup:create#备份过程
2024-05-16 00:56:06 UTC -- Dumping database ... 
Dumping PostgreSQL database gitlabhq_production ... [DONE]
2024-05-16 00:56:09 UTC -- Dumping database ... done
2024-05-16 00:56:09 UTC -- Dumping repositories ... 
{"command":"create","gl_project_path":"gitlab-instance-05c96a12/Monitoring","level":"info","msg":"started create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git","storage_name":"default","time":"2024-05-16T00:56:10.231Z"}
{"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"gitlab-instance-05c96a12/Monitoring","level":"warning","msg":"skipped create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git","storage_name":"default","time":"2024-05-16T00:56:10.244Z"}
{"command":"create","gl_project_path":"gitlab-instance-05c96a12/Monitoring.wiki","level":"info","msg":"started create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git","storage_name":"default","time":"2024-05-16T00:56:10.321Z"}
{"command":"create","gl_project_path":"gitlab-instance-05c96a12/Monitoring","level":"info","msg":"started create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.design.git","storage_name":"default","time":"2024-05-16T00:56:10.323Z"}
{"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"gitlab-instance-05c96a12/Monitoring","level":"warning","msg":"skipped create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.design.git","storage_name":"default","time":"2024-05-16T00:56:10.324Z"}
{"command":"create","gl_project_path":"dev/lidao_app","level":"info","msg":"started create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git","storage_name":"default","time":"2024-05-16T00:56:10.324Z"}
{"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"gitlab-instance-05c96a12/Monitoring.wiki","level":"warning","msg":"skipped create","relative_path":"@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git","storage_name":"default","time":"2024-05-16T00:56:10.324Z"}
{"command":"create","gl_project_path":"dev/lidao_app.wiki","level":"info","msg":"started create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.wiki.git","storage_name":"default","time":"2024-05-16T00:56:10.329Z"}
{"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"dev/lidao_app.wiki","level":"warning","msg":"skipped create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.wiki.git","storage_name":"default","time":"2024-05-16T00:56:10.336Z"}
{"command":"create","gl_project_path":"dev/lidao_app","level":"info","msg":"started create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.design.git","storage_name":"default","time":"2024-05-16T00:56:10.336Z"}
{"command":"create","error":"manager: repository empty: repository skipped","gl_project_path":"dev/lidao_app","level":"warning","msg":"skipped create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.design.git","storage_name":"default","time":"2024-05-16T00:56:10.336Z"}
{"command":"create","gl_project_path":"dev/lidao_app","level":"info","msg":"completed create","relative_path":"@hashed/d4/73/d4735e3a265e16eee03f59718b9b5d03019c07d8b6c51f90da3a666eec13ab35.git","storage_name":"default","time":"2024-05-16T00:56:10.355Z"}
2024-05-16 00:56:10 UTC -- Dumping repositories ... done
2024-05-16 00:56:10 UTC -- Dumping uploads ... 
2024-05-16 00:56:10 UTC -- Dumping uploads ... done
2024-05-16 00:56:10 UTC -- Dumping builds ... 
2024-05-16 00:56:10 UTC -- Dumping builds ... done
2024-05-16 00:56:10 UTC -- Dumping artifacts ... 
2024-05-16 00:56:10 UTC -- Dumping artifacts ... done
2024-05-16 00:56:10 UTC -- Dumping pages ... 
2024-05-16 00:56:10 UTC -- Dumping pages ... done
2024-05-16 00:56:10 UTC -- Dumping lfs objects ... 
2024-05-16 00:56:10 UTC -- Dumping lfs objects ... done
2024-05-16 00:56:10 UTC -- Dumping terraform states ... 
2024-05-16 00:56:10 UTC -- Dumping terraform states ... done
2024-05-16 00:56:10 UTC -- Dumping container registry images ... [DISABLED]
2024-05-16 00:56:10 UTC -- Dumping packages ... 
2024-05-16 00:56:10 UTC -- Dumping packages ... done
2024-05-16 00:56:10 UTC -- Creating backup archive: 1715820966_2024_05_16_15.9.3_gitlab_backup.tar ... 
2024-05-16 00:56:10 UTC -- Creating backup archive: 1715820966_2024_05_16_15.9.3_gitlab_backup.tar ... done
2024-05-16 00:56:10 UTC -- Uploading backup archive to remote storage  ... [SKIPPED]
2024-05-16 00:56:10 UTC -- Deleting old backups ... 
2024-05-16 00:56:10 UTC -- Deleting old backups ... done. (0 removed)
2024-05-16 00:56:10 UTC -- Deleting tar staging files ... 
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/backup_information.yml
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/db
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/repositories
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/uploads.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/builds.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/artifacts.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/pages.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/lfs.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/terraform_state.tar.gz
2024-05-16 00:56:10 UTC -- Cleaning up /var/opt/gitlab/backups/packages.tar.gz
2024-05-16 00:56:10 UTC -- Deleting tar staging files ... done
2024-05-16 00:56:10 UTC -- Deleting backups/tmp ... 
2024-05-16 00:56:10 UTC -- Deleting backups/tmp ... done
2024-05-16 00:56:10 UTC -- Warning: Your gitlab.rb and gitlab-secrets.json files contain sensitive data 
and are not included in this backup. You will need these files to restore a backup.
Please back them up manually.
2024-05-16 00:56:10 UTC -- Backup 1715820966_2024_05_16_15.9.3 is done.
2024-05-16 08:56:10 +0800 -- Deleting backup and restore lock file
[root@devops-gitlab gitlab]#  ll /var/opt/gitlab/backups
总用量 412
-rw-r--r-- 1 git git 419840 2024-05-16 08:56 1715820966_2024_05_16_15.9.3_gitlab_backup.tar

• 对于gitlab.rb 和gitlab-secrets.json包含敏感信息,需要手动备份.

• 参考链接: https://docs.gitlab.com/ce/raketasks/backup_restore.html

3）恢复

1. 停止写入类服务
gitlab-ctl stop unicorn
gitlab-ctl stop sidekiq2. gitlab-rake恢复 (不需要加tar)
#新版本GitLab 版本≥12.2:
gitlab-backup restore BACKUP=1715820966_2024_05_16_15.9.3#旧版本GitLab 版本≤12.1:
gitlab-rake gitlab:backup:restore BACKUP=备份文件名字不包含右边内容_gitlab-backup.tar
gitlab-rake gitlab:backup:restore BACKUP=1656036972_2022_06_24_12.0.9Do you want to continue (yes/no)? yes 输入yes
Do you want to continue (yes/no)? yes 输入yes3. 重启 gitlab 
gitlab-ctl restart

• 书写gitlab备份脚本+定时任务
• 备份相关数据(根据gitlab版本选择不同的备份命令)
• 备份gitlab配置 ,给压缩包加上密码

4）自动备份脚本

#每天备份1次
#!/bin/bash
#author :lidao996 
#desc : backup gitlab all
#1.备份gitlab数据
gitlab-backup create    
#2.备份gitlab配置和密码文件
tar zcf /backup/gitlab/gitlab-conf-$(date +%F).tar.gz   /etc/gitlab/
#3. 传输到备份服务器
rsync xxx 
#写入定时任务
00 03 * * * sh /server/scripts/backup-gitlab.sh &>/dev/null

• 书写恢复脚本
  • 运行的时候显示可以恢复到哪一天?
  • 显示 /var/opt/gitlab/backups/下面内容
  • 让用户进行选择恢复到哪一天的?输入日期2023_03_27
  • 然后进行恢复.

8. 迁移与升级

#1. 新功能,非常吸引  
#2. 修复bug,漏洞
#3. 公司制定,统一版本.为了标准化,自动化.

• gitlab升级的要求。

需要遵循以下升级步骤以确保主版本升级成功：
1.升级至之前主要版本的最新次要版本。  
2.升级到目标主要版本的第一个次要版本（X.0.Z）。
3.继续升级到较新的版本。

• 迁移

9. Https

为了防止内网渗透，将gitlab服务的访问添加了ssl,具体步骤如下：
1.gitlab配置https(请先做个快照.)
external_url 'https:Վˌgitlab.oldboylinux.cn'
nginx[' enable' ] = true
nginx['client_max_body_size' ]= '250m '
nginx['redirect_http_to_https' ]= true
nginx[' redirect_http_to_https _port' ] = 443
nginx['ssl_certificate' ] = "path/ key.crt"
nginx['ssl_certificate_key'] = "path/ key.key"
nginx['ssl_ciphers' ] ="ECDHE-RSA-AES256-GCMSHA384:ECDHE-RSA-AES128-GCM-SHA256"
nginx['ssl_prefer_server_ciphers']= "on"
nginx['ssl_protocols']= "TLSv1.2"
nginx['ssl_session_cache ' ] = "builtin:1000shared:sSL: 10m"
nginx['ssl_session_timeout']= "5m"核心：
nginx['ssl_certificate' ] =用于指定证书 .crt .pem
nginx['ssl_certificate_key']  = 用于指定私钥 .key2.gitlab重新初始化
gitlab-ctl reconfigure
重建配置：
gitlab-ctl restart

10. 邮件

1 配置邮箱 (发件人) (选作)

• 注册的时候 Gitlab 服务器,给用户发送邮件,激活 或者找回密码
• 用提交合并请求的时候,负责人可以收到邮件通知

## 配置发件人
51 ### Email Settings
52 gitlab_rails['gitlab_email_enabled'] = true
53 gitlab_rails['gitlab_email_from'] = '[e-mail]'
54 gitlab_rails['gitlab_email_display_name'] = 'Oldboy_gitlab_tongzhi'## 配置详细发件人信息
513 ### GitLab email server settings
514 ###! Docs: https://docs.gitlab.com/omnibus/settings/smtp.html
515 ###! **Use smtp instead of sendmail/postfix.**
516
517 gitlab_rails['smtp_enable'] = true
518 gitlab_rails['smtp_address'] = "smtp.163.com"
519 gitlab_rails['smtp_port'] = 465
520 gitlab_rails['smtp_user_name'] = "[e-mail]"
521 gitlab_rails['smtp_password'] = "????授权码"
522 gitlab_rails['smtp_domain'] = "163.com"
523 gitlab_rails['smtp_authentication'] = "login"
524 gitlab_rails['smtp_enable_starttls_auto'] = true
525 gitlab_rails['smtp_tls'] = truegitlab-ctl reconfigure 
gitlab-ctl restart

2 验证Gitlab组件 (选作 配置了邮箱)

gitlab-rails console 回车
Notify.test_email('收件人','主题','内容').diliver_now
Notify.test_email('[e-mail]','Gitlab-test','测试').diliver_now[root@gitlab ~]# gitlab-rails c

11. 优化(已完成)

## 优化暂时不用组件 (选作)
#关闭目前不使用的组件 默认都是true 修改为 false 
#关闭prometheus
1385 prometheus['enable'] = false
1386 prometheus['monitor_kubernetes'] = false
#关闭alertmanger
1452 alertmanager['enable'] = false
#关闭exporter 如果需要prometheus监控 则可以打开。
1474 node_exporter['enable'] = false
1493 redis_exporter['enable'] = false
1511 postgres_exporter['enable'] = false
1540 gitlab_monitor['enable'] = false
#gitlab_exporter
1553 prometheus_monitoring['enable'] = false
1560 grafana['enable'] = false

12. 查看帮助

docker run -it --rm -p 4000:4000 registry.gitlab.com/gitlab-org/gitlab-docs:12.0

13. Gitlab

• Gitlab vs Github/gitee区别 🌟🌟🌟🌟🌟

• 用户,用户组,项目(仓库) 关联 🌟🌟

• 应用案例: 🌟🌟

  • 用户上传到仓库。
  • 密钥登录案例。

• Gitlab备份与恢复