【Docker】Docker Swarm介绍与环境搭建

为什么不建议在生产环境中使用Docker Compose

• 多机器如何管理？
• 如何跨机器做scale横向扩展？
• 容器失败退出时如何新建容器确保服务正常运行？
• 如何确保零宕机时间？
• 如何管理密码，Key等敏感数据？

Docker Swarm介绍

Docker Swarm是Docker官方推出的容器集群管理工具，基于Go语言实现。使用它可以将多个Docker主机封装为单个大型的虚拟Docker主机，快速打造一套容器云平台。

Docker Swarm是生产环境中运行Docker应用程序最简单的方法。作为容器集群管理器，Swarm最大的优势之一就是100%支持标准的Docker API。各种基于标准API的工具比如Compose、docker-py、各种管理软件，甚至 Docker本身等都可以很容易的与Swarm进行集成。大大方便了用户将原先基于单节点的系统移植到Swarm上，同时Swarm内置了对 Docker网络插件的支持，用户可以很容易地部署跨主机的容器集群服务。

Docker Swarm和Docker Compose一样，都是Docker官方容器编排工具，但不同的是，Docker Compose是一个在单个服务器或主机上创建多个容器的工具，而Docker Swarm则可以在多个服务器或主机上创建容器集群服务，对于微服务的部署，显然Docker Swarm会更加适合。

Swarm的基本架构

Swarm单节点快速上手

激活Docker Swarm

docker info这个命令可以查看我们的docker engine有没有激活swarm模式， 默认是没有的，我们会看到

$ docker info | grep SwarmSwarm: inactive

激活swarm，有两个方法：

• 初始化一个swarm集群，自己成为manager
• 加入一个已经存在的swarm集群

初始化一个swarm集群，自己成为manager：

$ docker swarm init --advertise-addr 192.168.0.13
Swarm initialized: current node (1iohpmz3jxqecpeixv77bkz6k) is now a manager.To add a worker to this swarm, run the following command:docker swarm join --token SWMTKN-1-45kligs4iwu0sqdy2xxvf4mrauggwc5hlmo7sj9u5ve5cjtek4-54bo11h1uwa6yfva1jf9g2blb 192.168.0.13:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

加入一个已经存在的swarm集群：

$ docker swarm join --token SWMTKN-1-45kligs4iwu0sqdy2xxvf4mrauggwc5hlmo7sj9u5ve5cjtek4-54bo11h1uwa6yfva1jf9g2blb 192.168.0.13:2377

查看swarm集群中的机器节点：

$ docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
xk8ryecmxfqu65zvsgk41h8cd *   node1      Ready     Active         Leader           24.0.2

再次使用docker info这个命令可以看到我们的docker engine已经激活swarm模式：

$ docker info | grep SwarmSwarm: active

docker swarm init 背后发生了什么

主要是PKI和安全相关的自动化

• 创建swarm集群的根证书
• manager节点的证书
• 其它节点加入集群需要的tokens
• 创建Raft数据库用于存储证书，配置，密码等数据

RAFT相关资料

• http://thesecretlivesofdata.com/raft/
• https://raft.github.io/
• https://docs.docker.com/engine/swarm/raft/

Swarm三节点集群搭建

创建3节点swarm cluster的方法

• https://labs.play-with-docker.com/网站，优点是快速方便，缺点是环境不持久，4个小时后环境会被重置
• 在本地通过虚拟化软件搭建Linux虚拟机，优点是稳定，方便，缺点是占用系统资源，需要电脑内存最好8G及其以上
• 在云上使用云主机， 亚马逊，Google，微软Azure，阿里云，腾讯云等，缺点是需要消耗金钱（但是有些云服务，有免费试用）

多节点的环境涉及到机器之间的通信需求，所以防火墙和网络安全策略组是大家一定要考虑的问题，特别是在云上使用云主机的情况，下面这些端口记得打开防火墙以及设置安全策略组：

• Port 2377 TCP for communication with and between manager nodes
• Port 7946 TCP/UDP for overlay network node discovery
• Port 4789 UDP (configurable) for overlay network traffic

为了简化，以上所有端口都允许节点之间自由访问就行。

添加2个节点到Swarm

主机规划：

• 192.168.0.13：manager
• 192.168.0.12：worker
• 192.168.0.11：worker

添加192.168.0.12到Swarm：

$ docker swarm join --token SWMTKN-1-45kligs4iwu0sqdy2xxvf4mrauggwc5hlmo7sj9u5ve5cjtek4-54bo11h1uwa6yfva1jf9g2blb 192.168.0.13:2377
This node joined a swarm as a worker.

添加192.168.0.11到Swarm：

$ docker swarm join --token SWMTKN-1-45kligs4iwu0sqdy2xxvf4mrauggwc5hlmo7sj9u5ve5cjtek4-54bo11h1uwa6yfva1jf9g2blb 192.168.0.13:2377
This node joined a swarm as a worker.

查看Swarm中的节点：

$ docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
1iohpmz3jxqecpeixv77bkz6k *   node1      Ready     Active         Leader           24.0.2
k7sl3sg1x193nalsdjs6dw20w     node2      Ready     Active                          24.0.2
aitp94gxkvg997b5qev271v2y     node3      Ready     Active                          24.0.2

ID后面的*表示的是当前节点，docker node ls需要在manager节点执行。

发布一个service到swarm

可以使用docker service create来创建一个service：

$ docker service create --replicas 1 --name helloworld alpine ping docker.com
dq7ch2rdyijjvuh03k6sorw3q
overall progress: 1 out of 1 tasks
1/1: running   [==================================================>]
verify: Service converged

可以使用docker service ls查看启动了哪些服务：

$ docker service ls
ID             NAME         MODE         REPLICAS   IMAGE           PORTS
dq7ch2rdyijj   helloworld   replicated   1/1        alpine:latest

查看service详情

可以使用docker service inspect来查看service的详情：

$ docker service inspect helloworld
[{"ID": "dq7ch2rdyijjvuh03k6sorw3q","Version": {"Index": 21},"CreatedAt": "2023-10-12T02:28:33.280090826Z","UpdatedAt": "2023-10-12T02:28:33.280090826Z","Spec": {"Name": "helloworld","Labels": {},"TaskTemplate": {"ContainerSpec": {"Image": "alpine:latest@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978","Args": ["ping","docker.com"],"Init": false,"StopGracePeriod": 10000000000,"DNSConfig": {},"Isolation": "default"},"Resources": {"Limits": {},"Reservations": {}},"RestartPolicy": {"Condition": "any","Delay": 5000000000,"MaxAttempts": 0},"Placement": {"Platforms": [{"Architecture": "amd64","OS": "linux"},{"OS": "linux"},{"OS": "linux"},{"Architecture": "arm64","OS": "linux"},{"Architecture": "386","OS": "linux"},{"Architecture": "ppc64le","OS": "linux"},{"Architecture": "s390x","OS": "linux"}]},"ForceUpdate": 0,"Runtime": "container"},"Mode": {"Replicated": {"Replicas": 1}},"UpdateConfig": {"Parallelism": 1,"FailureAction": "pause","Monitor": 5000000000,"MaxFailureRatio": 0,"Order": "stop-first"},"RollbackConfig": {"Parallelism": 1,"FailureAction": "pause","Monitor": 5000000000,"MaxFailureRatio": 0,"Order": "stop-first"},"EndpointSpec": {"Mode": "vip"}},"Endpoint": {"Spec": {}}}
]

可以加上--pretty参数输出可读性更好的内容格式：

$ docker service inspect --pretty helloworldID:             dq7ch2rdyijjvuh03k6sorw3q
Name:           helloworld
Service Mode:   ReplicatedReplicas:      1
Placement:
UpdateConfig:Parallelism:   1On failure:    pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:   1On failure:    pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:         alpine:latest@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978Args:          ping docker.comInit:          false
Resources:
Endpoint Mode:  vip

可以使用docker service ps查看service运行在哪些node上：

$ docker service ps helloworld
ID             NAME           IMAGE           NODE      DESIRED STATE   CURRENT STATE           ERROR     PORTS
8xi2hbecua0s   helloworld.1   alpine:latest   node2     Running         Running 7 minutes ago

也可以使用docker ps在node2上查看启动了哪些容器：

$ docker ps
CONTAINER ID   IMAGE           COMMAND             CREATED         STATUS         PORTS     NAMES
9acee1709671   alpine:latest   "ping docker.com"   9 minutes ago   Up 9 minutes             helloworld.1.8xi2hbecua0s278o91jmod8li

对service进行扩容和缩容

可以使用docker service scale对服务进行扩容和缩容，任务数量比原来的多就是扩容，任务数量比原来的少就是缩容：

$ docker service scale helloworld=5
helloworld scaled to 5
overall progress: 5 out of 5 tasks
1/5: running   [==================================================>]
2/5: running   [==================================================>]
3/5: running   [==================================================>]
4/5: running   [==================================================>]
5/5: running   [==================================================>]
verify: Service converged$ docker service ps helloworld
ID             NAME           IMAGE         service  NODE      DESIRED STATE   CURRENT STATE            ERROR     PORTS
8xi2hbecua0s   helloworld.1   alpine:latest   node2     Running         Running 31 minutes ago
72s13jwy17fh   helloworld.2   alpine:latest   node1     Running         Running 21 seconds ago
l7js9yqaoela   helloworld.3   alpine:latest   node3     Running         Running 21 seconds ago
kyj28313lqng   helloworld.4   alpine:latest   node3     Running         Running 21 seconds ago
pzbgzk1hrihp   helloworld.5   alpine:latest   node2     Running         Running 22 seconds ago

删除一个service

可以使用docker service rm来删除一个service

$ docker service rm helloworld
helloworld$ docker service ls
ID        NAME      MODE      REPLICAS   IMAGE     PORTS

滚动更新service

$ docker service create --replicas 3 --name redis --update-delay 10s redis:3.0.6
p7x6a22i3pagnpvt9pyrzack8
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged
[node1] (local) root@192.168.0.13 ~
$ docker service ps redis
ID             NAME      IMAGE         NODE      DESIRED STATE   CURRENT STATE            ERROR     PORTS
tynsv7a56x1t   redis.1   redis:3.0.6   node1     Running         Running 23 seconds ago
55c4yfweaja7   redis.2   redis:3.0.6   node2     Running         Running 22 seconds ago
r3hcrtgxlehp   redis.3   redis:3.0.6   node3     Running         Running 23 seconds ago
[node1] (local) root@192.168.0.13 ~
$
[node1] (local) root@192.168.0.13 ~
$ docker service inspect --pretty redisID:             p7x6a22i3pagnpvt9pyrzack8
Name:           redis
Service Mode:   ReplicatedReplicas:      3
Placement:
UpdateConfig:Parallelism:   1Delay:         10sOn failure:    pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:   1On failure:    pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:         redis:3.0.6@sha256:6a692a76c2081888b589e26e6ec835743119fe453d67ecf03df7de5b73d69842Init:          false
Resources:
Endpoint Mode:  vip

将redis:3.0.6滚动升级为redis:3.0.7：

$ docker service update --image redis:3.0.7 redis
redis
overall progress: 3 out of 3 tasks
1/3: running   [==================================================>]
2/3: running   [==================================================>]
3/3: running   [==================================================>]
verify: Service converged
[node1] (local) root@192.168.0.13 ~
$ docker service inspect --pretty redisID:             p7x6a22i3pagnpvt9pyrzack8
Name:           redis
Service Mode:   ReplicatedReplicas:      3
UpdateStatus:State:         completedStarted:       About a minute agoCompleted:     22 seconds agoMessage:       update completed
Placement:
UpdateConfig:Parallelism:   1Delay:         10sOn failure:    pauseMonitoring Period: 5sMax failure ratio: 0Update order:      stop-first
RollbackConfig:Parallelism:   1On failure:    pauseMonitoring Period: 5sMax failure ratio: 0Rollback order:    stop-first
ContainerSpec:Image:         redis:3.0.7@sha256:730b765df9fe96af414da64a2b67f3a5f70b8fd13a31e5096fee4807ed802e20Init:          false
Resources:
Endpoint Mode:  vip[node1] (local) root@192.168.0.13 ~
$ docker service ps redis
ID             NAME          IMAGE         NODE      DESIRED STATE   CURRENT STATE                 ERROR     PORTS
sard15sxoo3b   redis.1       redis:3.0.7   node1     Running         Running about a minute ago
tynsv7a56x1t    \_ redis.1   redis:3.0.6   node1     Shutdown        Shutdown about a minute ago
xzkhsd9nuqxj   redis.2       redis:3.0.7   node2     Running         Running 53 seconds ago
55c4yfweaja7    \_ redis.2   redis:3.0.6   node2     Shutdown        Shutdown 58 seconds ago
jkadyq5a9ekq   redis.3       redis:3.0.7   node3     Running         Running about a minute ago
r3hcrtgxlehp    \_ redis.3   redis:3.0.6   node3     Shutdown        Shutdown about a minute ago

移除一个节点

现在的节点情况：

$ docker service ps redis
ID             NAME          IMAGE         NODE      DESIRED STATE   CURRENT STATE            ERROR     PORTS
sard15sxoo3b   redis.1       redis:3.0.7   node1     Running         Running 4 minutes ago
xzkhsd9nuqxj   redis.2       redis:3.0.7   node2     Running         Running 3 minutes ago
jkadyq5a9ekq   redis.3       redis:3.0.7   node3     Running         Running 4 minutes ago$ docker node ls
ID                            HOSTNAME   STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
bbg9dqcotcgfe2ikgyjgc7nge *   node1      Ready     Active         Leader           24.0.2
7kj69w9cdpyawh54fumibuu0u     node2      Ready     Active                          24.0.2
x8x5ox0e6pfx4jg311m564a45     node3      Ready     Active                          24.0.2

可以使用docker node update --availability drain来将一个节点排除：

$ docker node update --availability drain node2
node2

可以使用docker node inspect来查看节点的详情：

$ docker node inspect --pretty node2
ID:                     7kj69w9cdpyawh54fumibuu0u
Hostname:               node2
Joined at:              2023-10-12 03:40:28.774753447 +0000 utc
Status:State:                 ReadyAvailability:          Drain
。。。

Availability已经变为Drain。

再来看看redis的实例少了没：

$ docker service ps redis
ID             NAME          IMAGE         NODE      DESIRED STATE   CURRENT STATE             ERROR     PORTS
sard15sxoo3b   redis.1       redis:3.0.7   node1     Running         Running 5 minutes ago
mxukqpq3xb5h   redis.2       redis:3.0.7   node1     Running         Running 53 seconds ago
xzkhsd9nuqxj    \_ redis.2   redis:3.0.7   node2     Shutdown        Shutdown 54 seconds ago
jkadyq5a9ekq   redis.3       redis:3.0.7   node3     Running         Running 5 minutes ago

发现原来部署在node2上的redis容器停止了，自动在node1上启动了一个新的redis容器。

可以使用docker node update --availability active命令将node2重新激活：

$ docker node update --availability active node2
node2$ docker node inspect --pretty node2
ID:                     7kj69w9cdpyawh54fumibuu0u
Hostname:               node2
Joined at:              2023-10-12 03:40:28.774753447 +0000 utc
Status:State:                 ReadyAvailability:          ActiveAddress:               192.168.0.12
。。。。。。