kubernetes调试利器——kubectl debug工具

  通常情况下，业务容器所使用的镜像是非常精简的，而一旦业务容器出现问题，通过kubectl exec进入到容器时，我们会发现自己需要使用的工具都没有，也无法通过apt, apt-get, yum等包管理工具下载需要的工具。

  想要解决这个尴尬的窘境，有两种手段，其一是提前把需要使用的工具打入到镜像当中，除了问题我们可以随时进行debug。其二是利用kubectl debug工具。显然，第一种方式有很多弊端，譬如业务容器镜像过大，占用磁盘空间更多；另外每个人使用的工具可能会不同，我们不可能把所有的工具都打入到镜像当中，这是极不合理的。 而如果我们能够把需要的工具打入到一个debug镜像当中，需要的时候如果能把这个debug镜像跑起来，并且attach到我们需要排查问题的业务容器上，同时这两个容器可以共享network, pid名称空间的话，就能很好的解决这个问题。而恰好kubectl debug就有这样的功能

  kubectl debug命令的帮助文档如下

root@k8s-master1:~# kubectl debug --help
Debug cluster resources using interactive debugging containers.'debug' provides automation for common debugging tasks for cluster objects identified by resource and name. Pods will
be used by default if no resource is specified.The action taken by 'debug' varies depending on what resource is specified. Supported actions include:*  Workload: Create a copy of an existing pod with certain attributes changed, for example changing the image tag to a
new version.*  Workload: Add an ephemeral container to an already running pod, for example to add debugging utilities without
restarting the pod.*  Node: Create a new pod that runs in the node's host namespaces and can access the node's filesystem.Examples:# Create an interactive debugging session in pod mypod and immediately attach to it.kubectl debug mypod -it --image=busybox# Create an interactive debugging session for the pod in the file pod.yaml and immediately attach to it.# (requires the EphemeralContainers feature to be enabled in the cluster)kubectl debug -f pod.yaml -it --image=busybox# Create a debug container named debugger using a custom automated debugging image.kubectl debug --image=myproj/debug-tools -c debugger mypod# Create a copy of mypod adding a debug container and attach to itkubectl debug mypod -it --image=busybox --copy-to=my-debugger# Create a copy of mypod changing the command of mycontainerkubectl debug mypod -it --copy-to=my-debugger --container=mycontainer -- sh# Create a copy of mypod changing all container images to busyboxkubectl debug mypod --copy-to=my-debugger --set-image=*=busybox# Create a copy of mypod adding a debug container and changing container imageskubectl debug mypod -it --copy-to=my-debugger --image=debian --set-image=app=app:debug,sidecar=sidecar:debug# Create an interactive debugging session on a node and immediately attach to it.# The container will run in the host namespaces and the host's filesystem will be mounted at /hostkubectl debug node/mynode -it --image=busyboxOptions:--arguments-only=false:If specified, everything after -- will be passed to the new container as Args instead of Command.--attach=false:If true, wait for the container to start running, and then attach as if 'kubectl attach ...' were called.Default false, unless '-i/--stdin' is set, in which case the default is true.-c, --container='':Container name to use for debug container.--copy-to='':Create a copy of the target Pod with this name.--env=[]:Environment variables to set in the container.-f, --filename=[]:identifying the resource to debug--image='':Container image to use for debug container.--image-pull-policy='':The image pull policy for the container. If left empty, this value will not be specified by the client anddefaulted by the server.--profile='legacy':Debugging profile. Options are "legacy", "general", "baseline", "netadmin", or "restricted".-q, --quiet=false:If true, suppress informational messages.--replace=false:When used with '--copy-to', delete the original Pod.--same-node=false:When used with '--copy-to', schedule the copy of target Pod on the same node.--set-image=[]:When used with '--copy-to', a list of name=image pairs for changing container images, similar to how 'kubectlset image' works.--share-processes=true:When used with '--copy-to', enable process namespace sharing in the copy.-i, --stdin=false:Keep stdin open on the container(s) in the pod, even if nothing is attached.--target='':When using an ephemeral container, target processes in this container name.-t, --tty=false:Allocate a TTY for the debugging container.Usage:kubectl debug (POD | TYPE[[.VERSION].GROUP]/NAME) [ -- COMMAND [args...] ] [options]Use "kubectl options" for a list of global command-line options (applies to all commands).

  想要完成上述功能，主要是利用--target参数，这个参数主要用于指定debug Pod中的哪个容器;--image参数就是用于指定使用哪个镜像来debug，这个镜像包含我们需要使用的工具即可。用法如下：

root@k8s-master1:~# kubectl get pods
NAME                                READY   STATUS    RESTARTS       AGE
mysql-5578b9475d-5fc8d              1/1     Running   1 (130m ago)   156m
nginx-deployment-775b6549b5-bgdfp   1/1     Running   2 (130m ago)   3h39m
nginx-deployment-775b6549b5-ghz9t   1/1     Running   2 (130m ago)   3h39m
nginx-deployment-775b6549b5-pcw82   1/1     Running   1 (130m ago)   167m
root@k8s-master1:~#
root@k8s-master1:~# kubectl debug mysql-5578b9475d-5fc8d --image=ubuntu:20.04 -it --target=mysql
Targeting container "mysql". If you don't see processes from this container it may be because the container runtime doesn't support this feature.
Defaulting debug container name to debugger-jlw5l.
If you don't see a command prompt, try pressing enter.
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/# ps -aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
999            1  0.0  4.8 1163028 195532 ?      Ssl  03:26   0:07 mysqld
root         128  0.0  0.0   4248  3256 pts/0    Ss   05:40   0:00 bash
root         136  0.0  0.0   5900  2780 pts/0    R+   05:41   0:00 ps -aux
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/#
root@mysql-5578b9475d-5fc8d:/# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st1  0      0 1111892   2120 1823828    0    0    42   346  696 1192  1  4 95  0  0
root@mysql-5578b9475d-5fc8d:/# pidstt