CentOS 7单机伪集群部署DolphinScheduler 3.2.2：从零搭建可视化调度平台

1. 环境准备打造DolphinScheduler的温床在CentOS 7上部署DolphinScheduler伪集群就像给新房子打地基。我遇到过不少初学者在环境配置阶段就翻车最常见的就是JDK版本不对或者数据库权限没开。咱们先从最基础的开始把地基打牢。JDK安装是第一个门槛。我强烈建议用Oracle JDK 1.8实测OpenJDK偶尔会有奇怪的兼容性问题。安装完记得配置环境变量这个老生常谈但总有人栽跟头# 检查现有Java版本 java -version # 如果已有旧版本先卸载 yum remove -y java-* # 安装Oracle JDK 1.8 wget https://download.oracle.com/otn-pub/java/jdk/8u341-b10/jdk-8u341-linux-x64.tar.gz tar -zxvf jdk-8u341-linux-x64.tar.gz -C /usr/local/接着在/etc/profile末尾添加export JAVA_HOME/usr/local/jdk1.8.0_341 export PATH$JAVA_HOME/bin:$PATHMySQL 5.7安装更是个技术活。有次我帮客户排查问题发现他们用MariaDB导致调度任务莫名卡死。建议直接用官方MySQL别图省事用系统自带# 添加MySQL官方源 rpm -Uvh https://dev.mysql.com/get/mysql80-community-release-el7-7.noarch.rpm # 禁用8.0源启用5.7源 yum-config-manager --disable mysql80-community yum-config-manager --enable mysql57-community # 安装服务端和客户端 yum install -y mysql-community-server mysql-community-client安装完记得初始化数据库并创建专用账号CREATE DATABASE ds CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE USER ds_user% IDENTIFIED BY Ds_123456; GRANT ALL PRIVILEGES ON ds.* TO ds_user%; FLUSH PRIVILEGES;2. ZooKeeper部署集群的神经中枢ZooKeeper在伪集群里扮演着关键角色就像乐队的指挥。虽然单机部署简单但配置不当会导致任务状态同步异常。我推荐用3.8.4版本去年有个客户用3.4.x版本就遇到过选举死锁问题。安装过程其实很标准wget https://archive.apache.org/dist/zookeeper/zookeeper-3.8.4/apache-zookeeper-3.8.4-bin.tar.gz tar -zxvf apache-zookeeper-3.8.4-bin.tar.gz -C /usr/local/ mv /usr/local/apache-zookeeper-3.8.4-bin /usr/local/zookeeper配置文件才是重头戏。很多人直接复制样例文件结果漏掉关键参数。这是我的私藏配置模板zoo.cfgtickTime2000 initLimit10 syncLimit5 dataDir/usr/local/zookeeper/data dataLogDir/usr/local/zookeeper/logs clientPort2181 # 伪集群模式下这个很关键 maxClientCnxns60 autopurge.snapRetainCount3 autopurge.purgeInterval24启动前记得创建数据目录mkdir -p /usr/local/zookeeper/{data,logs} echo 1 /usr/local/zookeeper/data/myid服务管理有讲究建议用systemd托管cat /etc/systemd/system/zk.service EOF [Unit] DescriptionZooKeeper Service Afternetwork.target [Service] Typeforking ExecStart/usr/local/zookeeper/bin/zkServer.sh start ExecStop/usr/local/zookeeper/bin/zkServer.sh stop ExecReload/usr/local/zookeeper/bin/zkServer.sh restart Userroot Grouproot [Install] WantedBymulti-user.target EOF验证服务是否正常/usr/local/zookeeper/bin/zkCli.sh -server localhost:2181 [zk: localhost:2181(CONNECTED) 0] ls /3. DolphinScheduler核心部署终于到主角登场了3.2.2版本相比之前有个大改进——支持轻量级部署。不过伪集群配置还是有不少坑我去年在客户现场调试了整整两天才摸清门道。解压安装包时要注意权限问题wget https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-bin.tar.gz tar -zxvf apache-dolphinscheduler-3.2.2-bin.tar.gz -C /usr/local/ mv /usr/local/apache-dolphinscheduler-3.2.2-bin /usr/local/dolphinscheduler专用用户创建这一步很多人偷懒直接用root后面权限问题能让你怀疑人生useradd dolphinscheduler echo dolphinscheduler123 | passwd --stdin dolphinscheduler echo dolphinscheduler ALL(ALL) NOPASSWD: ALL /etc/sudoers chown -R dolphinscheduler:dolphinscheduler /usr/local/dolphinschedulerSSH免密登录是伪集群部署的关键。有次我遇到个奇葩问题发现是authorized_keys权限不对su - dolphinscheduler ssh-keygen -t rsa -P -f ~/.ssh/id_rsa cat ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys # 测试是否生效 ssh localhost环境配置文件dolphinscheduler_env.sh要特别注意这些参数export JAVA_HOME/usr/local/jdk1.8.0_341 export DATABASEmysql export SPRING_DATASOURCE_URLjdbc:mysql://localhost:3306/ds?useUnicodetruecharacterEncodingUTF-8 export SPRING_DATASOURCE_USERNAMEds_user export SPRING_DATASOURCE_PASSWORDDs_123456 export REGISTRY_ZOOKEEPER_CONNECT_STRINGlocalhost:2181install_env.sh这个文件决定伪集群的部署形态重点看这几个参数ipslocalhost masterslocalhost workerslocalhost:default alertServerlocalhost apiServerslocalhost installPath/usr/local/dolphinscheduler deployUserdolphinscheduler4. 数据库初始化与服务启动到这一步就像组装乐高前面零件都准备好了现在开始拼装。数据库初始化最容易出问题我总结了三类常见错误字符集不对、权限不足、表结构冲突。初始化数据库前建议先备份mysql -uds_user -pDs_123456 -e DROP DATABASE IF EXISTS ds; CREATE DATABASE ds CHARACTER SET utf8 COLLATE utf8_general_ci; cd /usr/local/dolphinscheduler bash tools/bin/upgrade-schema.sh服务启动顺序有讲究我习惯用这个顺序# 先启动Master bash ./bin/dolphinscheduler-daemon.sh start master-server # 再启动Worker bash ./bin/dolphinscheduler-daemon.sh start worker-server # 接着启动API bash ./bin/dolphinscheduler-daemon.sh start api-server # 最后启动Alert bash ./bin/dolphinscheduler-daemon.sh start alert-server检查服务状态的小技巧tail -f /usr/local/dolphinscheduler/master-server/logs/master-server.log tail -f /usr/local/dolphinscheduler/worker-server/logs/worker-server.log登录Web UI后别忘了改默认密码http://服务器IP:12345/dolphinscheduler 初始账号admin/dolphinscheduler1235. 常见问题排查指南部署完成只是开始运维才是重头戏。根据我的实战经验90%的问题集中在以下场景端口冲突是最常见的特别是12345端口被占用netstat -tunlp | grep 12345 # 如果被占用可以修改api-server/conf/application.yaml中的server.portZooKeeper连接失败通常有三种可能ZooKeeper服务没启动防火墙没开2181端口dolphinscheduler_env.sh配置错误任务卡住的排查思路# 查看Master日志 grep -A 10 ERROR /usr/local/dolphinscheduler/master-server/logs/master-server.log # 检查ZooKeeper节点 /usr/local/zookeeper/bin/zkCli.sh -server localhost:2181 ls /dolphinscheduler性能调优参数建议# 在master-server/conf/master.properties中调整 master.exec.threads100 master.exec.task.num20 master.dispatch.task.num30 # 在worker-server/conf/worker.properties中调整 worker.exec.threads100 worker.heartbeat.interval10最后提醒下伪集群模式不适合生产环境。如果要做性能测试建议至少拆分成3个节点MasterAPI一个节点Worker单独一个节点AlertZooKeeperMySQL一个节点。