Shell脚本之——Hadoop3单机版安装
目录
1.解压
2.文件重命名
3.配置环境变量
4.hadoop-env.sh
5.core-site.xml
6. hdfs-site.xml
7. mapred-site.xml
8.yarn-site.xml
9.完整脚本代码(注意修改主机名)
10.重启环境变量
11.初始化
12.启动服务
13.jps查询节点
1.解压
tar -zxf /opt/install/hadoop-3.1.3.tar.gz -C /opt/soft/
2.文件重命名
mv /opt/soft/hadoop-3.1.3 /opt/soft/hadoop313
3.配置环境变量
sed -i '73a\export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop' /etc/profilesed -i '73a\export HADOOP_LIBEXEC_DIR=$HADOOP_HOME/libexec' /etc/profilesed -i '73a\export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native' /etc/profilesed -i '73a\export HADOOP_INSTALL=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_YARN_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_HDFS_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_COMMON_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_MAPRED_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export YARN_NODEMANAGER_USER=root' /etc/profilesed -i '73a\export YARN_RESOURCEMANAGER_USER=root' /etc/profilesed -i '73a\export HDFS_ZKFC_USER=root' /etc/profilesed -i '73a\export HDFS_JOURNALNODE_USER=root' /etc/profilesed -i '73a\export HDFS_SECONDARYNAMENODE_USER=root' /etc/profilesed -i '73a\export HDFS_DATANODE_USER=root' /etc/profilesed -i '73a\export HDFS_NAMENODE_USER=root' /etc/profilesed -i '73a\export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_HOME/lib' /etc/profilesed -i '73a\export HADOOP_HOME=/opt/soft/hadoop313' /etc/profilesed -i '73a\#HADOOP_HOME' /etc/profile
4.hadoop-env.sh
#hadoop-env.shsed -i '53a\export JAVA_HOME=/opt/soft/jdk180' /opt/soft/hadoop313/etc/hadoop/hadoop-env.sh
5.core-site.xml
#core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>*</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.proxyuser.root.groups</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>*</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.proxyuser.root.hosts</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <description>读写队列缓存:128k</description>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>131072</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>io.file.buffer.size</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>root</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.http.staticuser.user</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <description>namenode上本地的hadoop临时文件夹</description>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.tmp.dir</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>hdfs://cp145:9000</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>fs.defaultFS</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xml
6. hdfs-site.xml
#hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>关闭权限验证</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>false</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.permissions.enabled</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>datanode上数据块的物理存储位置目录</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data/dfs/data</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.datanode.data.dir</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>namenode上存储hdfsq名字空间元数据的目录</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data/dfs/name</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.namenode.name.dir</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>hadoop中每一个block文件的备份数量</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>1</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.replication</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xml
7. mapred-site.xml
#mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>1024</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.reduce.memory.mb</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>1024</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.map.memory.mb</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>cp145:19888</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.jobhistory.webapp.address</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>cp145:10020</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.jobhistory.address</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/etc/hadoop:/opt/soft/hadoop313/share/hadoop/common/lib/*:/opt/soft/hadoop313/share/hadoop/common/*:/opt/soft/hadoop313/share/hadoop/hdfs/*:/opt/soft/hadoop313/share/hadoop/hdfs/lib/*:/opt/soft/hadoop313/share/hadoop/mapreduce/*:/opt/soft/hadoop313/share/hadoop/mapreduce/lib/*:/opt/soft/hadoop313/share/hadoop/yarn/*:/opt/soft/hadoop313/share/hadoop/yarn/lib/*</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.application.classpath</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <final>true</final>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <description>job执行框架:local,classic or yarn</description>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>yarn</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.framework.name</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xml
8.yarn-site.xml
#yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>/opt/soft/hadoop313/yarndata/log</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.log-dirs</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>/opt/soft/hadoop313/yarndata/yarn</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.local-dirs</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>mapreduce_shuffle</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.aux-services</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8042</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.webapp.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8050</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8040</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.localizer.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.resourcemanager.scheduler.class</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>20000</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.resourcemanager.connect.retry-interval.ms</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xml
9.完整脚本代码(注意修改主机名)
#! /bin/bash
echo 'auto install hadoop begining....'#global var
hadoop=trueif [ "$hadoop" = true ];thenecho 'hadoop install set true'echo 'setup hadoop 3'tar -zxf /opt/install/hadoop-3.1.3.tar.gz -C /opt/soft/mv /opt/soft/hadoop-3.1.3 /opt/soft/hadoop313sed -i '73a\export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop' /etc/profilesed -i '73a\export HADOOP_LIBEXEC_DIR=$HADOOP_HOME/libexec' /etc/profilesed -i '73a\export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native' /etc/profilesed -i '73a\export HADOOP_INSTALL=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_YARN_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_HDFS_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_COMMON_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export HADOOP_MAPRED_HOME=$HADOOP_HOME' /etc/profilesed -i '73a\export YARN_NODEMANAGER_USER=root' /etc/profilesed -i '73a\export YARN_RESOURCEMANAGER_USER=root' /etc/profilesed -i '73a\export HDFS_ZKFC_USER=root' /etc/profilesed -i '73a\export HDFS_JOURNALNODE_USER=root' /etc/profilesed -i '73a\export HDFS_SECONDARYNAMENODE_USER=root' /etc/profilesed -i '73a\export HDFS_DATANODE_USER=root' /etc/profilesed -i '73a\export HDFS_NAMENODE_USER=root' /etc/profilesed -i '73a\export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_HOME/lib' /etc/profilesed -i '73a\export HADOOP_HOME=/opt/soft/hadoop313' /etc/profilesed -i '73a\#HADOOP_HOME' /etc/profile#hadoop-env.shsed -i '53a\export JAVA_HOME=/opt/soft/jdk180' /opt/soft/hadoop313/etc/hadoop/hadoop-env.sh#core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>*</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.proxyuser.root.groups</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>*</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.proxyuser.root.hosts</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <description>读写队列缓存:128k</description>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>131072</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>io.file.buffer.size</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>root</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.http.staticuser.user</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <description>namenode上本地的hadoop临时文件夹</description>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>hadoop.tmp.dir</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <value>hdfs://cp145:9000</value>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <name>fs.defaultFS</name>' /opt/soft/hadoop313/etc/hadoop/core-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/core-site.xml#hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>关闭权限验证</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>false</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.permissions.enabled</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>datanode上数据块的物理存储位置目录</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data/dfs/data</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.datanode.data.dir</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>namenode上存储hdfsq名字空间元数据的目录</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/data/dfs/name</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.namenode.name.dir</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <description>hadoop中每一个block文件的备份数量</description>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <value>1</value>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <name>dfs.replication</name>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/hdfs-site.xml#mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>1024</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.reduce.memory.mb</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>1024</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.map.memory.mb</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>cp145:19888</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.jobhistory.webapp.address</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>cp145:10020</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.jobhistory.address</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>/opt/soft/hadoop313/etc/hadoop:/opt/soft/hadoop313/share/hadoop/common/lib/*:/opt/soft/hadoop313/share/hadoop/common/*:/opt/soft/hadoop313/share/hadoop/hdfs/*:/opt/soft/hadoop313/share/hadoop/hdfs/lib/*:/opt/soft/hadoop313/share/hadoop/mapreduce/*:/opt/soft/hadoop313/share/hadoop/mapreduce/lib/*:/opt/soft/hadoop313/share/hadoop/yarn/*:/opt/soft/hadoop313/share/hadoop/yarn/lib/*</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.application.classpath</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ </property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <final>true</final>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <description>job执行框架:local,classic or yarn</description>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <value>yarn</value>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <name>mapreduce.framework.name</name>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xmlsed -i '19a\ <property>' /opt/soft/hadoop313/etc/hadoop/mapred-site.xml#yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>/opt/soft/hadoop313/yarndata/log</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.log-dirs</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>/opt/soft/hadoop313/yarndata/yarn</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.local-dirs</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>mapreduce_shuffle</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.aux-services</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8042</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.webapp.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8050</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>cp145:8040</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.nodemanager.localizer.address</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.resourcemanager.scheduler.class</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ </property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <value>20000</value>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <name>yarn.resourcemanager.connect.retry-interval.ms</name>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlsed -i '17a\ <property>' /opt/soft/hadoop313/etc/hadoop/yarn-site.xmlecho 'setup hadoop 3 success!!!'
fi
10.重启环境变量
[root@lxm148 opt]# source /etc/profile
11.初始化
[root@lxm148 yarndata]# hdfs namenode -format
12.启动服务
[root@lxm148 yarndata]# start-all.sh
13.jps查询节点

相关文章:
Shell脚本之——Hadoop3单机版安装
目录 1.解压 2.文件重命名 3.配置环境变量 4.hadoop-env.sh 5.core-site.xml 6. hdfs-site.xml 7. mapred-site.xml 8.yarn-site.xml 9.完整脚本代码(注意修改主机名) 10.重启环境变量 11.初始化 12.启动服务 13.jps查询节点 1.解压 tar -zxf /opt/install/hadoo…...
代码随想录NO39 |0-1背包问题理论基础 416.分割等和子集
0-1背包问题理论基础 分割等和子集1. 0-1背包问题理论基础(二维数组实现)2. 0-1背包问题理论基础 二(一维数组实现)1. 0-1背包问题理论基础(二维数组实现) 背包问题一般分为这几种: 0-1背包问题:有n件物品和一个最多能背重量为w…...
FITC-PEG-FA,荧光素-聚乙二醇-叶酸,FA-PEG-FITC,实验室科研试剂,提供质量检测
FITC-PEG-FA,荧光素-聚乙二醇-叶酸 中文名称:荧光素-聚乙二醇-叶酸 英文名称:FITC-PEG-FA 英文别名:Fluorescein-PEG-Folic Acid 性状:基于不同的分子量,呈白色/类白色固体,或粘稠液体。 溶…...
简洁易懂:源码+实战讲解Redisson并发锁及看门狗自动续期
1 缘起 有一次同事问Redisson存储的键是否为hash? 我当时,没有看Redisson的相关源码,只知道应用, 所以没有办法回答,于是开始看看Redisson实现的源码, 顺便写了一个单机Redisson测试, 发现Redi…...
TCP 三次握手和四次挥手
✏️作者:银河罐头 📋系列专栏:JavaEE 🌲“种一棵树最好的时间是十年前,其次是现在” 目录TCP 建立连接(三次握手)为啥不能是 4 次?为啥不能是 2 次?三次握手的意义:TCP 断开连接(四…...
JavaWeb复习
JavaWeb复习一.概述1.概念2.B/S和C/S 架构二.HTTP通信协议概述1.概念2.HTTP1.0 与 HTTP1.1 版本3.HTTP 协议组成4.常见状态码5.GET 与 POST 请求方式三.Tomcat1.Web服务器介绍2.安装(Windows)3.Tomcat目录结构4.server.xml部分配置解释四.Servlet1.概念2…...
P14 PyTorch AutoGrad
前言:激活函数与loss的梯度PyTorch 提供了Auto Grad 功能,这里系统讲解一下torch.autograd.grad系统的工作原理,了解graph 结构目录:1: require_grad False2: require_grad True3: 多层bakcward 原理4: in…...
前端报表如何实现无预览打印解决方案或静默打印
在前端开发中,除了将数据呈现后,我们往往需要为用户提供,打印,导出等能力,导出是为了存档或是二次分析,而打印则因为很多单据需要打印出来作为主要的单据来进行下一环节的票据支撑, 而前端打印可…...
Operating System Course 2 - My OS
Computer Startup process上一篇:http://t.csdn.cn/XfUKt 讲到这个启动设备的第一个扇区:引导扇区。那么引导扇区的代码长什么样子?这里得看引导扇区代码源文件bootsect.s(.s后缀文件为用汇编语言编写的源代码文件)。另…...
离散数学 课时一 命题逻辑的基本概念
1 命题 1、命题:可以判断其真值的陈述句 2、真值:真或者假(1或者0) 3、真命题:真值为真的命题 4、假命题:真值为假的命题 5、原子命题:不可以再被分解成更简单的命题 6、复合命题:由原子命题通过联结词联结…...
Word文档带有权限密码怎么办?
Word文档的权限密码指的是什么?其实这是Word文档的保护方法之一,具体指Word文档的编辑、修改受到了限制,需要输入密码才能进行。 设置了权限密码的Word文档还是可以直接打开,只有当需要编辑或者修改内容的时候,才会发…...
C++多态
1. 多态的概念1.1 概念多态的概念:通俗来说,就是多种形态,具体点就是去完成某个行为,当不同的对象去完成时会产生出不同的状态举个例子:比如买票这个行为,当普通人买票时,是全价买票;…...
访问学者如何申请美国J1签证?
一、申请美国J1签证的步骤: 第一步:填写I901表。 填写I901表会收取SERVIS费用180美元,可以用VISA/Master卡直接网上支付。填完后打印收据单或者存成PDF后续再打印,记下I901收据编号。 第二步:DS-160表填写。 填写DS-…...
使用gitlab ci/cd来发布一个.net 项目
gitlab runner的安装和基本使用:https://bear-coding.blog.csdn.net/article/details/120591711安装并给项目配置完gitlab runner后再操作后面步骤。实现目标:master分支代码有变更的时候自动构建build。当开发人员在gitlab上给项目打一个tag标签分支的时候自动触发…...
笔试题-2023-蔚来-数字芯片设计【纯净题目版】
回到首页:2023 数字IC设计秋招复盘——数十家公司笔试题、面试实录 推荐内容:数字IC设计学习比较实用的资料推荐 题目背景 笔试时间:2022.08.24应聘岗位:校招-芯片逻辑综合工程师-智能硬件笔试时长:90min笔试平台:nowcoder牛客网题目类型:不定项选择题(15道)、填空题…...
ThreadLocal 详解
ThreadLocal简介JDK源码对ThreadLocal类的注释如下:ThreadLocal提供线程局部变量,使得每个线程都有自己的、独立初始化的变量副本ThreadLocal实例通常是类中的private static字段,用于将状态与线程相关联,如用户ID、事务ID只要线程…...
【Java 面试合集】重写以及重载有什么区别能简单说说嘛
重写以及重载有什么区别能简单说说嘛 前述 这是一道非常基础的面试题,我们在回答的过程中一定要逐一横向比较。 从方法的 修饰符,返回值,方法名,含义,参数等方面进行逐一分析来比较不同。 话不多话,看下…...
到底什么是股票委托接口?
在量化股票市场上,常见的股票委托接口其实有着不一样的交集,就拿股票交易接口,在量化股票跟程序化交易中,有共同之处就是在于直接委托执行下单,并且能很快的就能够将策略输出在账户持仓数据中,继续缓存下来…...
Linux驱动:VPU
1. 前言 限于作者能力水平,本文可能存在谬误,因此而给读者带来的损失,作者不做任何承诺。 2. 概述 VPU 是用来进行图像、视频数据进行硬件编、解码的硬件模块。内部集成了 Encoder、Decoder 功能部件进行图像、视频数据进行硬件编、解码&a…...
简介Servlet
目录 一、maven中心库 二、简介Servlet 三、实现Servlet动态页面 1、创建一个maven项目 2、引入依赖 3、创建目录结构 4、编写Servlet代码 5、打包 6、部署 7、验证程序 四、Servlet的运行原理 五、Tomcat伪代码 1、Tomcat初始化 a、让Tomcat先从指定的目录…...
Qwen3.5-4B-Claude-Opus保姆级教程:Web界面响应延迟归因与优化路径
Qwen3.5-4B-Claude-Opus保姆级教程:Web界面响应延迟归因与优化路径 1. 模型与部署环境概览 Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled-GGUF是基于Qwen3.5-4B的推理蒸馏模型,特别强化了结构化分析、分步骤回答以及代码与逻辑类问题的处理能力。该…...
终极免费EVE舰船配置神器:Pyfa完整实战指南
终极免费EVE舰船配置神器:Pyfa完整实战指南 【免费下载链接】Pyfa Python fitting assistant, cross-platform fitting tool for EVE Online 项目地址: https://gitcode.com/gh_mirrors/py/Pyfa 在EVE Online这个充满挑战的宇宙中,打造一艘完美的…...
寻音捉影·侠客行多场景落地:覆盖会议/媒体/司法/金融/教育五大垂直领域
寻音捉影侠客行多场景落地:覆盖会议/媒体/司法/金融/教育五大垂直领域 1. 产品核心功能解析 寻音捉影侠客行是一款基于先进语音识别技术的音频关键词检索工具,它能够像江湖中的隐士高手一样,在浩瀚的音频海洋中精准定位特定关键词。这款工具…...
FOC算法避坑指南:克拉克变换的‘等幅值’与‘等功率’到底怎么选?基于STM32的实测对比
FOC算法避坑指南:克拉克变换的‘等幅值’与‘等功率’到底怎么选?基于STM32的实测对比 在STM32平台上实现磁场定向控制(FOC)时,克拉克变换系数的选择往往让工程师陷入两难:究竟该用2/3(等幅值&…...
OneAgent智能体全球发布会圆满落幕:引领金融AI交易新时代
2026年3月25日,聚焦金融AI领域的盛会《OneAgent智能体全球产品发布会》在中国杭州成功落幕。本次发布会吸引了全球金融科技领域的行业专家、投资机构以及技术爱好者的关注,标志着OneAgent在全球AI金融市场的战略布局正式启动。AI原生对冲交易新物种&…...
Windows服务器部署:OpenClaw守护进程+Qwen3-32B镜像长期运行
Windows服务器部署:OpenClaw守护进程Qwen3-32B镜像长期运行 1. 为什么需要服务器级部署? 去年我尝试在个人笔记本上运行OpenClaw时,经常遇到两个头疼的问题:一是夜间执行任务时电脑休眠导致流程中断,二是长时间运行后…...
保姆级教程:给你的ClickHouse 23.8装上‘仪表盘’(Prometheus+Grafana监控配置详解)
ClickHouse 23.8全链路监控实战:从Prometheus埋点到Grafana告警设计 当你的ClickHouse集群查询延迟突然从200ms飙升到5秒,是内存不足?磁盘IO瓶颈?还是并发查询堆积?本文将带你构建一套完整的监控告警体系,让…...
HP-Socket开发者技能认证考试大纲更新全指南:周期解析与参与攻略
HP-Socket开发者技能认证考试大纲更新全指南:周期解析与参与攻略 【免费下载链接】HP-Socket High Performance TCP/UDP/HTTP Communication Component 项目地址: https://gitcode.com/gh_mirrors/hp/HP-Socket HP-Socket作为高性能TCP/UDP/HTTP通信组件&…...
动态规划详解:从入门到精通,这四个案例让你彻底掌握DP思想
面试必考、算法进阶的核心,一篇文章帮你打通任督二脉在算法学习的过程中,动态规划(Dynamic Programming,简称DP)绝对是让很多人头疼的一个难点。很多初学者看到DP问题就发怵,其实只要掌握了核心思想&#x…...
燃油车虎视眈眈,电车涨价的图谋必将落空,油价上涨的利好将消失
近期以来多家电车企业涨价,美国电车涨价尤为明显,最高涨幅2万元,而国产电车涨价3000-1.4万元不等,凸显出电车似乎突然间对市场乐观起来,导致他们信心十足的在于3月份以来的油价上涨,但是这种涨价将迅速导致…...
