本地部署Hive集群

规划

服务	机器
Hive本体	部署在Node1
元数据服务所需的关系型数据库(MYSQL)	部署在Node1

安装MYSQL数据库

# 更新密钥
rpm --import https://repo.mysql.com/RPM-GPG-KEY-mysql-2022# 安装Mysql yum库
rpm -Uvh http://repo.mysql.com//mysql57-community-release-el7-7.noarch.rpm# yum安装MysqL
yum -y install mysql-community-server# 启动MySQL并跳过授权表，使用--user=mysql选项来指定MySQL以mysql用户身份运行
mysqld --skip-grant-tables --skip-networking --user=mysql &#  使用mysql登录并重置密码
mysql -uroot
USE mysql;
set global validate_password_policy=LOW;
set global validate_password_length=4;
ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
grant all privileges on *.* to root@"%" identified by 'root' with grant option;
FLUSH PRIVILEGES;# 重启mysql并重新登录
systemctl stop mysqld
systemctl start mysqld
systemctl enable mysqldmysql -uroot -p

配置Hadoop

 Hive的运行依赖于Hadoop(HDFS、Mapreduce、YARN都依赖)
同时涉及到hDFS文件系统的访问，所以需要配置Hadoop的代理用户
即设置hadoop用户允许代理(模拟)其他用户

配置如下内容在Hadoop的core-site.xml中，并分发到其他节点，且重启HDFS集群

  <property><name>hadoop.proxyuser.hadoop.hosts</name><value>*</value></property><property><name>hadoop.proxyuser.hadoop.groups</name><value>*</value></property>

 下载解压Hive

hive和mysql驱动安装包位置：

apache-hive-3.1.3-bin.tar.gzhttps://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gzmysql-connector-java-5.1.34.jarhttps://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.34/mysql-connector-java-5.1.34.jar

tar -zxf apache-hive-3.1.3-bin.tar.gz -C /export/server/mv mysql-connector-java-5.1.34.jar  /export/server/apache-hive-3.1.3-bin/lib/ln -s /export/server/apache-hive-3.1.3-bin hive

配置Hive

在Hive的conf目录内，新建hive-env.sh文件，填入以下环境变量内容：

export HADOOP_HOME=/export/server/hadoop
export HIVE_CONF_DIR=/export/server/hive/conf
export HIVE_AUX_JARS_PATH=/export/server/hive/lib

在Hive的conf目录内，新建hive-site.xml文件，填入以下内容：

vim hive-site.xml<configuration>
<!-- 存储元数据mysql相关配置 -->
<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://node1:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false&amp;useUnicode=true&amp;characterEncoding=UTF-8</value>
</property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value>
</property><property><name>javax.jdo.option.ConnectionUserName</name><value>root</value>
</property><property><name>javax.jdo.option.ConnectionPassword</name><value>hadoop</value>
</property><!-- H2S运行绑定host -->
<property><name>hive.server2.thrift.bind.host</name><value>node1</value>
</property><!-- 远程模式部署metastore metastore地址 --> # 远程模式，metastore必须单独配置
<property><name>hive.metastore.uris</name><value>thrift://node1:9083</value>
</property><!-- 关闭元数据存储授权  --> 
<property><name>hive.metastore.event.db.notification.api.auth</name><value>false</value>
</property>
</configuration>

初始化元数据库

mysql -uroot -p123456
create database hive charset utf8;
exit;cd /export/server/hive/bin
./schematool -initSchema -dbType mysql -verbos

启动Hive(使用Hadoop用户)

chown -R hadoop:hadoop apache-hive-3.1.3-bin hive
cd /export/server/hive/
mkdir logs
nohup bin/hive --service metastore  >> logs/metastore.log 2>&1 &
cd ../bin
./hive