Ambari大数据集群快速搭建(二)
上一节修改好基本配置后,接下来开始安装集群部署所需依赖。
修改搭建集群前的基本配置及环境依赖
-
三台虚拟机关闭防火墙并查看状态
systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld

-
配置三台机器免密互信
主节点创建公钥,并将公钥放入keysssh-keygen -t dsa -P ‘’ -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
从节点创建接收文件夹
mkdir ~/.ssh
主节点发送公钥给两个从节点
scp ~/.ssh/authorized_keys root@siger-slave01.test.com:~/.ssh/authorized_keys
scp ~/.ssh/authorized_keys root@siger-slave02.test.com:~/.ssh/authorized_keys
所有主机执行
chmod 600 ~/.ssh/authorized_keys
-
安装jdk
移动刚才上传的jdk安装包到指定目录并解压mv /opt/java /usr/local/
tar -zxvf /usr/local/java/jdk-8u102-linux-x64.tar.gz

通过scp将解压后的jdk传到其他slave节点
scp -r /usr/local/java/ root@siger-slave01.test.com:/usr/local/
scp -r /usr/local/java/ root@siger-slave02.test.com:/usr/local/所有节点编辑环境变量
vim /etc/profile
在文件最后增加以下内容
JAVA_HOME=/usr/local/java/jdk1.8.0_102 JRE_HOME=/usr/local/java/jdk1.8.0_102/jre PATH=$PATH:/sbin/:$JAVA_HOME/bin:$CATALINA_HOME/bin CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar: export JAVA_HOME JRE_HOME PATH CLASSPATH
所有节点更改后,编译环境变量并查看jdk是否安装成功. /etc/profile
java
javac
java -version

-
python3环境安装
移动安装包到指定目录下mv /opt/python3 /usr/local/
所有节点下操作相同,安装所需依赖(96M左右)
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gcc make
主节点进入Python3安装包目录,解压,编译安装
cd /usr/local/python3
tar -xvJf Python-3.6.2.tar.xz
cd Python-3.6.2
./configure prefix=/usr/local/python3 --with-ssl
make && make install



从master节点上分发给所有slave节点
scp -r /usr/local/python3/ root@siger-slave01.test.com:/usr/local/
scp -r /usr/local/python3/ root@siger-slave02.test.com:/usr/local/所有节点操作相同,建立软链接,并查看
ln -s /usr/local/python3/bin/python3 /usr/bin/python3
ln -s /usr/local/python3/bin/pip3 /usr/bin/pip3
python3 -V
pip3 -V

-
mysql安装
主节点操作,安装mysql存储ambari集群元文件。
查看是否有其他数据库服务rpm -qa|grep -i mysql
rpm -qa|grep -i MariaDB

卸载其他数据库服务,如果不卸载,安装mysql会出错。
rpm -ev mariadb-libs-5.5.56-2.el7.x86_64 --nodeps
移动安装包到安装目录并解压
mv /opt/mysql /usr/local/
cd /usr/local/mysql/
tar -xvf mysql-5.7.19-1.el7.x86_64.rpm-bundle.tar

逐个安装
rpm -ivh mysql-community-libs-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-client-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-libs-compat-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-common-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-minimal-debuginfo-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-devel-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-server-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-embedded-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-server-minimal-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-embedded-compat-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-test-5.7.19-1.el7.x86_64.rpm --nodeps
rpm -ivh mysql-community-embedded-devel-5.7.19-1.el7.x86_64.rpm --nodeps安装完成启动mysql
service mysqld start
设置开机启动项
systemctl enable mysqld.service
查看初始密码并设置新密码
grep “password” /var/log/mysqld.log
mysql -u root -p
set password = password(‘123456’);

创建mysql用户及权限
create database ambari character set utf8;
CREATE USER ‘ambari’@’%‘IDENTIFIED BY ‘Siger_123’;
GRANT ALL PRIVILEGES ON . TO ‘ambari’@’%’;
CREATE USER ‘ambari’@'localhost’IDENTIFIED BY ‘Siger_123’;
GRANT ALL PRIVILEGES ON . TO ‘ambari’@‘localhost’;
FLUSH PRIVILEGES;
基本工作做好了,开始集群搭建。
Ambari大数据集群快速搭建(三)
魔乐社区(Modelers.cn) 是一个中立、公益的人工智能社区,提供人工智能工具、模型、数据的托管、展示与应用协同服务,为人工智能开发及爱好者搭建开放的学习交流平台。社区通过理事会方式运作,由全产业链共同建设、共同运营、共同享有,推动国产AI生态繁荣发展。
更多推荐


所有评论(0)