- 操作系统版本：CentOS 7
- 内存：物理内存8G。虚拟机四个节点，node01 4G, node02 2G, node03 1G, node04 1G。每个节点交换分区大小8G
- 硬盘：各虚拟机节点硬盘大小200G
https://zaixianwangyebianji.bmcx.com/
各应用环境变量都配置在 ~/.bashrc 下
| 组件 | node01 |
node02 |
node03 |
node04 |
|
hadoop2.10.2 |
NN(主) |
NN(备) |
|
|
|
DN |
DN |
DN |
DN |
|
|
ZK |
ZK |
ZK |
|
|
|
ZKFC |
ZKFC |
|
|
|
|
JNN |
JNN |
JNN |
|
|
|
RS |
RS |
|
|
|
|
NM |
NM |
NM |
|
|
|
apache-hive-2.3.9-bin |
服务端 | 客户端 |
|
|
|
kafka_2.13-2.5.0，2.13是scala的版本号，2.5.0是kafka的版本号 |
kafka |
kafka |
|
|
| spark-3.0.0 | YARN模式 |
|
|
|
| Java 1.8.0_331 | java |
java |
java |
java |
| mysql 5.7 | Server/Client |
|
|
|
| netcat 7.50 | nc |
|
|
|
|
zookeeper-3.4.11 |
zk | zk |
zk |
|
|
apache-flume-1.9.0 |
flume 采集 |
flume消费 |
|
|
|
sqoop-1.4.7.bin__hadoop-2.6.0 |
sqoop |
|
|
|
|
hbase-1.3.6 |
HMaster(主) |
HMaster(主) |
HRegionServer |
HRegionServer |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|