linuxsir首页 LinuxSir.Org | Linux、BSD、Solaris、Unix | 开源传万世,因有我参与欢迎您!
网站首页 | 设为首页 | 加入收藏
您所在的位置:主页 > Linux数据库 >

HBase高可用集群部署(CDH)

时间:2017-05-16  来源:未知  作者:linuxsir首页

一、概要

本文记录HBase高可用集群部署过程,在部署HBase之前需要事先部署好Hadoop集群,因为HBase的数据需要存放在hdfs上,Hadoop集群的部署后续会有一篇文章记录,本文假设Hadoop集群已经部署好,分布式HBase集群需要依赖zk,并且zk可以是HBase自己托管的也可以是我们自己单独搭建的,这里我们使用自己单独搭建的zk集群,我们的Hadoop集群是用的cdh的发行版,所以HBase也会使用cdh的源。

二、环境
1、软件版本
CentOS6
zookeeper-3.4.5+cdh5.9.0+98-1.cdh5.9.0.p0.30.el6.x86_64
hadoop-2.6.0+cdh5.9.0+1799-1.cdh5.9.0.p0.30.el6.x86_64
hbase-1.2.0+cdh5.9.0+205-1.cdh5.9.0.p0.30.el6.x86_64

2、角色

a、zk集群

10.10.20.64:2181
10.10.40.212:2181
10.10.102.207:2181

b、hbase
10.10.40.212 HMaster
10.10.20.64  殉道者杀手HMaster
10.10.10.114 HRegionServer
10.10.40.169 HRegionServer
10.10.30.174 HRegionServer

三、部署
1、配置cdh的yum源
vim /etc/yum.repos.d/cloudera-cdh.repo
[cloudera-cdh5]
# Packages for Cloudera's Distribution for Hadoop, Version 5.4.4, on RedHat or CentOS 6 x86_64
name=Cloudera's Distribution for Hadoop, Version 5.4.8
baseurl=http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.9.0/
gpgkey=http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera  殉道者杀手  殉道者杀手
gpgcheck=1
  殉道者杀手
[cloudera-gplextras5b2]
# Packages for Cloudera's GPLExtras, Version 5.4.4, on RedHat or CentOS 6 x86_64
name=Cloudera's GPLExtras, Version 5.4.8
baseurl=http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/5.9.0/
gpgkey=http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/RPM-GPG-KEY-cloudera  殉道者杀手  殉道者杀手
gpgcheck=1

2、安装zk集群(所有zk节点都操作)

1、安装

yum -y install zookeeper zookeeper-server

b、配置
vim /etc/zookeeper/conf/zoo.cfg  殉道者杀手
  殉道者杀手
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/lib/zookeeper
clientPort=2181
maxClientCnxns=0
server.1=10.10.20.64:2888:3888
server.2=10.10.40.212:2888:3888
server.3=10.10.102.207:2888:3888
autopurge.snapRetainCount=3
autopurge.purgeInterval=1

mkdir -p /data/lib/zookeeper  殉道者杀手#建zk的dir目录

echo 1 >/data/lib/zookeeper/myid  殉道者杀手#10.10.20.64上操作
echo 2 >/data/lib/zookeeper/myid  殉道者杀手#10.10.40.212上操作
echo 3 >/data/lib/zookeeper/myid  殉道者杀手#10.10.102.207上操作

c、启动服务
/etc/init.d/zookeeper-server start

3、安装配置hbase集群
a、安装
yum -y install hbase hbase-master  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手#HMaster节点操作  殉道者杀手  殉道者杀手
yum -y install hbase hbase-regionserver  殉道者杀手#HRegionServer节点操作

b、配置(所有base节点操作)
vim /etc/hbase/conf/hbase-site.xml  殉道者杀手
  殉道者杀手
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  殉道者杀手  殉道者杀手<property>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<name>hbase.zookeeper.quorum</name>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<value>10.10.20.64:2181,10.10.40.212:2181,10.10.102.207:2181</value>
  殉道者杀手  殉道者杀手</property>
  殉道者杀手  殉道者杀手<property>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<name>hbase.zookeeper.property.clientPort</name>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<value>2181</value>
  殉道者杀手  殉道者杀手</property>
  殉道者杀手  殉道者杀手<property>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<name>hbase.zookeeper.property.dataDir</name>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<value>/data/lib/zookeeper/</value>
  殉道者杀手  殉道者杀手</property>
  殉道者杀手  殉道者杀手<property>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<name>hbase.rootdir</name>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<value>hdfs://mycluster:8020/hbase</value>
  殉道者杀手  殉道者杀手</property>
  殉道者杀手  殉道者杀手<property>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<name>hbase.cluster.distributed</name>
  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手<value>true</value>
  殉道者杀手  殉道者杀手 <description>集群的模式,分布式还是单机模式,如果设置成false的话,HBase进程和Zookeeper进程在同一个JVM进程
  殉道者杀手  殉道者杀手 </description>
  殉道者杀手  殉道者杀手</property>
</configuration>

echo "export HBASE_MANAGES_ZK=false" >>/etc/hbase/conf/hbase-env.sh
#设置hbase使用独立的zk集群

vim /etc/hbase/conf/regionservers
ip-10-10-30-174.ec2.internal
ip-10-10-10-114.ec2.internal
ip-10-10-40-169.ec2.internal
#添加HRegionServer的主机名到regionservers,我没有在/etc/hosts下做主机名的映射,直接用了ec2的默认主机名

c、启动服务
/etc/init.d/hbase-master start  殉道者杀手  殉道者杀手  殉道者杀手 #HMaster节点操作
/etc/init.d/hbase-regionserver start #HRegionServer节点操作

4、验证

a、验证基本功能
[root@ip-10-10-20-64 ~]# hbase  殉道者杀手shell  殉道者杀手
2017-05-10 16:31:20,225 INFO  殉道者杀手[main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.0-cdh5.9.0, rUnknown, Fri Oct 21 01:19:47 PDT 2016
  殉道者杀手
hbase(main):001:0>  殉道者杀手status
1 active master, 1 backup masters, 3 servers, 0 dead, 1.3333 average load
  殉道者杀手
hbase(main):002:0> list
TABLE  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
test  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
test1  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
2 row(s) in 0.0330 seconds
  殉道者杀手
=> ["test", "test1"]
hbase(main):003:0> describe 'test'
Table test is ENABLED  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
test  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
COLUMN FAMILIES DESCRIPTION  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
{NAME => 'id', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERSIO
NS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
{NAME => 'name', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS
IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
{NAME => 'text', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', MIN_VERS
IONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手  殉道者杀手
3 row(s) in 0.1150 seconds
  殉道者杀手
hbase(main):004:0>

b、验证HA功能
1、hbase默认的web管理端口是60010,两个HMaster谁先启动谁就是主active节点,10.10.40.212先启动,10.10.20.64后启动,web截图如下:

wKioL1kSy5XRDLyYAAFNEiPWdq8622.png-wh_50
wKiom1kSy5az5p8nAAEheuczSu0415.png-wh_50

2、停止10.10.40.212的HMaster进程,查看10.10.20.64是否会提升为master
/etc/init.d/hbase-master stop

wKiom1kSzgzxbSyNAAFvwEWvXYk365.png-wh_50

Ubuntu 14.04下HBase单机和伪分布式模式安装配置  殉道者杀手http://www.linuxidc.com/Linux/2017-04/143048.htm

Hadoop+HBase搭建云存储总结 PDF http://www.linuxidc.com/Linux/2013-05/83844.htm

Ubuntu Server 14.04 下 Hbase数据库安装  殉道者杀手http://www.linuxidc.com/Linux/2016-05/131499.htm

HBase 结点之间时间不一致造成regionserver启动失败 http://www.linuxidc.com/Linux/2013-06/86655.htm

深入理解HBase架构原理 http://www.linuxidc.com/Linux/2017-01/139173.htm

Hadoop集群安装&HBase实验环境搭建 http://www.linuxidc.com/Linux/2013-04/83560.htm

基于Hadoop集群的HBase集群的配置 http://www.linuxidc.com/Linux/2013-03/80815.htm‘

Hadoop安装部署笔记之-HBase完全分布模式安装 http://www.linuxidc.com/Linux/2012-12/76947.htm

CentOS 6.4下HBase集群安装  殉道者杀手http://www.linuxidc.com/Linux/2016-11/137303.htm

HBase 的详细介绍:请点这里
HBase 的下载地址:请点这里

友情链接