Hbase主机无法正常启动

Question

我正在使用Hadoop2.4.0/Hbase 0.98.0/Hive 0.14.0

Hadoop和HBase运行良好,直到我重新启动我的HMaster.hbase-hduser-master-master.log文件中出现以下错误:

 2015-02-17 05:46:15,157 INFO  [master:master:60000] master.TableNamespaceManager: Namespace table not found. Creating...
    2015-02-17 05:46:15,193 DEBUG [master:master:60000] lock.ZKInterProcessLockBase: Acquired a lock for /hbase/table-lock/hbase:namespace/write-master:600000000000004
    2015-02-17 05:46:15,212 DEBUG [master:master:60000] lock.ZKInterProcessLockBase: Released /hbase/table-lock/hbase:namespace/write-master:600000000000004
    2015-02-17 05:46:15,212 FATAL [master:master:60000] master.HMaster: Master server abort: loaded coprocessors are: []
    2015-02-17 05:46:15,213 FATAL [master:master:60000] master.HMaster: Unhandled exception. Starting shutdown.
    org.apache.hadoop.hbase.TableExistsException: hbase:namespace
        at org.apache.hadoop.hbase.master.handler.CreateTableHandler.prepare(CreateTableHandler.java:120)
        at org.apache.hadoop.hbase.master.TableNamespaceManager.createNamespaceTable(TableNamespaceManager.java:232)
        at org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:86)
        at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:1049)
        at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:913)
        at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:606)
        at java.lang.Thread.run(Unknown Source)
    2015-02-17 05:46:15,214 INFO  [master:master:60000] master.HMaster: Aborting
    2015-02-17 05:46:15,214 INFO  [master,60000,1424180766819-BalancerChore] balancer.BalancerChore: master,60000,1424180766819-BalancerChore exiting
    2015-02-17 05:46:15,215 INFO  [master,60000,1424180766819-ClusterStatusChore] balancer.ClusterStatusChore: master,60000,1424180766819-ClusterStatusChore exiting
    2015-02-17 05:46:15,215 INFO  [CatalogJanitor-master:60000] master.CatalogJanitor: CatalogJanitor-master:60000 exiting
    2015-02-17 05:46:15,216 DEBUG [master:master:60000] master.HMaster: Stopping service threads
    2015-02-17 05:46:15,216 INFO  [master:master:60000] ipc.RpcServer: Stopping server on 60000
    2015-02-17 05:46:15,216 INFO  [RpcServer.listener,port=60000] ipc.RpcServer: RpcServer.listener,port=60000: stopping
    2015-02-17 05:46:15,218 INFO  [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped
    2015-02-17 05:46:15,218 INFO  [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping
    2015-02-17 05:46:15,218 INFO  [master:master:60000.oldLogCleaner] cleaner.LogCleaner: master:master:60000.oldLogCleaner exiting
    2015-02-17 05:46:15,218 INFO  [master:master:60000.oldLogCleaner] master.ReplicationLogCleaner: Stopping replicationLogCleaner-0x14b97c83f580008, quorum=slave:2181,master:2181, baseZNode=/hbase
    2015-02-17 05:46:15,219 INFO  [master:master:60000.archivedHFileCleaner] cleaner.HFileCleaner: master:master:60000.archivedHFileCleaner exiting
    2015-02-17 05:46:15,219 INFO  [master:master:60000] master.HMaster: Stopping infoServer
    2015-02-17 05:46:15,223 INFO  [master:master:60000.oldLogCleaner] zookeeper.ZooKeeper: Session: 0x14b97c83f580008 closed
    2015-02-17 05:46:15,223 INFO  [master:master:60000-EventThread] zookeeper.ClientCnxn: EventThread shut down
    2015-02-17 05:46:15,229 INFO  [master:master:60000] mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60010
    2015-02-17 05:46:15,236 DEBUG [master:master:60000] catalog.CatalogTracker: Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@19f9598
    2015-02-17 05:46:15,236 INFO  [master:master:60000] client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x14b97c83f580007
    2015-02-17 05:46:15,237 INFO  [master:master:60000-EventThread] zookeeper.ClientCnxn: EventThread shut down
    2015-02-17 05:46:15,238 INFO  [master:master:60000] zookeeper.ZooKeeper: Session: 0x14b97c83f580007 closed
    2015-02-17 05:46:15,238 INFO  [master,60000,1424180766819.splitLogManagerTimeoutMonitor] master.SplitLogManager$TimeoutMonitor: master,60000,1424180766819.splitLogManagerTimeoutMonitor exiting
    2015-02-17 05:46:15,243 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
    2015-02-17 05:46:15,243 INFO  [master:master:60000] zookeeper.ZooKeeper: Session: 0x14b97c83f580006 closed
    2015-02-17 05:46:15,243 INFO  [master:master:60000] master.HMaster: HMaster main thread exiting
    2015-02-17 05:46:15,243 ERROR [main] master.HMasterCommandLine: Master exiting
    java.lang.RuntimeException: HMaster Aborted
        at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:192)
        at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:134)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
        at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2785)

这里有什么不对,HMaster Aborted意味着什么？

有关更多信息,这就是我的hbase-site.xml样子:

<property>
<name>hbase.rootdir</name>
<value>hdfs://master:54310/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master,slave</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/usr/local/hbase/zookeeper</value>
</property>

Answer 1

我今天遇到了这个问题!我的解决方案如下:

第1步:停止Hbase.

第2步:运行以下命令

hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair

此命令用于修复Hbase的MetaData

第3步:删除zookeeper中的数据(警告它会让你丢失旧数据)

./opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/zookeeper/bin/zkCli.sh

你可以ls /用来扫描zookeeper中的数据

用于rmr /hbase删除zookeeper中的hbase数据

第4步:启动hbase

Answer 2

这是基于另一个答案,但澄清升级cloudera 5.4

步骤1:

service hbase-regionserver stop

service hbase-master stop

第2步:

hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair

第3步:删除zookeeper中的数据(警告它会让你丢失旧数据)

cd /usr/lib/zookeeper/bin/
./zkCli.sh

它打开了zookeeper shell.

然后运行:

ls / 
rmr /hbase

第4步:启动hbase

service hbase-master restart
service hbase-regionserver restart