1、启动hadoop，发现50070页面无法访问，jps后发现namenode未启动，到hadoop的logs日志中查看发现报错

java.io.IOException: Premature EOF from inputStream

at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:201)

at org.apache.hadoop.hdfs.server.namenode.FSImageFormat$LoaderDelegator.load(FSImageFormat.java:221)

at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:924)

at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:908)

at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImageFile(FSImage.java:727)

at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:663)

at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1058)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:762)

at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:582)

at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:641)

at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:808)

at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:792)

at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1484)

at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1550)

2016-08-12 08:52:00,113 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage

java.io.IOException: Failed to load an FSImage file!

at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:674)

at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:281)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1058)

at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:762)

at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:582)

at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:641)

at org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:808)

查不到合适的解决方案，就采取格式化namenode ：hadoop namenode -format 格式化之前要删除根目录下hdfs下的文件，并删除hadoop的logs文件，可能还要删别的，我只删除了这俩。

再次启动hadoop ，jps后发现namenode正常启动，50070正常访问，结果datanode无法启动，到hadoop的日志文件中查看发现报错

2、datanode无法启动

2016-08-12 10:06:14,316 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting

2016-08-12 10:06:14,317 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting

2016-08-12 10:06:14,592 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /data1/cdh/in_use.lock acquired by nodename 15432@is-nn-01

2016-08-12 10:06:14,594 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /data1/cdh: namenode clusterID = CID-c61b3952-cf9d-47d4-97df-ee86aa63cdf4; datanode clusterID = CID-b0cf57dd-646c-4c7a-9654-9568305f2ac1

2016-08-12 10:06:14,607 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /data2/cdh/in_use.lock acquired by nodename 15432@is-nn-01

2016-08-12 10:06:14,668 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /data2/cdh: namenode clusterID = CID-c61b3952-cf9d-47d4-97df-ee86aa63cdf4; datanode clusterID = CID-b0cf57dd-646c-4c7a-9654-9568305f2ac1

2016-08-12 10:06:14,670 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /data3/cdh/in_use.lock acquired by nodename 15432@is-nn-01

2016-08-12 10:06:14,670 WARN org.apache.hadoop.hdfs.server.common.Storage: java.io.IOException: Incompatible clusterIDs in /data3/cdh: namenode clusterID = CID-c61b3952-cf9d-47d4-97df-ee86aa63cdf4; datanode clusterID = CID-b0cf57dd-646c-4c7a-9654-9568305f2ac1

2016-08-12 10:06:14,671 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for Block pool (Datanode Uuid unassigned) service to is-nn-01/192.168.248.128:8020. Exiting.

java.io.IOException: All specified directories are failed to load.

at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:479)

at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1397)

at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1362)

at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:316)

at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:227)

at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:839)

at java.lang.Thread.run(Thread.java:745)

2016-08-12 10:06:14,673 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool (Datanode Uuid unassigned) service to is-nn-01/192.168.248.128:8020

2016-08-12 10:06:14,675 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Removed Block pool (Datanode Uuid unassigned)

2016-08-12 10:06:16,675 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Exiting Datanode

2016-08-12 10:06:16,676 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0

2016-08-12 10:06:16,678 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

从日志上看是因为 datanode的clusterID 和 namenode的clusterID 不匹配。

解决：

根据日志中的路径所报的路径，像我的是clusterIDs in /data1/cdh，到根目录下的data1中的cdh中(还有一个current文件夹下面有个VERSION的文件中)将日志信息中提示的namenode 地址copy 替换各个data文件夹中的clusterID即可，其他节点也要根据各个hadoop日志中报错信息相应替换ID 即可

重新启动 jps 全部都有 开心。

引用：

出现该问题的原因：在第一次格式化dfs后，启动并使用了hadoop，后来又重新执行了格式化命令(hdfs namenode -format)，这时namenode的clusterID会重新生成，而datanode的clusterID 保持不变。