hadoop常见错误

错误一：本地运行Java 代码时出出现：

2018-09-14 19:45:07,266 DEBUG org.apache.hadoop.util.Shell.checkHadoopHome() - Failed to detect a valid hadoop home directory

java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.

2018-09-14 19:45:07,275 ERROR org.apache.hadoop.util.Shell.getWinUtilsPath() - Failed to locate the winutils binary in the hadoop binary path

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

解决：把hadoop安装目录下的hadoop.dll考到system32目录下

错误二：运行mapreduce

Job job_1536830416611_0021 failed with state FAILED due to: Application application_1536830416611_0021 failed 2 times due to Error launching appattempt_1536830416611_0021_000002. Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

This token is expired. current time is 1536878358609 found 1536837804741

Note: System times on machines may be out of sync. Check system time and time zones.

没有同步时间

解决：

在namenode 和datanode所在的服务器上都输入：

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

ntpdate pool.ntp.org  进行时间同步

如果在ntpdate pool.ntp.org  时 出现  the NTP socket is in use, exiting

用 ps -ef|grep ntp 查出进程后 杀死

错误三：

启动zookeeper 服务时

zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper-3.4.10/bin/../conf/zoo.cfg

Error contacting service. It is probably not running.

删除 /etc/hosts 中的 127.0.0.0拿一行

错误四：

linux的shell脚本运行出现失败

shell 脚本在/usr/local/bin 下 不管在哪执行都可以不用写路径

-bash: /usr/local/bin/xcall.sh: /bin/bash^M: bad interpreter:

在行末模式下输入set ff回车，查看当前的模式，如果时doc就通过set fileformat=unix修改成unix

错误五：

Connection refused

​ at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

​ at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)

​ at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)

​ at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

18/09/19 19:41:53 INFO ipc.Client: Retrying connect to server: hadoop03/192.168.136.130:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)(所有journalnode都得开启，才能format namenode)

错误六：

2018-09-19 20:22:04,275 FATAL org.apache.hadoop.ha.ZKFailoverController: Unable to start failover controller. Parent znode does not exist.

Run with -formatZK flag to initialize ZooKeeper.

（要保证所有设置的zookeper集群的zookeeper服务都开启 才能 格式化 ）