转自： http://v2agile.com/2017/05/03/java-jce-hadoop-kerberos/

目的是解决明明 klist 正常返回，却依然无法访问对方 HDFS 的问题。

Java JCE and Hadoop and Kerberos

issue:

执行hadoop命令的时候总是报错如下

yarn logs -applicationId application_1493267710027_0414

17/05/02 03:35:36 WARN ipc.Client: Exception encountered while connecting to the server :
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
    at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)
    at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:563)
    at org.apache.hadoop.ipc.Client$Connection.access$1900(Client.java:378)
    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:732)
    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:728)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:727)
    at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:378)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1492)
    at org.apache.hadoop.ipc.Client.call(Client.java:1402)
    at org.apache.hadoop.ipc.Client.call(Client.java:1363)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
    at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:773)

但是kinit明明没有问题。

这个时候需要看一下kerberos的debug log

vi /usr/hdp/current/hadoop-yarn-client/bin/yarn
export YARN_OPTS="-Dsun.security.krb5.debug=true ${YARN_OPTS} -Dhdp.version=${HDP_VERSION}"

这个时候可以看到一些有用的信息了

[03:35]:[yarn@en0003:~]$ yarn logs -applicationId application_1493267710027_0414 -appOwner xxx
/usr/java/latest/bin/java
Java config name: null
Native config name: /etc/krb5.conf
Loaded from native config
>>>KinitOptions cache name is /tmp/krb5cc_204
>>>DEBUG <CCacheInputStream>  client principal is rm/n0003.xyz.com@XYZ.COM
>>>DEBUG <CCacheInputStream> server principal is krbtgt/XYZ.COM@XYZ.COM.COM
>>>DEBUG <CCacheInputStream> key type: 18
>>>DEBUG <CCacheInputStream> auth time: Tue May 02 03:09:56 GMT-07:00 2017
>>>DEBUG <CCacheInputStream> start time: Tue May 02 03:09:56 GMT-07:00 2017
>>>DEBUG <CCacheInputStream> end time: Tue May 02 13:09:56 GMT-07:00 2017
>>>DEBUG <CCacheInputStream> renew_till time: Tue May 09 03:09:56 GMT-07:00 2017
>>> CCacheInputStream: readFlags()  FORWARDABLE; RENEWABLE; INITIAL; PRE_AUTH;
>>>DEBUG <CCacheInputStream>  client principal is rm/en0003.xyz.com@XYZ.COM
>>>DEBUG <CCacheInputStream> server principal is X-CACHECONF:/krb5_ccache_conf_data/pa_type/krbtgt/XYZ.COM@ XYZ.COM
>>>DEBUG <CCacheInputStream> key type: 0
>>>DEBUG <CCacheInputStream> auth time: Wed Dec 31 17:00:00 GMT-07:00 1969
>>>DEBUG <CCacheInputStream> start time: null
>>>DEBUG <CCacheInputStream> end time: Wed Dec 31 17:00:00 GMT-07:00 1969
>>>DEBUG <CCacheInputStream> renew_till time: null
>>> CCacheInputStream: readFlags()
>>> unsupported key type found the default TGT: 18
17/05/02 03:35:34 INFO impl.TimelineClientImpl: Timeline service address: https://en0004.xyz.com:8190/ws/v1/timeline/
17/05/02 03:35:36 WARN ipc.Client: Exception encountered while connecting to the server :
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
    at org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)
    at org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:563)
    at org.apache.hadoop.ipc.Client$Connection.access$1900(Client.java:378)
    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:732)
    at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:728)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:727)
    at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:378)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1492)
    at org.apache.hadoop.ipc.Client.call(Client.java:1402)
    at org.apache.hadoop.ipc.Client.call(Client.java:1363)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
    at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:773)

注意其中的>>> unsupported key type found the default TGT: 18

Reason

https://bugzilla.redhat.com/show_bug.cgi?id=1243337

根据解释，这里的type 18， 表示的是AES256的一种加密算法。而当前的JDK并不支持这种方式。所以导致无法解密。

Solution

方案1

修改kerberos client上的默认的加密方式

[libdefaults]
  renew_lifetime = 7d
  forwardable = true
  default_realm = XYZ.COM
  ticket_lifetime = 24h
  dns_lookup_realm = false
  dns_lookup_kdc = false
  #default_tgs_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
  #default_tkt_enctypes = aes des3-cbc-sha1 rc4 des-cbc-md5
 default_tgs_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5 des-cbc-crc
 default_tkt_enctypes = arcfour-hmac-md5 aes256-cts aes128-cts des-cbc-md5 des-cbc-crc
 permitted_enctypes = aes256-cts aes128-cts arcfour-hmac-md5 des-cbc-md5 des-cbc-crc

修改之前的加密类型

 [05:12]:[yarn@node2.xyz.com:~]$ klist -e
Ticket cache: FILE:/tmp/krb5cc_204
Default principal: nm/node2.xyz.com@XYZ.COM

Valid starting     Expires            Service principal
05/02/17 05:12:17  05/02/17 15:12:17  krbtgt/XYX.COM@XYZ.COM
    renew until 05/09/17 05:12:17, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

修改之后的加密类型

[05:27]:[yarn@node2.xyz.com:~]$ klist -e
Ticket cache: FILE:/tmp/krb5cc_204
Default principal: rm/node2.xyz.com@XYZ.COM

Valid starting       Expires              Service principal
05/02/2017 05:27:20  05/02/2017 15:27:20  krbtgt/XYZ.COM@XYZ.COM
    renew until 05/09/2017 05:27:20, Etype (skey, tkt): arcfour-hmac, aes256-cts-hmac-sha1-96

方案2

下载JCE也可以,安装到jre/lib/security/也可以

http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html

方案3

使用OpenJDK， 因为OpenJDK 本身就是自带Unlimited Encrytion功能的