美文网首页程序员
HDFS 配置Kerberos

HDFS 配置Kerberos

作者: 莫茫Link | 来源:发表于2018-07-17 17:56 被阅读0次

安装Kerberos

参见Kerberos安装

配置SASL

#在namenode节点上执行
#创建CA，作为集群统一签发证书的机构
openssl req -new -x509 -keyout test_ca_key -out test_ca_cert -days 9999 -subj '/C=CN/ST=beijing/L=beijing/O=lql/OU=security/CN=hadoop.com'
#在所有datanode节点上执行
#为每一个节点创建key和证书
keytool -keystore keystore -alias localhost -validity 9999 -genkey -keyalg RSA -keysize 2048 -dname "CN=hadoop.com, OU=test, O=test, L=beijing, ST=beijing, C=cn"
#add the generated CA to the clients’ truststore so that the clients can trust this CA
keytool -keystore truststore -alias CARoot -import -file test_ca_cert
#export the certificate from the keystore
keytool -certreq -alias localhost -keystore keystore -file cert
#sign it with the CA
openssl x509 -req -CA test_ca_cert -CAkey test_ca_key -in cert -out cert_signed -days 9999 -CAcreateserial -passin pass:123456
#import both the certificate of the CA and the signed certificate into the keystore
keytool -keystore keystore -alias CARoot -import -file test_ca_cert
keytool -keystore keystore -alias localhost -import -file cert_signed

说明：

  • 遇到输入密码时：我这里统一输入的是123456
  • keystore: the keystore file that stores the certificate. The keystore file contains the private key of the certificate; therefore, it needs to be kept safely.
  • validity: the valid time of the certificate in days.

创建principal

这里为了简单，只创建hdfs这个用户

kadmin.local -q "addprinc -randkey hdfs/node-1@HADOOP.COM"
kadmin.local -q "addprinc -randkey hdfs/node-2@HADOOP.COM"
kadmin.local -q "addprinc -randkey hdfs/node-3@HADOOP.COM"

生成keytab文件

kadmin.local -q "xst  -k hdfs.keytab  hdfs/node-1@HADOOP.COM"
kadmin.local -q "xst  -k hdfs.keytab  hdfs/node-2@HADOOP.COM"
kadmin.local -q "xst  -k hdfs.keytab  hdfs/node-3@HADOOP.COM"

修改配置文件

core-site.xml 加入

<property>
  <name>hadoop.security.authentication</name>
  <value>kerberos</value>
</property>

<property>
  <name>hadoop.security.authorization</name>
  <value>true</value>
</property>
<property>
  <name>hadoop.security.auth_to_local</name>
  <value>
RULE:[2:$1@$0](.*@HADOOP.COM)s/.*/root/
DEFAULT
</value>
</property>

说明：
这里所有的进程都是由root用户执行，如果用户是hdfs的话，可将上面的root改为hdfs

hdfs-site.xml

<property>
  <name>dfs.block.access.token.enable</name>
  <value>true</value>
</property>

<property>
  <name>dfs.web.authentication.kerberos.principal</name>
  <value>hdfs/_HOST@HADOOP.COM</value>
</property>

<property>
  <name>dfs.web.authentication.kerberos.keytab</name>
  <value>/opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab</value>
</property>

<!-- NameNode security config -->
<property>
  <name>dfs.namenode.keytab.file</name>
  <value>/opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab</value>
</property>
<property>
  <name>dfs.namenode.kerberos.principal</name>
  <value>hdfs/_HOST@HADOOP.COM</value>
</property>
<property>
  <name>dfs.namenode.https-address</name>
  <value>node-1:50470</value>
</property>
<property>
  <name>dfs.secondary.namenode.keytab.file</name>
  <value>/opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab</value>
</property>
<property>
  <name>dfs.secondary.namenode.kerberos.principal</name>
  <value>hdfs/_HOST@HADOOP.COM</value>
</property>
<property>
  <name>dfs.namenode.secondary.https-address</name>
  <value>node-1:50091</value>
</property>
<property>

<!-- DataNode security config -->
<property>  
  <name>dfs.datanode.data.dir.perm</name>  
  <value>700</value>  
</property>

  <name>dfs.datanode.keytab.file</name>
  <value>/opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab</value>
</property>
<property>
  <name>dfs.datanode.kerberos.principal</name>
  <value>hdfs/_HOST@HADOOP.COM</value>
</property>

<!-- datanode SASL配置 -->
<property>
  <name>dfs.datanode.address</name>
  <value>0.0.0.0:61004</value>
</property>
<property>
  <name>dfs.datanode.http.address</name>
  <value>0.0.0.0:61006</value>
</property>
<property>
  <name>dfs.http.policy</name>
  <value>HTTPS_ONLY</value>
</property>
<property>
  <name>dfs.data.transfer.protection</name>
  <value>integrity</value>
</property>

<!-- nfs security config -->
<property>
    <name>nfs.keytab.file</name>
    <value>/opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab</value> <!-- path to the nfs gateway keytab -->
  </property>

  <property>
    <name>nfs.kerberos.principal</name>
    <value>hdfs/_HOST@HADOOP.COM</value>
  </property>

ssl-server.xml

cp ssl-server.xml.example ssl-server.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.server.truststore.location</name>
  <value>/root/sasl/truststore</value>
  <description>Truststore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.truststore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.server.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.server.keystore.location</name>
  <value>/root/sasl/keystore</value>
  <description>Keystore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.password</name>
  <value>123456</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.keypassword</name>
  <value>123456</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.exclude.cipher.list</name>
  <value>TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
  SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_RC4_128_MD5</value>
  <description>Optional. The weak security cipher suites that you want excluded
  from SSL communication.</description>
</property>

</configuration>

ssl-client.xml

cp ssl-client.xml.example ssl-client.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->
<configuration>

<property>
  <name>ssl.client.truststore.location</name>
  <value>/root/sasl/truststore</value>
  <description>Truststore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.truststore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.client.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.client.keystore.location</name>
  <value>/root/sasl/keystore</value>
  <description>Keystore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.keystore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.keypassword</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

</configuration>

说明：
ssl-server/client.xml就是配置下keystore、trustkeystore、password等

启动

停止hadoop进程
用hdfs用户登录：

kinit -k -t /opt/hadoop-2.9.1/etc/hadoop/hdfs-dn.keytab hdfs/node-3@HADOOP.COM

开启hadoop进程

注意

  • 如果本机有多个域名，注意将用户注册到多个域名上，否则可能报错

References

相关文章

网友评论

    本文标题：HDFS 配置Kerberos

    本文链接：https://www.haomeiwen.com/subject/qiegpftx.html

    延伸阅读

    深度阅读

    • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

    栏目导航

    热点阅读

    程序员

    关于我们|服务条款|联系我们|HDFS 配置Kerberos|投稿指南|网站地图|RSS订阅|排版工具|手机版

    提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

    Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

    备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

    本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

    所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！