美文网首页我爱编程
HDFS的HTTPS配置

HDFS的HTTPS配置

作者: xuefly | 来源:发表于2018-03-31 08:46 被阅读756次

    隶属于文章系列：大数据安全实战
    https://www.jianshu.com/p/76627fd8399c

    步骤：

    1. 在CA服务器上创建证书
    2. 在CA客户端导入证书
    3. 修改hdfs的hdfs-site.xml
    4. 配置hdfs的ssl配置文件
    • 在CA服务器上
      在部署成功了才发现CA服务器上并不是要运行什么服务。随便选一台就行。
    openssl req -new -x509 -keyout /var/opt/ssl/CA/private/test_ca_key -out /var/opt/ssl/CA/private/test_ca_cert -days 9999 -subj '/C=CN/ST=zhejiang/L=hangzhou/O=dtdream/OU=security/CN=zelda.com'


Generating a 2048 bit RSA private key
.....................................................+++
...................................................................................................+++
writing new private key to '/etc/pki/CA/private/test_ca_key'
#1234
Enter PEM pass phrase:
#1234
Verifying - Enter PEM pass phrase:
-----

    查看效果

    [root@v-app2-cloud kduser]# ll /etc/pki/CA/private/
total 8
-rw-r--r-- 1 root root 1383 Mar 12 16:37 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 12 16:37 test_ca_key
[root@v-app2-cloud kduser]#
    • 分布创建的证书到各个客户单
    [hadoop@vm10-247-24-53 hadoop]$ ansible hadoop --become -m copy  -a  "src=/var/opt/ssl dest=/var/opt/"
10.247.24.54 | SUCCESS => {
    "changed": true,
    "dest": "/var/opt/",
    "failed": false,
    "src": "/var/opt/ssl"
}
10.247.24.28 | SUCCESS => {
    "changed": true,
    "dest": "/var/opt/",
    "failed": false,
    "src": "/var/opt/ssl"
}
10.247.24.49 | SUCCESS => {
    "changed": true,
    "dest": "/var/opt/",
    "failed": false,
    "src": "/var/opt/ssl"
}
10.247.24.63 | SUCCESS => {
    "changed": true,
    "dest": "/var/opt/",
    "failed": false,
    "src": "/var/opt/ssl"
}
10.247.24.53 | SUCCESS => {
    "changed": false,
    "dest": "/var/opt/",
    "failed": false,
    "src": "/var/opt/ssl"
}

[hadoop@vm10-247-24-53 hadoop]$ ansible hadoop -m shell -a  "ls -l /var/opt/ssl/CA/private"
10.247.24.54 | SUCCESS | rc=0 >>
total 8
-rw-r--r-- 1 root root 1334 Mar 28 15:58 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 28 15:58 test_ca_key

10.247.24.28 | SUCCESS | rc=0 >>
total 8
-rw-r--r-- 1 root root 1334 Mar 28 15:56 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 28 15:56 test_ca_key

10.247.24.49 | SUCCESS | rc=0 >>
total 8
-rw-r--r-- 1 root root 1334 Mar 28 15:58 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 28 15:58 test_ca_key

10.247.24.63 | SUCCESS | rc=0 >>
total 8
-rw-r--r-- 1 root root 1334 Mar 28 15:59 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 28 15:59 test_ca_key

10.247.24.53 | SUCCESS | rc=0 >>
total 8
-rw-r--r-- 1 root root 1334 Mar 28 15:54 test_ca_cert
-rw-r--r-- 1 root root 1834 Mar 28 15:54 test_ca_key
    • 在各个客户端的节点上执行如下：
    # 进入证书分发的目录
cd /var/opt/ssl/CA/private/ ;

keytool -keystore keystore -alias localhost -validity 9999 -genkey -keyalg RSA -keysize 2048 -dname "CN=vm10-247-24-53.ksc.com, OU=test, O=test, L=hangzhou, ST=zhejiang, C=cn"

keytool -keystore truststore -alias CARoot -import -file test_ca_cert;
keytool -certreq -alias localhost -keystore keystore -file cert;
openssl x509 -req -CA test_ca_cert -CAkey test_ca_key -in cert -out cert_signed -days 9999 -CAcreateserial;
keytool -keystore keystore -alias CARoot -import -file test_ca_cert ;
keytool -keystore keystore -alias localhost -import -file cert_signed ;

    注意keytool -keystore keystore -alias localhost -validity 9999 -genkey -keyalg RSA -keysize 2048 -dname "CN=vm10-247-24-53.ksc.com, OU=test, O=test, L=hangzhou, ST=zhejiang, C=cn" 中的CN=vm10-247-24-53.ksc.com替换为各自的主机名

    • 检验客户端证书
      123456为上一步设置的密码。
    keytool  -list -v -keystore /var/opt/ssl/CA/private/keystore  -storepass 123456

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 2 entries

Alias name: caroot
Creation date: Mar 13, 2018
Entry type: trustedCertEntry

Owner: CN=zelda.com, OU=security, O=dtdream, L=hangzhou, ST=zhejiang, C=CN
Issuer: CN=zelda.com, OU=security, O=dtdream, L=hangzhou, ST=zhejiang, C=CN
Serial number: 9edcd7d2ea0b191e
Valid from: Mon Mar 12 16:56:18 CST 2018 until: Thu Jul 27 16:56:18 CST 2045
Certificate fingerprints:
        MD5:  6E:99:F2:B8:87:44:A1:2F:BD:48:05:1A:BC:42:00:2F
        SHA1: A1:67:DC:78:60:E0:AE:72:58:12:29:61:17:9F:D7:C4:88:F1:BD:62
        SHA256: 50:F2:2B:99:92:33:6B:D5:34:C1:51:BD:6B:CB:1C:72:A8:18:70:66:21:41:D1:E1:F5:24:71:87:B5:35:63:15
        Signature algorithm name: SHA1withRSA
        Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 9D 5C A4 0F A8 CC 1B 29   49 BF 36 4B 02 60 E7 23  .\.....)I.6K.`.#
0010: BC 38 A0 BA                                        .8..
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
 CA:true
 PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 9D 5C A4 0F A8 CC 1B 29   49 BF 36 4B 02 60 E7 23  .\.....)I.6K.`.#
0010: BC 38 A0 BA                                        .8..
]
]



*******************************************
*******************************************


Alias name: localhost
Creation date: Mar 13, 2018
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=v-hadoop-kbds.sz.kingdee.net, OU=test, O=test, L=hangzhou, ST=zhejiang, C=cn
Issuer: CN=zelda.com, OU=security, O=dtdream, L=hangzhou, ST=zhejiang, C=CN
Serial number: ac53b1f8fbaf29ba
Valid from: Tue Mar 13 10:47:18 CST 2018 until: Fri Jul 28 10:47:18 CST 2045
Certificate fingerprints:
        MD5:  4D:35:81:68:07:27:A1:5E:A6:0D:C3:BE:13:BC:B5:BF
        SHA1: 0F:3F:FF:3C:CA:64:43:40:EA:2A:6C:79:85:8C:BB:BB:27:46:57:9E
        SHA256: 8B:D6:36:4F:A1:83:C9:79:C4:A2:DA:3D:A3:6D:87:1A:18:E8:E8:80:3B:AC:D3:00:0D:25:31:CD:7B:DC:14:80
        Signature algorithm name: SHA256withRSA
        Version: 1
Certificate[2]:
Owner: CN=zelda.com, OU=security, O=dtdream, L=hangzhou, ST=zhejiang, C=CN
Issuer: CN=zelda.com, OU=security, O=dtdream, L=hangzhou, ST=zhejiang, C=CN
Serial number: 9edcd7d2ea0b191e
Valid from: Mon Mar 12 16:56:18 CST 2018 until: Thu Jul 27 16:56:18 CST 2045
Certificate fingerprints:
        MD5:  6E:99:F2:B8:87:44:A1:2F:BD:48:05:1A:BC:42:00:2F
        SHA1: A1:67:DC:78:60:E0:AE:72:58:12:29:61:17:9F:D7:C4:88:F1:BD:62
        SHA256: 50:F2:2B:99:92:33:6B:D5:34:C1:51:BD:6B:CB:1C:72:A8:18:70:66:21:41:D1:E1:F5:24:71:87:B5:35:63:15
        Signature algorithm name: SHA1withRSA
        Version: 3

Extensions:

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 9D 5C A4 0F A8 CC 1B 29   49 BF 36 4B 02 60 E7 23  .\.....)I.6K.`.#
0010: BC 38 A0 BA                                        .8..
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
 CA:true
 PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 9D 5C A4 0F A8 CC 1B 29   49 BF 36 4B 02 60 E7 23  .\.....)I.6K.`.#
0010: BC 38 A0 BA                                        .8..
]
]



*******************************************
*******************************************
    • 在hdfs-site.xml 中修改配置：
    <property>
  <name>dfs.datanode.address</name>
  <value>0.0.0.0:61004</value>
</property>
<property>
  <name>dfs.datanode.http.address</name>
  <value>0.0.0.0:61006</value>
</property>

<property>
  <name>dfs.http.policy</name>
  <value>HTTPS_ONLY</value>
</property>

    dfs.http.policy必须为：HTTPS_ONLY

    • 修改etc/hadoop下的ssl-client.xml
    <configuration>

<property>
  <name>ssl.client.truststore.location</name>
  <value>/var/opt/ssl/CA/private/truststore</value>
  <description>Truststore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.truststore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.client.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.client.keystore.location</name>
  <value>/var/opt/ssl/CA/private/truststore/keystore</value>
  <description>Keystore to be used by clients like distcp. Must be
  specified.
  </description>
</property>

<property>
  <name>ssl.client.keystore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.keypassword</name>
  <value>123456></value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.client.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

</configuration>
    • 修改etc/hadoop下的ssl-server.xml
    <property>
  <name>ssl.server.truststore.location</name>
  <value>/var/opt/ssl/CA/private/truststore</value>
  <description>Truststore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.truststore.password</name>
  <value>123456</value>
  <description>Optional. Default value is "".
  </description>
</property>

<property>
  <name>ssl.server.truststore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.truststore.reload.interval</name>
  <value>10000</value>
  <description>Truststore reload check interval, in milliseconds.
  Default value is 10000 (10 seconds).
  </description>
</property>

<property>
  <name>ssl.server.keystore.location</name>
  <value>/var/opt/ssl/CA/private/keystore</value>
  <description>Keystore to be used by NN and DN. Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.password</name>
  <value>123456</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.keypassword</name>
  <value>123456</value>
  <description>Must be specified.
  </description>
</property>

<property>
  <name>ssl.server.keystore.type</name>
  <value>jks</value>
  <description>Optional. The keystore file format, default value is "jks".
  </description>
</property>

<property>
  <name>ssl.server.exclude.cipher.list</name>
  <value>TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
  SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
  SSL_RSA_WITH_RC4_128_MD5</value>
  <description>Optional. The weak security cipher suites that you want excluded
  from SSL communication.</description>
</property>

    相关文章

      网友评论

        本文标题：HDFS的HTTPS配置

        本文链接：https://www.haomeiwen.com/subject/yrdicftx.html

        延伸阅读

        深度阅读

        • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

        栏目导航

        热点阅读

        我爱编程

        关于我们|服务条款|联系我们|HDFS的HTTPS配置|投稿指南|网站地图|RSS订阅|排版工具|手机版

        提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

        Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

        备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

        本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

        所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！