CDP 多Namenode配置

<p data-mpa-powered-by="yiban.io" style="visibility: visible;"><span style="font-family: mp-quote, -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif; visibility: visible;">此功能为 CDP Private Cloud Base 提供了使用多个备用名称节点的能力。</span></p><section data-mpa-template="t" mpa-from-tpl="t" style="visibility: visible;"><h2 mpa-is-content="t" style="margin: 5px; padding-right: 10px; padding-left: 10px; font-weight: bold; max-width: 100%; font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; letter-spacing: 0.544px; text-align: justify; white-space: normal; widows: 1; border-left: 5px solid rgb(2, 30, 170); line-height: 32px; background-color: rgb(255, 255, 255); border-top-color: rgb(2, 30, 170); border-bottom-color: rgb(2, 30, 170); border-right-color: rgb(2, 30, 170); box-sizing: border-box !important; overflow-wrap: break-word !important; visibility: visible;"><span style="color: rgb(136, 136, 136); border-color: rgb(2, 30, 170); visibility: visible;"><span style="color: rgb(2, 30, 170); border-color: rgb(2, 30, 170); visibility: visible;" mpa-is-content="t">先决条件</span></span></h2></section><ul class="list-paddingleft-1" style="visibility: visible;"><li style="visibility: visible;"><p style="visibility: visible;">Cloudera Manager 版本必须为 7.7.1 或更高版本</p></li><li style="visibility: visible;"><p style="visibility: visible;">CDP 包裹版本必须为 7.1.8 或更高版本</p></li><li style="visibility: visible;"><p style="visibility: visible;">HDFS 应该启用高可用性</p></li><li style="visibility: visible;"><p style="visibility: visible;">HDFS的最新升级必须完成</p></li><li style="visibility: visible;"><p style="visibility: visible;">根据 Cloudera Manager，集群必须是健康的，并且没有任何陈旧的配置</p></li></ul><p style="visibility: visible;">此功能仅对 7.1.8 集群启用。这需要启用高可用性功能。在为集群配置额外的 NameNode 后，您无法回滚 Cloudera Manager 升级或 CDP 包升级。在升级到 718 完成后，您必须添加一个额外的名称节点。</p><section data-mpa-template="t" mpa-from-tpl="t" style="visibility: visible;"><h2 mpa-is-content="t" style="margin: 5px; padding-right: 10px; padding-left: 10px; font-weight: bold; max-width: 100%; font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; letter-spacing: 0.544px; text-align: justify; white-space: normal; widows: 1; border-left: 5px solid rgb(2, 30, 170); line-height: 32px; background-color: rgb(255, 255, 255); border-top-color: rgb(2, 30, 170); border-bottom-color: rgb(2, 30, 170); border-right-color: rgb(2, 30, 170); box-sizing: border-box !important; overflow-wrap: break-word !important; visibility: visible;"><span style="color: rgb(136, 136, 136); border-color: rgb(2, 30, 170); visibility: visible;"><span style="color: rgb(2, 30, 170); border-color: rgb(2, 30, 170); visibility: visible;" mpa-is-content="t">多个 Namenodes 配置</span></span></h2></section><p style="visibility: visible;">Cloudera Manager 中默认不启用此功能。您必须首先在 Cloudera Manager 配置中启用此功能。然后，您可以使用多个备用 NameNode 配置 HDFS 服务。</p><h2 style="visibility: visible;">在 Cloudera Manager 中启用多个名称节点支持功能</h2><ol class="list-paddingleft-1" style="visibility: visible;"><li style="visibility: visible;"><p style="visibility: visible;">导航到/etc/default/cloudera-scm-server 文件。</p></li><li style="visibility: visible;"><p style="visibility: visible;">在cloudera-scm-server文件的末尾 添加 export CMF_FF_HDFS_MULTIPLE_SNN=true 。</p></li></ol><p style="visibility: visible;"><img class="rich_pages wxw-img" data-backh="167" data-backw="578" data-ratio="0.2885304659498208" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS31HFzdPemjYR7elXibRLTnENceUxTQOHh07ia8LQ1NM8QDOriam4S4iasYw/640?wx_fmt=png" data-type="png" data-w="2232" style="width: 100% !important; height: auto !important; display: initial; visibility: visible !important;" data-index="1" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/059c1ab88b19bf1f.png" data-fail="0"></p><ol class="list-paddingleft-1" style="visibility: visible;"><li style="visibility: visible;"><p style="visibility: visible;">重新启动 Cloudera Manager服务。</p></li></ol><section class="code-snippet__fix code-snippet__js" style="visibility: visible;"><ul class="code-snippet__line-index code-snippet__js" style="visibility: visible;"><li style="visibility: visible;"></li></ul><pre class="code-snippet__js" data-lang="nginx" style="visibility: visible;"><code style="visibility: visible;"><span class="code-snippet_outer" style="visibility: visible;"><span class="code-snippet__attribute" style="visibility: visible;">systemctl</span> restart cloudera-scm-server</span></code></pre></section><p style="visibility: visible;"><img class="rich_pages wxw-img" data-backh="261" data-backw="578" data-ratio="0.45227475468331846" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3oN2Lh8lD8FXLUpMnMUUbbOWWw4HONmvI0vriaYdotz7qqVKbu5nOnHw/640?wx_fmt=png" data-type="png" data-w="2242" style="width: 100%; height: auto; display: none;" data-index="2" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/18fd0626336272a2.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3oN2Lh8lD8FXLUpMnMUUbbOWWw4HONmvI0vriaYdotz7qqVKbu5nOnHw/640?wx_fmt=png" data-index="2" style="visibility: visible; width: 677px !important; height: 306.19px !important;"><span class="weui-primary-loading" style="visibility: visible;"><span class="" style="visibility: visible;"></span></span></span></p><section data-mpa-template="t" mpa-from-tpl="t" style="visibility: visible;"><h2 mpa-is-content="t" style="margin: 5px; padding-right: 10px; padding-left: 10px; font-weight: bold; max-width: 100%; font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif; font-variant-numeric: normal; font-variant-east-asian: normal; letter-spacing: 0.544px; text-align: justify; white-space: normal; widows: 1; border-left: 5px solid rgb(2, 30, 170); line-height: 32px; background-color: rgb(255, 255, 255); border-top-color: rgb(2, 30, 170); border-bottom-color: rgb(2, 30, 170); border-right-color: rgb(2, 30, 170); box-sizing: border-box !important; overflow-wrap: break-word !important; visibility: visible;"><span style="color: rgb(136, 136, 136); border-color: rgb(2, 30, 170); visibility: visible;"><span style="color: rgb(2, 30, 170); border-color: rgb(2, 30, 170); visibility: visible;" mpa-is-content="t">已知问题及其解决方法</span></span></h2></section><p>在继续进一步配置其他名称节点之前，您必须了解以下已知问题。</p><h2>已知问题描述</h2><p>如果最近在集群上重新启动了JournalNodes，如果在重新启动 JournalNodes 后没有创建新的fsImage ，则 HDFS 服务的添加新 NameNode向导可能无法引导新的 NameNode 。但是，在重新启动 JournalNodes 时，编辑日志会在系统中滚动。</p><h2>已知问题解决方法</h2><p>如果在Add new NameNode向导期间引导失败，您必须执行以下步骤：</p><ol class="list-paddingleft-1"><li><p>删除新添加的NameNode和FailoverController</p></li><li><p>将活动的 HDFS NameNode 移动到安全模式</p></li><li><p>在活动的 HDFS NameNode 上执行 Save Namespace 操作</p></li><li><p>在活动的 HDFS NameNode 上离开安全模式</p></li><li><p>尝试再次添加新的 NameNode</p></li></ol><p><strong>笔记</strong></p><p>进入安全模式会禁用对 HDFS 的写入操作并导致服务中断。如果无法进入安全模式，只需删除HDFS服务中新添加的NameNode和FailoverController即可。您必须等到 HDFS 自动创建新的 fsImage 并尝试使用向导再次添加新的 NameNode。</p><section data-mpa-template="t" mpa-from-tpl="t"><h2 mpa-is-content="t" style="margin: 5px;padding-right: 10px;padding-left: 10px;font-weight: bold;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;font-variant-numeric: normal;font-variant-east-asian: normal;letter-spacing: 0.544px;text-align: justify;white-space: normal;widows: 1;border-left: 5px solid rgb(2, 30, 170);line-height: 32px;background-color: rgb(255, 255, 255);border-top-color: rgb(2, 30, 170);border-bottom-color: rgb(2, 30, 170);border-right-color: rgb(2, 30, 170);box-sizing: border-box !important;overflow-wrap: break-word !important;"><span style="color:#888888;border-color: rgb(2, 30, 170);"><span style="color: rgb(2, 30, 170);border-color: rgb(2, 30, 170);" mpa-is-content="t">使用 HDFS 服务添加多个名称节点</span></span></h2></section><p><span style="font-family: mp-quote, -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;">您必须确保 Cloudera Manager 处于干净状态，没有任何健康问题并且集群上没有陈旧的配置。如果存在任何运行状况问题或过时的配置，那么您必须与命令分开重新启动集群，而不是在命令中。为此，您必须清除Rolling Restart HDFS 和所有相关服务以激活步骤 10b 中的更改选项。此外，如果有任何健康问题或无法从集群中清除的陈旧配置，您必须单独重新启动集群。</span></p><p>可以使用 Cloudera Manager 中的 HDFS 服务添加多个名称节点。</p><ol class="list-paddingleft-1"><li><p>Hdfs服务已经启用高可用</p></li></ol><p><img class="rich_pages wxw-img" data-backh="313" data-backw="578" data-ratio="0.542240862792091" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3ejX00IOpcmR4rQqWA8eNkw8BbZcO6ScGPibWl46M6OLT4JQ0UDE4SVw/640?wx_fmt=png" data-type="png" data-w="3338" style="width: 100%; height: auto; display: none;" data-index="3" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/3e5e73572c5837ca.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3ejX00IOpcmR4rQqWA8eNkw8BbZcO6ScGPibWl46M6OLT4JQ0UDE4SVw/640?wx_fmt=png" data-index="3" style="width: 677px !important; height: 367.097px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>导航到集群，选择HDFS服务，然后转到实例，单击联邦和高可用性按钮</p></li></ol><p><img class="rich_pages wxw-img" data-backh="229" data-backw="578" data-ratio="0.3969896519285042" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3HnQryibgatd1xxPdqrhM2EsscnGHoicCIonHUcfr0NepY4p7Jw0HV5eQ/640?wx_fmt=png" data-type="png" data-w="2126" style="width: 100%; height: auto; display: none;" data-index="4" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/e78c3fdd8a4f032e.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3HnQryibgatd1xxPdqrhM2EsscnGHoicCIonHUcfr0NepY4p7Jw0HV5eQ/640?wx_fmt=png" data-index="4" style="width: 677px !important; height: 268.762px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>单击操作，单击添加其他名称节点。添加 其他 NameNode向导打开。</p></li></ol><p><img class="rich_pages wxw-img" data-backh="357" data-backw="578" data-ratio="0.6175908221797323" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS32m7T89PUmIP3zD0PiaKLw8zAYDdrLhng7QU3DiaG1wT0ys15eoUlqZVw/640?wx_fmt=png" data-type="png" data-w="2092" style="width: 100%; height: auto; display: none;" data-index="5" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/42b533494c03ac4c.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS32m7T89PUmIP3zD0PiaKLw8zAYDdrLhng7QU3DiaG1wT0ys15eoUlqZVw/640?wx_fmt=png" data-index="5" style="width: 677px !important; height: 418.109px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>在“分配角色”页面中，选择未分配给任何名称节点的主机。</p></li></ol><p><img class="rich_pages wxw-img" data-backh="518" data-backw="578" data-ratio="0.8964862298195632" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3EfgeyuVic2lU1jaBDKHEffP6chhicMNjKDmn3b99f7vXhXrLTHnZf3Fg/640?wx_fmt=png" data-type="png" data-w="2106" style="width: 100%; height: auto; display: none;" data-index="6" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/48909488d78b239a.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3EfgeyuVic2lU1jaBDKHEffP6chhicMNjKDmn3b99f7vXhXrLTHnZf3Fg/640?wx_fmt=png" data-index="6" style="width: 677px !important; height: 606.921px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>单击继续。</p></li><li><p>在Review Changes页面中，您可以指定 namenode 数据目录。</p></li><ol class="list-paddingleft-1" style="list-style-type: lower-alpha;"><li><p>如果要删除已提供的名称节点数据目录中存在的数据，则必须选择清除备用名称节点名称目录中存在的任何现有数据选项。请记住备份名称节点数据目录。</p></li><li><p>如果要重启 HDFS 服务及其依赖项，则必须选择Rolling Restart HDFS and alldependent services 以激活更改选项。请记住，如果您没有启用高可用性功能，滚动重启可能会导致集群停机。注意：如果您未选择 Rolling Restart 选项，则可以单击 Continue 稍后重新启动集群。</p></li></ol></ol><p><img class="rich_pages wxw-img" data-backh="514" data-backw="578" data-ratio="0.8885741265344664" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3JATQ0v2pQkdMWSRocpKF3xicnvEwQRCNqs85ib7HXoTgWACj5D4xsn0w/640?wx_fmt=png" data-type="png" data-w="2118" style="width: 100%; height: auto; display: none;" data-index="7" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/94d01cc42dc5b892.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3JATQ0v2pQkdMWSRocpKF3xicnvEwQRCNqs85ib7HXoTgWACj5D4xsn0w/640?wx_fmt=png" data-index="7" style="width: 677px !important; height: 601.565px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>单击继续。这将运行所需的命令并添加额外的名称节点。</p></li></ol><p><img class="rich_pages wxw-img" data-backh="516" data-backw="578" data-ratio="0.8933962264150943" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3y5SR644ibnZcPvd0eTD2xsLfn9iaia0QQj6UfMFqS03YXvFa1EWBlDsqA/640?wx_fmt=png" data-type="png" data-w="2120" style="width: 100%; height: auto; display: none;" data-index="8" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/c1968999a318dc48.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3y5SR644ibnZcPvd0eTD2xsLfn9iaia0QQj6UfMFqS03YXvFa1EWBlDsqA/640?wx_fmt=png" data-index="8" style="width: 677px !important; height: 604.829px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><ol class="list-paddingleft-1"><li><p>单击完成。</p></li></ol><p><img class="rich_pages wxw-img" data-backh="518" data-backw="578" data-ratio="0.8961904761904762" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3bY0ibQLdBsGd5BBpYSOVR14N0icvPs8KLHpL5QiclSEPEicgDqbMhqhgbA/640?wx_fmt=png" data-type="png" data-w="2100" style="width: 100%; height: auto; display: none;" data-index="9" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/fdef9c6a1f3ad5f4.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS3bY0ibQLdBsGd5BBpYSOVR14N0icvPs8KLHpL5QiclSEPEicgDqbMhqhgbA/640?wx_fmt=png" data-index="9" style="width: 677px !important; height: 606.721px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><p><img class="rich_pages wxw-img" data-backh="239" data-backw="578" data-ratio="0.4127740705433746" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS37Owo2Viaw5rvyNwxljX3PP5qib1Su9ED5icS5RQv3SF9t6TekuKicFZpcw/640?wx_fmt=png" data-type="png" data-w="2098" style="width: 100%; height: auto; display: none;" data-index="10" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/6e7b5b38b7a4e3a3.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS37Owo2Viaw5rvyNwxljX3PP5qib1Su9ED5icS5RQv3SF9t6TekuKicFZpcw/640?wx_fmt=png" data-index="10" style="width: 677px !important; height: 279.448px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><p>可以看到已经有三个NameNode， 其中一个是活动的，两个是备用的。</p><p><img class="rich_pages wxw-img" data-backh="320" data-backw="578" data-ratio="0.5536144578313253" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS36jZ9ByHBPoBJ4R9JDoSvD98DQLuZWlsqTsibCq9h0tgYfPZzc2VqvRw/640?wx_fmt=png" data-type="png" data-w="3320" style="width: 100%; height: auto; display: none;" data-index="11" data-origin-display="" _width="100%" crossorigin="anonymous" alt="图片" src="https://img.haomeiwen.com/i13265002/6f183f604db3d7ef.png"><span class="js_img_placeholder wx_widget_placeholder" data-src="https://mmbiz.qpic.cn/mmbiz_png/9IZsf6icv8GomcV7uMPic1tXf4abnS0mS36jZ9ByHBPoBJ4R9JDoSvD98DQLuZWlsqTsibCq9h0tgYfPZzc2VqvRw/640?wx_fmt=png" data-index="11" style="width: 677px !important; height: 374.797px !important;"><span class="weui-primary-loading"><span class=""></span></span></span></p><p style="display: none;"><mp-style-type data-value="3"></mp-style-type></p><blockquote><p>本文使用 <a href="https://www.jianshu.com/p/5709df6fb58d" class="internal">文章同步助手</a> 同步</p></blockquote>