美文网首页
Hue支持Hdp3.0 Oozie

Hue支持Hdp3.0 Oozie

作者: 文竹小二 | 来源:发表于2018-09-18 20:57 被阅读120次

    1、问题描述

    当在HDP中配置了Oozie的SSO后,会发现在Hue中提交oozie workflow失败异常。在解决问题的过程中,共出现了两个异常。第二个异常是在解决第一个异常的基础上出现的。

    2、异常一

    2.1异常信息

    [13/Sep/2018 18:50:15 +0800] editor2 ERROR Error submitting coordinator
    Traceback (most recent call last):
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 700, in _submit_coordinator
    wf_dir = Submission(request.user, wf, request.fs, request.jt, mapping, local_tz=coordinator.data['properties']['timezone']).deploy()
    File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/submission2.py", line 194, in deploy
    self._update_properties(jt_address) # Needed for coordinator deploying workflows with credentials
    File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/submission2.py", line 351, in _update_properties
    credentials.fetch(self.api)
    File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/credentials.py", line 40, in fetch
    configuration = oozie_api.get_configuration()
    File "/home/hue/hue-4.2.0/desktop/libs/liboozie/src/liboozie/oozie_api.py", line 319, in get_configuration
    resp = self._root.get('admin/configuration', params)
    File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 122, in get
    return self.invoke("GET", relpath, params, headers=headers, allow_redirects=True, clear_cookies=clear_cookies)
    File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 78, in invoke
    log_response=log_response)
    File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/resource.py", line 97, in _invoke
    clear_cookies=clear_cookies)
    File "/home/hue/hue-4.2.0/desktop/core/src/desktop/lib/rest/http_client.py", line 211, in execute
    raise self._exc_class(ex)
    RestException: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)

    2.2解决方法

    由于是报Hue证书问题,故在Hue的hue.ini配置文件中增加ssl_cacerts属性。


    image.png

    3、异常二

    3.1异常信息

    [18/Sep/2018 09:35:40 +0800] middleware INFO Processing exception: 'str' object has no attribute 'get': Traceback (most recent call last):
    File "/home/hue/hue-4.2.0/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, *callback_kwargs)
    File "/home/hue/hue-4.2.0/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py", line 371, in inner
    return func(
    args, **kwargs)
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/decorators.py", line 113, in decorate
    return view_func(request, *args, **kwargs)
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/decorators.py", line 75, in decorate
    return view_func(request, *args, **kwargs)
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 367, in submit_workflow
    return _submit_workflow_helper(request, workflow, submit_action=reverse('oozie:editor_submit_workflow', kwargs={'doc_id': workflow.id}))
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 426, in _submit_workflow_helper
    'is_oozie_mail_enabled': _is_oozie_mail_enabled(request.user),
    File "/home/hue/hue-4.2.0/apps/oozie/src/oozie/views/editor2.py", line 435, in _is_oozie_mail_enabled
    return oozie_conf.get('oozie.email.smtp.host') != 'localhost'
    AttributeError: 'str' object has no attribute 'get'

    3.2问题定位

    在不加SSO的情况下,hue并不会出现该异常。通过在出现异常代码行前新加输出oozie_conf的打印日志,发现输出的是登录页面。由于是Hue的后端调用oozie的Rest API,所以并不需要输出登录页面,解决此问题大致有了一个固定方向就是禁止执行SSO流程。

    根据查看官方文档,发现SSO使用的类org.apache.hadoop.security.authentication.server.JWTRedirectAuthenticationHandler大致实现方式为如下图标红所示。


    image.png

    因此查看了AltKerberosAuthenticationHandler.java中关键方法是根据请求中是否带有User-Agent头来判断是否是浏览器访问。


    image.png

    通过抓包请求,确实也发现hue的请求传递了User-Agent请求头。


    image.png

    3.3解决方法

    在hue-4.2.0/desktop/core/src/desktop/lib/rest/http_client.py中,通过增加如下标红代码行移除User-Agent请求头。


    image.png

    相关文章

      网友评论

          本文标题:Hue支持Hdp3.0 Oozie

          本文链接:https://www.haomeiwen.com/subject/qgomnftx.html