ambari安装hdp集群时，ResourceManager无法

作者: 咖啡大象不含咖啡 | 来源:发表于2017-12-03 19:22 被阅读58次

详细的报错信息为：

stderr: 
2017-12-03 10:38:59,607 - Error while executing command 'start':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 214, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 60, in start
    action='start'
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service.py", line 59, in service
    initial_wait=5
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 274, in action_run
    raise ex
Fail: Execution of 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1' returned 1.
 stdout:
2017-12-03 10:38:36,662 - u"Group['hadoop']" {'ignore_failures': False}
2017-12-03 10:38:36,662 - Modifying group hadoop
2017-12-03 10:38:36,720 - u"Group['users']" {'ignore_failures': False}
2017-12-03 10:38:36,720 - Modifying group users
2017-12-03 10:38:36,764 - u"User['zookeeper']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:36,764 - Modifying user zookeeper
2017-12-03 10:38:36,811 - u"User['ams']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:36,811 - Modifying user ams
2017-12-03 10:38:36,856 - u"User['ambari-qa']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'users']}
2017-12-03 10:38:36,856 - Modifying user ambari-qa
2017-12-03 10:38:36,903 - u"User['hdfs']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:36,904 - Modifying user hdfs
2017-12-03 10:38:36,947 - u"User['yarn']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:36,948 - Modifying user yarn
2017-12-03 10:38:36,995 - u"User['mapred']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:36,995 - Modifying user mapred
2017-12-03 10:38:37,039 - u"User['hbase']" {'gid': 'hadoop', 'ignore_failures': False, 'groups': [u'hadoop']}
2017-12-03 10:38:37,039 - Modifying user hbase
2017-12-03 10:38:37,083 - u"File['/var/lib/ambari-agent/data/tmp/changeUid.sh']" {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-12-03 10:38:37,366 - u"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']" {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-12-03 10:38:37,412 - Skipping u"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']" due to not_if
2017-12-03 10:38:37,412 - u"Directory['/hadoop/hbase']" {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2017-12-03 10:38:37,733 - u"File['/var/lib/ambari-agent/data/tmp/changeUid.sh']" {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-12-03 10:38:38,015 - u"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/hadoop/hbase']" {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2017-12-03 10:38:38,058 - Skipping u"Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/hadoop/hbase']" due to not_if
2017-12-03 10:38:38,059 - u"Group['hdfs']" {'ignore_failures': False}
2017-12-03 10:38:38,059 - Modifying group hdfs
2017-12-03 10:38:38,102 - u"User['hdfs']" {'ignore_failures': False, 'groups': [u'hadoop', 'hadoop', 'hdfs', u'hdfs']}
2017-12-03 10:38:38,104 - Modifying user hdfs
2017-12-03 10:38:38,149 - u"Directory['/etc/hadoop']" {'mode': 0755}
2017-12-03 10:38:38,303 - u"Directory['/etc/hadoop/conf.empty']" {'owner': 'root', 'group': 'hadoop', 'recursive': True}
2017-12-03 10:38:38,455 - u"Link['/etc/hadoop/conf']" {'not_if': 'ls /etc/hadoop/conf', 'to': '/etc/hadoop/conf.empty'}
2017-12-03 10:38:38,500 - Skipping u"Link['/etc/hadoop/conf']" due to not_if
2017-12-03 10:38:38,510 - u"File['/etc/hadoop/conf/hadoop-env.sh']" {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2017-12-03 10:38:38,765 - u"Execute['('setenforce', '0')']" {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-12-03 10:38:38,831 - Skipping u"Execute['('setenforce', '0')']" due to not_if
2017-12-03 10:38:38,832 - u"Directory['/var/log/hadoop']" {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:39,250 - u"Directory['/var/run/hadoop']" {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:39,658 - u"Directory['/tmp/hadoop-hdfs']" {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:39,987 - u"File['/etc/hadoop/conf/commons-logging.properties']" {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2017-12-03 10:38:40,223 - u"File['/etc/hadoop/conf/health_check']" {'content': Template('health_check-v2.j2'), 'owner': 'hdfs'}
2017-12-03 10:38:40,457 - u"File['/etc/hadoop/conf/log4j.properties']" {'content': '...', 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2017-12-03 10:38:40,703 - u"File['/etc/hadoop/conf/hadoop-metrics2.properties']" {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2017-12-03 10:38:40,944 - u"File['/etc/hadoop/conf/task-log4j.properties']" {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2017-12-03 10:38:41,367 - u"Directory['/var/run/hadoop-yarn']" {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:41,787 - u"Directory['/var/run/hadoop-yarn/yarn']" {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:42,296 - u"Directory['/var/log/hadoop-yarn/yarn']" {'owner': 'yarn', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:42,814 - u"Directory['/var/run/hadoop-mapreduce']" {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:43,232 - u"Directory['/var/run/hadoop-mapreduce/mapred']" {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:43,732 - u"Directory['/var/log/hadoop-mapreduce']" {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:44,142 - u"Directory['/var/log/hadoop-mapreduce/mapred']" {'owner': 'mapred', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:44,652 - u"Directory['/var/log/hadoop-yarn']" {'owner': 'yarn', 'ignore_failures': True, 'recursive': True, 'cd_access': 'a'}
2017-12-03 10:38:45,065 - u"XmlConfig['core-site.xml']" {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': {u'final': {u'fs.defaultFS': u'true'}}, 'owner': 'hdfs', 'configurations': ...}
2017-12-03 10:38:45,074 - Generating config: /etc/hadoop/conf/core-site.xml
2017-12-03 10:38:45,075 - u"File['/etc/hadoop/conf/core-site.xml']" {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2017-12-03 10:38:45,263 - Writing u"File['/etc/hadoop/conf/core-site.xml']" because contents don't match
2017-12-03 10:38:45,430 - u"XmlConfig['mapred-site.xml']" {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...}
2017-12-03 10:38:45,439 - Generating config: /etc/hadoop/conf/mapred-site.xml
2017-12-03 10:38:45,440 - u"File['/etc/hadoop/conf/mapred-site.xml']" {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2017-12-03 10:38:45,643 - Writing u"File['/etc/hadoop/conf/mapred-site.xml']" because contents don't match
2017-12-03 10:38:45,787 - Changing owner for /etc/hadoop/conf/mapred-site.xml from 506 to yarn
2017-12-03 10:38:45,832 - u"XmlConfig['yarn-site.xml']" {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...}
2017-12-03 10:38:45,840 - Generating config: /etc/hadoop/conf/yarn-site.xml
2017-12-03 10:38:45,840 - u"File['/etc/hadoop/conf/yarn-site.xml']" {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2017-12-03 10:38:46,087 - Writing u"File['/etc/hadoop/conf/yarn-site.xml']" because contents don't match
2017-12-03 10:38:46,232 - u"XmlConfig['capacity-scheduler.xml']" {'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'mode': 0644, 'configuration_attributes': {}, 'owner': 'yarn', 'configurations': ...}
2017-12-03 10:38:46,240 - Generating config: /etc/hadoop/conf/capacity-scheduler.xml
2017-12-03 10:38:46,241 - u"File['/etc/hadoop/conf/capacity-scheduler.xml']" {'owner': 'yarn', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding': 'UTF-8'}
2017-12-03 10:38:46,430 - Writing u"File['/etc/hadoop/conf/capacity-scheduler.xml']" because contents don't match
2017-12-03 10:38:46,597 - Changing owner for /etc/hadoop/conf/capacity-scheduler.xml from 504 to yarn
2017-12-03 10:38:46,646 - u"File['/var/log/hadoop-yarn/yarn/hadoop-mapreduce.jobsummary.log']" {'owner': 'yarn', 'group': 'hadoop'}
2017-12-03 10:38:46,846 - u"HdfsDirectory['/system/yarn/node-labels']" {'security_enabled': False, 'keytab': [EMPTY], 'conf_dir': '/etc/hadoop/conf', 'hdfs_user': 'hdfs', 'kinit_path_local': '', 'mode': 0700, 'owner': 'yarn', 'group': 'hadoop', 'action': ['create'], 'bin_dir': '/usr/hdp/current/hadoop-client/bin'}
2017-12-03 10:38:46,848 - u"Execute['hadoop --config /etc/hadoop/conf fs -mkdir -p /system/yarn/node-labels && hadoop --config /etc/hadoop/conf fs -chmod  700 /system/yarn/node-labels && hadoop --config /etc/hadoop/conf fs -chown  yarn:hadoop /system/yarn/node-labels']" {'not_if': "ambari-sudo.sh su hdfs -l -s /bin/bash -c 'hadoop --config /etc/hadoop/conf fs -ls /system/yarn/node-labels'", 'user': 'hdfs', 'path': ['/usr/hdp/current/hadoop-client/bin']}
2017-12-03 10:38:50,989 - Skipping u"Execute['hadoop --config /etc/hadoop/conf fs -mkdir -p /system/yarn/node-labels && hadoop --config /etc/hadoop/conf fs -chmod  700 /system/yarn/node-labels && hadoop --config /etc/hadoop/conf fs -chown  yarn:hadoop /system/yarn/node-labels']" due to not_if
2017-12-03 10:38:50,989 - u"File['/etc/hadoop/conf/yarn.exclude']" {'owner': 'yarn', 'group': 'hadoop'}
2017-12-03 10:38:51,179 - u"File['/etc/security/limits.d/yarn.conf']" {'content': Template('yarn.conf.j2'), 'mode': 0644}
2017-12-03 10:38:51,422 - u"File['/etc/security/limits.d/mapreduce.conf']" {'content': Template('mapreduce.conf.j2'), 'mode': 0644}
2017-12-03 10:38:51,657 - u"File['/etc/hadoop/conf/yarn-env.sh']" {'content': InlineTemplate(...), 'owner': 'yarn', 'group': 'hadoop', 'mode': 0755}
2017-12-03 10:38:51,893 - u"File['/etc/hadoop/conf/mapred-env.sh']" {'content': InlineTemplate(...), 'owner': 'hdfs'}
2017-12-03 10:38:52,129 - u"File['/etc/hadoop/conf/taskcontroller.cfg']" {'content': Template('taskcontroller.cfg.j2'), 'owner': 'hdfs'}
2017-12-03 10:38:52,358 - u"XmlConfig['mapred-site.xml']" {'owner': 'mapred', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...}
2017-12-03 10:38:52,365 - Generating config: /etc/hadoop/conf/mapred-site.xml
2017-12-03 10:38:52,366 - u"File['/etc/hadoop/conf/mapred-site.xml']" {'owner': 'mapred', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2017-12-03 10:38:52,598 - Writing u"File['/etc/hadoop/conf/mapred-site.xml']" because contents don't match
2017-12-03 10:38:52,753 - Changing owner for /etc/hadoop/conf/mapred-site.xml from 505 to mapred
2017-12-03 10:38:52,797 - u"XmlConfig['capacity-scheduler.xml']" {'owner': 'hdfs', 'group': 'hadoop', 'conf_dir': '/etc/hadoop/conf', 'configuration_attributes': {}, 'configurations': ...}
2017-12-03 10:38:52,810 - Generating config: /etc/hadoop/conf/capacity-scheduler.xml
2017-12-03 10:38:52,810 - u"File['/etc/hadoop/conf/capacity-scheduler.xml']" {'owner': 'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2017-12-03 10:38:53,019 - Writing u"File['/etc/hadoop/conf/capacity-scheduler.xml']" because contents don't match
2017-12-03 10:38:53,168 - Changing owner for /etc/hadoop/conf/capacity-scheduler.xml from 505 to hdfs
2017-12-03 10:38:53,213 - u"File['/var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid']" {'action': ['delete'], 'not_if': 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1'}
2017-12-03 10:38:53,352 - Deleting u"File['/var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid']"
2017-12-03 10:38:53,399 - u"Execute['ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/current/hadoop-client/libexec && /usr/hdp/current/hadoop-yarn-resourcemanager/sbin/yarn-daemon.sh --config /etc/hadoop/conf start resourcemanager']" {'not_if': 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1', 'user': 'yarn'}
2017-12-03 10:38:54,516 - u"Execute['ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1']" {'initial_wait': 5, 'not_if': 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1', 'user': 'yarn'}
2017-12-03 10:38:59,607 - Error while executing command 'start':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 214, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 60, in start
    action='start'
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service.py", line 59, in service
    initial_wait=5
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 274, in action_run
    raise ex
Fail: Execution of 'ls /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid >/dev/null 2>&1 && ps -p `cat /var/run/hadoop-yarn/yarn/yarn-yarn-resourcemanager.pid` >/dev/null 2>&1' returned 1. 
2017-12-03 10:38:59,652 - Command: /usr/bin/hdp-select status hadoop-yarn-resourcemanager > /tmp/tmp6b05ML
Output: hadoop-yarn-resourcemanager - 2.2.9.0-3393

解决办法：
修改/etc/hadoop/conf.empty/capacity-scheduler.xml，将以下两个参数的值改为：1

    <property>
      <name>yarn.scheduler.capacity.root.accessible-node-labels.default.capacity</name>
      <value>1</value>
    </property>
    <property>
      <name>yarn.scheduler.capacity.root.accessible-node-labels.default.maximum-capacity</name>
      <value>1</value>
    </property>

ambari参数修改界面

重启resource manager 即可。

ambari安装hdp集群时，ResourceManager无法

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

Hadoop运维日常