背景
最近新搭建了Train版的Openstack做测试使用,在所有服务组件都搭建完成之后,创建虚拟机时提示异常失败。
追查问题
首先登陆控制节点查看相关日志的信息
#nova-api.log日志报错信息
2021-05-19 16:06:04.318 8244 INFO nova.api.openstack.wsgi [req-72746704-80e4-4e55-ba1f-71ed0e264117 992c698e1d2c42eeae0814b0bae92c63 276ba88b657342d0917a55cec365acb9 - default default] HTTP exception thrown: Failed to delete allocations for consumer 09988e21-c6f7-4b95-af6e-db8bffbba245. Error: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /allocations/09988e21-c6f7-4b95-af6e-db8bffbba245
on this server.</p>
</body></html>
2021-05-19 16:05:41.907 9392 WARNING nova.scheduler.utils [req-3d64b666-9e03-4ba7-99ba-5f063d831e24 992c698e1d2c42eeae0814b0bae92c63 276ba88b657342d0917a55cec365acb9 - default default] [instance: 09988e21-c6f7-4b95-af6e-db8bffbba245] Setting instance to ERROR state.: NoValidHost_Remote: No valid host was found.
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager [req-40fae53a-6fe0-4200-b775-95a6d13c059d 992c698e1d2c42eeae0814b0bae92c63 276ba88b657342d0917a55cec365acb9 - default default] Failed to schedule instances: NoValidHost_Remote: No valid host was found.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 235, in inner
return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 199, in select_destinations
raise exception.NoValidHost(reason="")
#nova-conductor.log日志报错信息
NoValidHost: No valid host was found.
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager Traceback (most recent call last):
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1379, in schedule_and_build_instances
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager instance_uuids, return_alternates=True)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 839, in _schedule_instances
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager return_alternates=return_alternates)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager instance_uuids, return_objects, return_alternates)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 181, in call
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager transport_options=self.transport_options)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 129, in _send
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager transport_options=transport_options)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 674, in send
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager transport_options=transport_options)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 664, in _send
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager raise result
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager NoValidHost_Remote: No valid host was found.
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager Traceback (most recent call last):
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 235, in inner
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager return func(*args, **kwargs)
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 199, in select_destinations
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager raise exception.NoValidHost(reason="")
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager
2021-05-20 11:23:05.715 9393 ERROR nova.conductor.manager NoValidHost: No valid host was found.
#nova-scheduler.log日志报错信息
2021-05-20 11:23:05.692 9467 WARNING keystoneauth.discover [req-40fae53a-6fe0-4200-b775-95a6d13c059d 992c698e1d2c42eeae0814b0bae92c63 276ba88b657342d0917a55cec365acb9 - default default] Failed to contact the endpoint at http://test-controller:8778 for discovery. Fallback to using that endpoint as the base url.: Forbidden: Forbidden (HTTP 403)
2021-05-20 11:23:05.696 9467 ERROR nova.scheduler.client.report [req-40fae53a-6fe0-4200-b775-95a6d13c059d 992c698e1d2c42eeae0814b0bae92c63 276ba88b657342d0917a55cec365acb9 - default default] Failed to retrieve allocation candidates from placement API for filters: RequestGroup(aggregates=[],forbidden_aggregates=set([]),forbidden_traits=set(['COMPUTE_STATUS_DISABLED']),in_tree=None,provider_uuids=[],requester_id=None,required_traits=set([]),resources={DISK_GB=10,MEMORY_MB=1024,VCPU=1},use_same_provider=False)
Got 403: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /allocation_candidates
on this server.</p>
</body></html>
查看计算节点的日志信息:
2021-05-20 11:39:23.533 26358 WARNING keystoneauth.discover [req-d17ce15d-3605-452b-817c-9bd27f74dd41 - - - - -] Failed to contact the endpoint at http://test-controller:8778 for discovery. Fallback to using that endpoint as the base url.: Forbidden: Forbidden (HTTP 403)
2021-05-20 11:39:23.537 26358 ERROR nova.compute.resource_tracker [req-d17ce15d-3605-452b-817c-9bd27f74dd41 - - - - -] Skipping removal of allocations for deleted instances: Failed to retrieve allocations for resource provider 10d34a96-3e5f-47a7-856f-9c3e21fde051: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /resource_providers/10d34a96-3e5f-47a7-856f-9c3e21fde051/allocations
on this server.</p>
</body></html>
: ResourceProviderAllocationRetrievalFailed: Failed to retrieve allocations for resource provider 10d34a96-3e5f-47a7-856f-9c3e21fde051: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
2021-05-20 11:39:23.547 26358 WARNING keystoneauth.discover [req-d17ce15d-3605-452b-817c-9bd27f74dd41 - - - - -] Failed to contact the endpoint at http://test-controller:8778 for discovery. Fallback to using that endpoint as the base url.: Forbidden: Forbidden (HTTP 403)
2021-05-20 11:39:23.550 26358 ERROR nova.scheduler.client.report [req-d17ce15d-3605-452b-817c-9bd27f74dd41 - - - - -] [None] Failed to retrieve resource provider tree from placement API for UUID 10d34a96-3e5f-47a7-856f-9c3e21fde051. Got 403: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /resource_providers
on this server.</p>
</body></html>
.
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager [req-d17ce15d-3605-452b-817c-9bd27f74dd41 - - - - -] Error updating resources for node test-compute1.: ResourceProviderRetrievalFailed: Failed to get resource provider with UUID 10d34a96-3e5f-47a7-856f-9c3e21fde051
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager Traceback (most recent call last):
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 8785, in _update_available_resource_for_node
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager startup=startup)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 887, in update_available_resource
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager self._update_available_resource(context, resources, startup=startup)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 328, in inner
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager return f(*args, **kwargs)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 972, in _update_available_resource
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager self._update(context, cn, startup=startup)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 1237, in _update
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager self._update_to_placement(context, compute_node, startup)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 68, in wrapped_f
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager return Retrying(*dargs, **dkw).call(f, *args, **kw)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 223, in call
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager return attempt.get(self._wrap_exception)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 261, in get
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager six.reraise(self.value[0], self.value[1], self.value[2])
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/retrying.py", line 217, in call
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 1151, in _update_to_placement
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager context, compute_node.uuid, name=compute_node.hypervisor_hostname)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 858, in get_provider_tree_and_ensure_root
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager parent_provider_uuid=parent_provider_uuid)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 640, in _ensure_resource_provider
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager rps_to_refresh = self.get_providers_in_tree(context, uuid)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/scheduler/client/report.py", line 503, in get_providers_in_tree
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager raise exception.ResourceProviderRetrievalFailed(uuid=uuid)
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager ResourceProviderRetrievalFailed: Failed to get resource provider with UUID 10d34a96-3e5f-47a7-856f-9c3e21fde051
2021-05-20 11:39:23.551 26358 ERROR nova.compute.manager
根据以上的报错信息可以看出跟placement权限有关
解决办法
调整配置文件的的访问权限
/etc/httpd/conf.d/00-placement-api.conf
Listen 8778
<VirtualHost *:8778>
WSGIProcessGroup placement-api
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
WSGIDaemonProcess placement-api processes=3 threads=1 user=placement group=placement
WSGIScriptAlias / /usr/bin/placement-api
#以下是增加部分#
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
#以上是增加部分#
<IfVersion >= 2.4>
ErrorLogFormat "%M"
</IfVersion>
ErrorLog /var/log/placement/placement-api.log
#SSLEngine On
#SSLCertificateFile ...
#SSLCertificateKeyFile ...
</VirtualHost>
Alias /placement-api /usr/bin/placement-api
<Location /placement-api>
SetHandler wsgi-script
Options +ExecCGI
WSGIProcessGroup placement-api
WSGIApplicationGroup %{GLOBAL}
WSGIPassAuthorization On
</Location>
重启服务
systemctl restart httpd
网友评论