美文网首页openstackOpenStack
cinder配额 quota 使用及代码分析

cinder配额 quota 使用及代码分析

作者: 笨手笨脚越 | 来源:发表于2017-07-17 21:49 被阅读144次

    封装抽象

    quota相关功能实现在cinder\quota.py实现,包含了引擎、资源、驱动三个大类抽象封装。看起来quota.py用的是设计模式“抽象工厂模式”,引擎选择驱动、定义资源,调用驱动管理资源。

    一、资源

    资源其实就是对配额的封装,封装了资源名、默认值、数据库统计函数dbapi等。

    资源分类列表:

    类名 描述 属性
    cinder.quota.BaseResource 定义单个配额资源 name(资源名)、flag(控制默认值)、parent_project_id(当前租户的附租户)、quota方法用于获取资源的使用量、default方法用户获取默认值
    cinder.quota.AbsoluteResource 无预留的资源 pass无定义
    cinder.quota.ReservableResource 可预留的资源 sync (dbapi 方法名,统计配额使用信息。如范例)
    cinder.quota.CountableResource 可统计的资源,cinder代码里没看到使用 count(统计函数)
    cinder.quota.VolumeTypeResource 为卷类型定义的资源,继承ReservableResource volume_type_name(卷类型名)、volume_type_id(卷类型id)
    注意:
    • ReservableResource:相比BaseResource,多了sync方法,sync会被驱动调用,用于在计算配额之前,先同步配额信息(到本地和数据库)。ReservableResource只能用于project绑定的资源。

    • CountableResource:相比BaseResource,多了count方法,count方法必须给出一个函数,自己计算配额,其返回值里会包含配额实际使用值。

    Resource类关系图.png

    范例:ReservableResource资源'volume'的sync:cinder.db.sqlalchemy.api._sync_volumes

    def _sync_volumes(context, project_id, session, volume_type_id=None,
                      volume_type_name=None):
        # 根据volume_type_id和project_id统计卷数量和卷空间使用量
        (volumes, _gigs) = _volume_data_get_for_project(
            context, project_id, volume_type_id=volume_type_id, session=session)
        key = 'volumes'
        if volume_type_name:
            key += '_' + volume_type_name
        return {key: volumes}
    
    

    二、引擎

    定义了资源集。调用驱动来实现查询统计功能。

    引擎列表:

    类名 描述 resources
    cinder.quota.QuotaEngine 配额引擎,基类
    cinder.quota.VolumeTypeQuotaEngine 卷类型配额引擎 'volumes', 'per_volume_gigabytes','snapshots','gigabytes','backups','backup_gigabytes'
    cinder.quota.CGQuotaEngine 一致性组的配额的引擎 consistencygroups
    cinder.quota.GroupQuotaEngine 组配额的引擎 groups

    三、驱动

    驱动是可配置的,对应配置项quota_driver,默认值是cinder.quota.DbQuotaDriver

        cfg.StrOpt('quota_driver',
                   default="cinder.quota.DbQuotaDriver",
                   help='Default driver to use for quota checks')
    

    驱动列表:

    名称 描述
    cinder.quota.DbQuotaDriver Number of volume gigabytes allowed per tenant
    cinder.quota.NestedDbQuotaDriver Number of Block Storage snapshots allowed per tenant.

    quota主要操作四张数据表:

    • reservations表,定义每个项目配额的增量。
    图片.png
    • quota_usage表,定义每个项目配额的已使用量和预留量。
    quota_usage表.png
    • quota_classes表,定义了配额类的配额。操作界面上的默认配额就是保存在这个表里。
    quota_classes表.png
    • quotas表,定义了项目的配额。是如果仅仅是调用API接口或者client指令 openstack project create pro3 创建项目,是不会同时创建项目对应的专用配额的。但是如果在管理界面上创建项目,horizon会同时调用cinder的quota接口创建三个“gigabytes”、“volumes”、“snapshots” cinder专用配额,另外还会调neutron、nova的配额接口创建它们专用的配额。
    quotas表.png 创建project操作界面.png

    指令功能介绍

    quotas相关的指令:

    quota-class-show Lists quotas for a quota class.
    quota-class-update Updates quotas for a quota class.
    quota-defaults Lists default quotas for a tenant.
    quota-delete Delete the quotas for a tenant.
    quota-show Lists quotas for a tenant.
    quota-update Updates quotas for a tenant.
    quota-usage Lists quota usage for a tenant.

    quota-class-show、quota-class-update、quota-defaults是对quota_classes表操作,quota-delete、quota-show、quota-update 主要对quota表操作,quota-usage是对quota_classes、quota_usage操作。

    作为管理员用户,我们可以修改块存储租户的配额,也可以对新租户定义默认配额。

    1. 列出默认List all default quotas for all tenants, as follows:

    $ cinder quota-defaults tenantID

    [root@node1 ~]# cinder quota-defaults admin
    +------------------------------+-------+
    | Property                     | Value |
    +------------------------------+-------+
    | backup_gigabytes             | 1000  |
    | backups                      | 10    |
    | gigabytes                    | 1000  |
    | gigabytes_netapp_volume_type | -1    |
    | gigabytes_nfs_common         | -1    |
    | gigabytes_vmware             | -1    |
    | gigabytes_vmware-type        | -1    |
    | per_volume_gigabytes         | -1    |
    | snapshots                    | 10    |
    | snapshots_netapp_volume_type | -1    |
    | snapshots_nfs_common         | -1    |
    | snapshots_vmware             | -1    |
    | snapshots_vmware-type        | -1    |
    | volumes                      | 10    |
    | volumes_netapp_volume_type   | -1    |
    | volumes_nfs_common           | -1    |
    | volumes_vmware               | -1    |
    | volumes_vmware-type          | -1    |
    +------------------------------+-------+
    

    quota-defaults 代码分析:

    cinder.api.contrib.quotas.QuotaSetsController#defaults:

        return self._format_quota_set(id, QUOTAS.get_defaults(context, project_id=id))
        # QUOTAS 是类对象 cinder.quota.VolumeTypeQuotaEngine() 
    

    cinder.quota.QuotaEngine#get_defaults:

        def get_defaults(self, context, project_id=None):
            return self._driver.get_defaults(context, self.resources,
                                             project_id)
    

    _driver 是 cinder.quota.QuotaEngine#_driver,代码可得cinder.quota.DbQuotaDriver

        @property
        def _driver(self):
            # _driver_class是__init__构造函数里传入设置的,没传为None
            if self._driver_class:
                return self._driver_class
            
            if not self._quota_driver_class:
                # 读配置项cfg.StrOpt('quota_driver',default="cinder.quota.DbQuotaDriver",
                self._quota_driver_class = CONF.quota_driver
        
            if isinstance(self._quota_driver_class, six.string_types):
                # 动态导入类对象
                self._quota_driver_class = importutils.import_object(
                    self._quota_driver_class)
    
            self._driver_class = self._quota_driver_class
            return self._driver_class
    

    cinder.quota.DbQuotaDriver#get_defaults:

        def get_defaults(self, context, resources, project_id=None):
            quotas = {}
            default_quotas = {}
            # cfg.BoolOpt('use_default_quota_class',default=True,
            if CONF.use_default_quota_class:
                # 查询'quota_classes'表,过滤出class_name = 'defualt'的记录, 
                default_quotas = db.quota_class_get_defaults(context)
    
            # resources 来自cinder.quota.VolumeTypeQuotaEngine的@property 方法
            for resource in resources.values():
                if default_quotas:
                    if resource.name not in default_quotas:
                        versionutils.report_deprecated_feature(LOG, _(
                            "Default quota for resource: %(res)s is set "
                            "by the default quota flag: quota_%(res)s, "
                            "it is now deprecated. Please use the "
                            "default quota class for default "
                            "quota.") % {'res': resource.name})
                # default_quotas的值 复写 resources ,如果default_quotas里不包含resource,则使用resource的default属性。default属性说明见下文!
                quotas[resource.name] = default_quotas.get(resource.name,
                                                           resource.default)
            return quotas
    
    

    代码可知cinder.quota.DbQuotaDriver的get_defaults方法不区分租户project_id,所以如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver",调用 cinder quota-defaults ${project_id} 的结果都是一样的。

    cinder.quota.VolumeTypeQuotaEngine#resources:

        @property
        def resources(self):
            """Fetches all possible quota resources."""
            result = {}
            # Global quotas.
            argses = [('volumes', '_sync_volumes', 'quota_volumes'),
                      ('per_volume_gigabytes', None, 'per_volume_size_limit'),
                      ('snapshots', '_sync_snapshots', 'quota_snapshots'),
                      ('gigabytes', '_sync_gigabytes', 'quota_gigabytes'),
                      ('backups', '_sync_backups', 'quota_backups'),
                      ('backup_gigabytes', '_sync_backup_gigabytes',
                       'quota_backup_gigabytes')]
            # 根据上面定义的argses获得ReservableResource列表,
            for args in argses:
                resource = ReservableResource(*args)
                result[resource.name] = resource
    
            # 查询得volume_type列表
            volume_types = db.volume_type_get_all(context.get_admin_context(),
                                                  False)
            for volume_type in volume_types.values():
                for part_name in ('volumes', 'gigabytes', 'snapshots'):
                    # 对每一个volume_type,按照规则 name = "%s_%s" % (part_name, self.volume_type_name)设置resource.name
                    resource = VolumeTypeResource(part_name, volume_type)
                    result[resource.name] = resource
            # 返回ReservableResource和VolumeTypeResource组合的resoure列表
            return result
        
    #### (Pdb) p result
    {'per_volume_gigabytes': <cinder.quota.ReservableResource object at 0x98e7090>, 'gigabytes': <cinder.quota.ReservableResource object at 0x98e70d0>, 'backup_gigabytes': <cinder.quota.ReservableResource object at 0x98e7150>, 'snapshots': <cinder.quota.ReservableResource object at 0x98e7050>, 'volumes': <cinder.quota.ReservableResource object at 0x9b8ffd0>, 'backups': <cinder.quota.ReservableResource object at 0x98e7110>}
    

    看下ReservableResource和VolumeTypeResource 类的继承关系:

    ReservableResource和VolumeTypeResource 对象的default属性,都继承自cinder.quota.BaseResource#default:

        @property
        def default(self):
            """Return the default value of the quota."""
    
            if self.parent_project_id:
                return 0
            # 如果self.flag不是空,则返回CONF[self.flag],否则返回-1
            return CONF[self.flag] if self.flag else -1
    

    总结:

    • 如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver"或者不配置,cinder quota-defaults 指令打印结果和租户id无关。
    • cinder quota-defaults 指令打印列表,包含了'volumes', 'per_volume_gigabytes','snapshots','gigabytes','backups','backup_gigabytes' 六个元素,另外还包含'volumes_${volume_type}', 'gigabytes_${volume_type}', 'snapshots_${volume_type}' 多个元素,元素的值由表 quota_classes 里 class_name='default' 的记录设置,如果表里没有,则由cinder.quota.BaseResource#default得到。

    2. 根据quota-class列出quota列表:

    cinder quota-class-show <class>

    quota-defaults 代码分析:

    如果配置项'quota_driver'使用"cinder.quota.DbQuotaDriver"或者不配置,走的是cinder.quota.DbQuotaDriver#get_class_quotas:

        def get_class_quotas(self, context, resources, quota_class,
                             defaults=True):
            """Given list of resources, retrieve the quotas for given quota class.
    
            :param context: The request context, for access checks.
            :param resources: A dictionary of the registered resources.
            :param quota_class: The name of the quota class to return
                                quotas for.
            :param defaults: If True, the default value will be reported
                             if there is no specific value for the
                             resource.
            """
    
            quotas = {}
            default_quotas = {}
            # 根据class_name在数据库表quota_classes查询出归属某类的配额属性
            class_quotas = db.quota_class_get_all_by_name(context, quota_class)
            if defaults:
                # 数据库表quota_classes查出默认类的配额属性
                default_quotas = db.quota_class_get_defaults(context)
            
            for resource in resources.values():
                # 检查cinder.quota.VolumeTypeQuotaEngine#resources的元素是否有跟class_quotas交集,
                # 如果有,则用class_quotas的元素值覆盖
                if resource.name in class_quotas:
                    quotas[resource.name] = class_quotas[resource.name]
                    continue
                # 检查cinder.quota.VolumeTypeQuotaEngine#resources的元素是否有跟default_quotas交集,
                # 如果有,则用default_quotas的元素值覆盖
                if defaults:
                    quotas[resource.name] = default_quotas.get(resource.name,
                                                               resource.default)
    
            return quotas
    

    总结:

    • 根据class_name在数据库表quota_classes表查出记录集class_quotas,再从class_name='default'在数据库表quota_classes表查出记录集default_quotas。
    • 用class_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
    • 用default_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。

    3. 根据quota-class列出quota列表:

    cinder quota-show <tenant_id>

    cinder.quota.DbQuotaDriver#get_project_quotas:

        def get_project_quotas(self, context, resources, project_id,
                               quota_class=None, defaults=True,
                               usages=True):
            """Retrieve quotas for a project.
    
            Given a list of resources, retrieve the quotas for the given
            project.
    
            :param context: The request context, for access checks.
            :param resources: A dictionary of the registered resources.
            :param project_id: The ID of the project to return quotas for.
            :param quota_class: If project_id != context.project_id, the
                                quota class cannot be determined.  This
                                parameter allows it to be specified.  It
                                will be ignored if project_id ==
                                context.project_id.
            :param defaults: If True, the quota class value (or the
                             default value, if there is no value from the
                             quota class) will be reported if there is no
                             specific value for the resource.
            :param usages: If True, the current in_use, reserved and allocated
                           counts will also be returned.
            """
    
            quotas = {}
            # 在数据库表 quotas 根据 project_id 查询字段 hard_limit
            project_quotas = db.quota_get_all_by_project(context, project_id)
            allocated_quotas = None
            default_quotas = None
            # API调用的时候传入的usages=False
            if usages:
                # 根据 project_id 查询数据库表 quota_usages ,获得各quota resource的 in_use和 reserved
                project_usages = db.quota_usage_get_all_by_project(context,
                                                                   project_id)
                # 在数据库表 quotas 根据 project_id 查询字段 allocated
                allocated_quotas = db.quota_allocated_get_all_by_project(
                    context, project_id)
                allocated_quotas.pop('project_id')
    
            # Get the quotas for the appropriate class.  If the project ID
            # matches the one in the context, we use the quota_class from
            # the context, otherwise, we use the provided quota_class (if
            # any) 
            # 如果context有quota_class,用context.quota_class查询class_quotas
            if project_id == context.project_id:
                quota_class = context.quota_class
            if quota_class:
                class_quotas = db.quota_class_get_all_by_name(context, quota_class)
            else:
                class_quotas = {}
    
            for resource in resources.values():
                # Omit default/quota class values
                # defaults = True
                if not defaults and resource.name not in project_quotas:
                    continue
    
                quota_val = project_quotas.get(resource.name)
                # 如果 project_quota 是空,用 class_quota 的值赋值
                if quota_val is None:
                    quota_val = class_quotas.get(resource.name)
                    # 如果 class_quota 赋的值也是空,用 default_quotas 的值赋值
                    if quota_val is None:
                        # Lazy load the default quotas
                        if default_quotas is None:
                            default_quotas = self.get_defaults(
                                context, resources, project_id)
                        quota_val = default_quotas[resource.name]
    
                quotas[resource.name] = {'limit': quota_val}
    
                # Include usages if desired.  This is optional because one
                # internal consumer of this interface wants to access the
                # usages directly from inside a transaction.
                if usages: # False
                    usage = project_usages.get(resource.name, {})
                    quotas[resource.name].update(
                        in_use=usage.get('in_use', 0),
                        reserved=usage.get('reserved', 0), )
                if allocated_quotas: # None
                    quotas[resource.name].update(
                        allocated=allocated_quotas.get(resource.name, 0), )
            return quotas
    

    总结:

    • 用project_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值
    • 如果project_quotas的元素值为空,用class_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。
    • 如果class_quotas的元素值为空,用default_quotas的元素值覆盖cinder.quota.VolumeTypeQuotaEngine#resources的同名的元素值。

    3. quota-usage 根据租户列出配额的使用量

    cinder quota-usage <tenant_id>

    核心代码跟 cinder quota-show 一样,走的是cinder.quota.DbQuotaDriver#get_project_quotas

    cinder.quota.DbQuotaDriver#get_project_quotas:

        def get_project_quotas(self, context, resources, project_id,
                               quota_class=None, defaults=True,
                               usages=True):
    
            quotas = {}
            # 在数据库表 quotas 根据 project_id 查询字段 hard_limit
            project_quotas = db.quota_get_all_by_project(context, project_id)
            allocated_quotas = None
            default_quotas = None
            # API调用的时候传入的usages=True
            if usages:
                # 根据 project_id 查询数据库表 quota_usages ,获得各quota resource的 in_use和 reserved
                project_usages = db.quota_usage_get_all_by_project(context,
                                                                   project_id)
                # 在数据库表 quotas 根据 project_id 查询字段 allocated
                allocated_quotas = db.quota_allocated_get_all_by_project(
                    context, project_id)
                allocated_quotas.pop('project_id')
    
            # Get the quotas for the appropriate class.  If the project ID
            # matches the one in the context, we use the quota_class from
            # the context, otherwise, we use the provided quota_class (if
            # any) 
            # 如果context有quota_class,用context.quota_class查询class_quotas
            if project_id == context.project_id:
                quota_class = context.quota_class
            if quota_class:
                class_quotas = db.quota_class_get_all_by_name(context, quota_class)
            else:
                class_quotas = {}
    
            for resource in resources.values():
                # Omit default/quota class values
                # defaults = True
                if not defaults and resource.name not in project_quotas:
                    continue
    
                quota_val = project_quotas.get(resource.name)
                # 如果 project_quota 是空,用 class_quota 的值赋值
                if quota_val is None:
                    quota_val = class_quotas.get(resource.name)
                    # 如果 class_quota 赋的值也是空,用 default_quotas 的值赋值
                    if quota_val is None:
                        # Lazy load the default quotas
                        if default_quotas is None:
                            default_quotas = self.get_defaults(
                                context, resources, project_id)
                        quota_val = default_quotas[resource.name]
    
                quotas[resource.name] = {'limit': quota_val}
    
                # Include usages if desired.  This is optional because one
                # internal consumer of this interface wants to access the
                # usages directly from inside a transaction.
                if usages: # True
                    # 设置usage
                    usage = project_usages.get(resource.name, {})
                    quotas[resource.name].update(
                        in_use=usage.get('in_use', 0),
                        reserved=usage.get('reserved', 0), )
                if allocated_quotas: # None
                    quotas[resource.name].update(
                        allocated=allocated_quotas.get(resource.name, 0), )
            return quotas
    

    参考:

    《OpenStack配额(Quota)设计与实现 》

    相关文章

      网友评论

      本文标题:cinder配额 quota 使用及代码分析

      本文链接:https://www.haomeiwen.com/subject/ldgqkxtx.html