美文网首页ZStack博客集我爱编程ZStack
ZStack源码剖析之二次开发——在Utility上堆代码

ZStack源码剖析之二次开发——在Utility上堆代码

作者: 泊浮目 | 来源:发表于2018-05-25 17:28 被阅读15次

    本文首发于泊浮目的专栏:https://segmentfault.com/blog/camile

    背景

    在上篇文章中(ZStack源码剖析之二次开发——可扩展框架
    ),我们简单的了解了一下ZStack核心引擎的二次开发技巧。在这篇文章中,我们将一起来了解ZStack-Utility(即ZStack的Agent端)的二开姿势。

    例子

    我们以ZStack管理节点调用startVm这个api为例子,一起来看一下在agent上的执行逻辑。

        def start(self):
            http_server = kvmagent.get_http_server()
    
            http_server.register_async_uri(self.KVM_START_VM_PATH, self.start_vm)
    

    首先,得注册一个http path用来接受reqeust。

        @kvmagent.replyerror
        def start_vm(self, req):
            cmd = jsonobject.loads(req[http.REQUEST_BODY])
            rsp = StartVmResponse()
            try:
                self._record_operation(cmd.vmInstanceUuid, self.VM_OP_START)
    
                self._start_vm(cmd)
                logger.debug('successfully started vm[uuid:%s, name:%s]' % (cmd.vmInstanceUuid, cmd.vmName))
            except kvmagent.KvmError as e:
                e_str = linux.get_exception_stacktrace()
                logger.warn(e_str)
                if "burst" in e_str and "Illegal" in e_str and "rate" in e_str:
                    rsp.error = "QoS exceed max limit, please check and reset it in zstack"
                elif "cannot set up guest memory" in e_str:
                    logger.warn('unable to start vm[uuid:%s], %s' % (cmd.vmInstanceUuid, e_str))
                    rsp.error = "No enough physical memory for guest"
                else:
                    rsp.error = e_str
                err = self.handle_vfio_irq_conflict(cmd.vmInstanceUuid)
                if err != "":
                    rsp.error = "%s, details: %s" % (err, rsp.error)
                rsp.success = False
            return jsonobject.dumps(rsp)
    

    直接进入主干逻辑,self._start_vm(cmd)

        @lock.lock('libvirt-startvm')
        def _start_vm(self, cmd):
            try:
                vm = get_vm_by_uuid_no_retry(cmd.vmInstanceUuid, False)
    
                if vm:
                    if vm.state == Vm.VM_STATE_RUNNING:
                        raise kvmagent.KvmError(
                            'vm[uuid:%s, name:%s] is already running' % (cmd.vmInstanceUuid, vm.get_name()))
                    else:
                        vm.destroy()
    
                vm = Vm.from_StartVmCmd(cmd)
                vm.start(cmd.timeout)
            except libvirt.libvirtError as e:
                logger.warn(linux.get_exception_stacktrace())
                if "Device or resource busy" in str(e.message):
                    raise kvmagent.KvmError(
                        'unable to start vm[uuid:%s, name:%s], libvirt error: %s' % (
                        cmd.vmInstanceUuid, cmd.vmName, str(e)))
    
                try:
                    vm = get_vm_by_uuid(cmd.vmInstanceUuid)
                    if vm and vm.state != Vm.VM_STATE_RUNNING:
                        raise kvmagent.KvmError(
                           'vm[uuid:%s, name:%s, state:%s] is not in running state, libvirt error: %s' % (
                            cmd.vmInstanceUuid, cmd.vmName, vm.state, str(e)))
    
                except kvmagent.KvmError:
                    raise kvmagent.KvmError(
                        'unable to start vm[uuid:%s, name:%s], libvirt error: %s' % (cmd.vmInstanceUuid, cmd.vmName, str(e)))
    

    关键逻辑:

                vm = Vm.from_StartVmCmd(cmd)
                vm.start(cmd.timeout)
    

    先看from_StartVmCmd

        @staticmethod
        def from_StartVmCmd(cmd):
            use_virtio = cmd.useVirtio
            use_numa = cmd.useNuma
    
            elements = {}
    
            def make_root():
                root = etree.Element('domain')
                root.set('type', 'kvm')
                # self._root.set('type', 'qemu')
                root.set('xmlns:qemu', 'http://libvirt.org/schemas/domain/qemu/1.0')
                elements['root'] = root
    
            def make_cpu():
                if use_numa:
                    root = elements['root']
                    e(root, 'vcpu', '128', {'placement': 'static', 'current': str(cmd.cpuNum)})
                    # e(root,'vcpu',str(cmd.cpuNum),{'placement':'static'})
                    tune = e(root, 'cputune')
                    e(tune, 'shares', str(cmd.cpuSpeed * cmd.cpuNum))
                    # enable nested virtualization
                    if cmd.nestedVirtualization == 'host-model':
                        cpu = e(root, 'cpu', attrib={'mode': 'host-model'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    elif cmd.nestedVirtualization == 'host-passthrough':
                        cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    elif IS_AARCH64:
                        cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    else:
                        cpu = e(root, 'cpu')
                        # e(cpu, 'topology', attrib={'sockets': str(cmd.socketNum), 'cores': str(cmd.cpuOnSocket), 'threads': '1'})
                    mem = cmd.memory / 1024
                    e(cpu, 'topology', attrib={'sockets': str(32), 'cores': str(4), 'threads': '1'})
                    numa = e(cpu, 'numa')
                    e(numa, 'cell', attrib={'id': '0', 'cpus': '0-127', 'memory': str(mem), 'unit': 'KiB'})
                else:
                    root = elements['root']
                    # e(root, 'vcpu', '128', {'placement': 'static', 'current': str(cmd.cpuNum)})
                    e(root, 'vcpu', str(cmd.cpuNum), {'placement': 'static'})
                    tune = e(root, 'cputune')
                    e(tune, 'shares', str(cmd.cpuSpeed * cmd.cpuNum))
                    # enable nested virtualization
                    if cmd.nestedVirtualization == 'host-model':
                        cpu = e(root, 'cpu', attrib={'mode': 'host-model'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    elif cmd.nestedVirtualization == 'host-passthrough':
                        cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    elif IS_AARCH64:
                        cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
                        e(cpu, 'model', attrib={'fallback': 'allow'})
                    else:
                        cpu = e(root, 'cpu')
                    e(cpu, 'topology', attrib={'sockets': str(cmd.socketNum), 'cores': str(cmd.cpuOnSocket), 'threads': '1'})
    
            def make_memory():
                root = elements['root']
                mem = cmd.memory / 1024
                if use_numa:
                    e(root, 'maxMemory', str(68719476736), {'slots': str(16), 'unit': 'KiB'})
                    # e(root,'memory',str(mem),{'unit':'k'})
                    e(root, 'currentMemory', str(mem), {'unit': 'k'})
                else:
                    e(root, 'memory', str(mem), {'unit': 'k'})
                    e(root, 'currentMemory', str(mem), {'unit': 'k'})
    
            def make_os():
                root = elements['root']
                os = e(root, 'os')
                if IS_AARCH64:
                    e(os, 'type', 'hvm', attrib={'arch': 'aarch64'})
                    e(os, 'loader', '/usr/share/edk2.git/aarch64/QEMU_EFI-pflash.raw', attrib={'readonly': 'yes', 'type': 'pflash'})
                else:
                    e(os, 'type', 'hvm', attrib={'machine': 'pc'})
                # if not booting from cdrom, don't add any boot element in os section
                if cmd.bootDev[0] == "cdrom":
                    for boot_dev in cmd.bootDev:
                        e(os, 'boot', None, {'dev': boot_dev})
    
                if cmd.useBootMenu:
                    e(os, 'bootmenu', attrib={'enable': 'yes'})
    
            def make_features():
                root = elements['root']
                features = e(root, 'features')
                for f in ['acpi', 'apic', 'pae']:
                    e(features, f)
                if cmd.kvmHiddenState == True:
                    kvm = e(features, "kvm")
                    e(kvm, 'hidden', None, {'state': 'on'})
    
            def make_devices():
                root = elements['root']
                devices = e(root, 'devices')
                if cmd.addons and cmd.addons['qemuPath']:
                    e(devices, 'emulator', cmd.addons['qemuPath'])
                else:
                    e(devices, 'emulator', kvmagent.get_qemu_path())
                tablet = e(devices, 'input', None, {'type': 'tablet', 'bus': 'usb'})
                e(tablet, 'address', None, {'type':'usb', 'bus':'0', 'port':'1'})
                if IS_AARCH64:
                    keyboard = e(devices, 'input', None, {'type': 'keyboard', 'bus': 'usb'})
                elements['devices'] = devices
    
            def make_cdrom():
                devices = elements['devices']
    
                MAX_CDROM_NUM = len(Vm.ISO_DEVICE_LETTERS)
                EMPTY_CDROM_CONFIGS = None
    
                if IS_AARCH64:
                    # AArch64 Does not support the attachment of multiple iso
                    EMPTY_CDROM_CONFIGS = [
                        EmptyCdromConfig(None, None, None)
                    ]
                else:
                    # bus 0 unit 0 already use by root volume
                    EMPTY_CDROM_CONFIGS = [
                        EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[0], '0', '1'),
                        EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[1], '1', '0'),
                        EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[2], '1', '1')
                    ]
    
                if len(EMPTY_CDROM_CONFIGS) != MAX_CDROM_NUM:
                    logger.error('ISO_DEVICE_LETTERS or EMPTY_CDROM_CONFIGS config error')
    
                def makeEmptyCdrom(targetDev, bus, unit):
                    cdrom = e(devices, 'disk', None, {'type': 'file', 'device': 'cdrom'})
                    e(cdrom, 'driver', None, {'name': 'qemu', 'type': 'raw'})
                    if IS_AARCH64:
                        e(cdrom, 'target', None, {'dev': 'sdc', 'bus': 'scsi'})
                    else:
                        e(cdrom, 'target', None, {'dev': targetDev, 'bus': 'ide'})
                        e(cdrom, 'address', None,{'type' : 'drive', 'bus' : bus, 'unit' : unit})
                    e(cdrom, 'readonly', None)
                    return cdrom
    
                if not cmd.bootIso:
                    for config in EMPTY_CDROM_CONFIGS:
                        makeEmptyCdrom(config.targetDev, config.bus, config.unit)
                    return
    
                notEmptyCdrom = set([])
                for iso in cmd.bootIso:
                    notEmptyCdrom.add(iso.deviceId)
                    cdromConfig = EMPTY_CDROM_CONFIGS[iso.deviceId]
                    if iso.path.startswith('ceph'):
                        ic = IsoCeph()
                        ic.iso = iso
                        devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
                    elif iso.path.startswith('fusionstor'):
                        ic = IsoFusionstor()
                        ic.iso = iso
                        devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
                    else:
                        cdrom = makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit)
                        e(cdrom, 'source', None, {'file': iso.path})
    
                emptyCdrom = set(range(MAX_CDROM_NUM)).difference(notEmptyCdrom)
                for i in emptyCdrom:
                    cdromConfig = EMPTY_CDROM_CONFIGS[i]
                    makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus, cdromConfig.unit)
    
            def make_volumes():
                devices = elements['devices']
                volumes = [cmd.rootVolume]
                volumes.extend(cmd.dataVolumes)
    
                def filebased_volume(_dev_letter, _v):
                    disk = etree.Element('disk', {'type': 'file', 'device': 'disk', 'snapshot': 'external'})
                    e(disk, 'driver', None, {'name': 'qemu', 'type': linux.get_img_fmt(_v.installPath), 'cache': _v.cacheMode})
                    e(disk, 'source', None, {'file': _v.installPath})
    
                    if _v.shareable:
                        e(disk, 'shareable')
    
                    if _v.useVirtioSCSI:
                        e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'scsi'})
                        e(disk, 'wwn', _v.wwn)
                        e(disk, 'address', None, {'type': 'drive', 'controller': '0', 'unit': str(_v.deviceId)})
                        return disk
    
                    if _v.useVirtio:
                        e(disk, 'target', None, {'dev': 'vd%s' % _dev_letter, 'bus': 'virtio'})
                    elif IS_AARCH64:
                        e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'scsi'})
                    else:
                        e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'ide'})
                    return disk
    
                def iscsibased_volume(_dev_letter, _v):
                    def blk_iscsi():
                        bi = BlkIscsi()
                        portal, bi.target, bi.lun = _v.installPath.lstrip('iscsi://').split('/')
                        bi.server_hostname, bi.server_port = portal.split(':')
                        bi.device_letter = _dev_letter
                        bi.volume_uuid = _v.volumeUuid
                        bi.chap_username = _v.chapUsername
                        bi.chap_password = _v.chapPassword
    
                        return bi.to_xmlobject()
    
                    def virtio_iscsi():
                        vi = VirtioIscsi()
                        portal, vi.target, vi.lun = _v.installPath.lstrip('iscsi://').split('/')
                        vi.server_hostname, vi.server_port = portal.split(':')
                        vi.device_letter = _dev_letter
                        vi.volume_uuid = _v.volumeUuid
                        vi.chap_username = _v.chapUsername
                        vi.chap_password = _v.chapPassword
    
                        return vi.to_xmlobject()
    
                    if _v.useVirtio:
                        return virtio_iscsi()
                    else:
                        return blk_iscsi()
    
                def ceph_volume(_dev_letter, _v):
                    def ceph_virtio():
                        vc = VirtioCeph()
                        vc.volume = _v
                        vc.dev_letter = _dev_letter
                        return vc.to_xmlobject()
    
                    def ceph_blk():
                        if not IS_AARCH64:
                            ic = IdeCeph()
                        else:
                            ic = ScsiCeph()
                        ic.volume = _v
                        ic.dev_letter = _dev_letter
                        return ic.to_xmlobject()
    
                    def ceph_virtio_scsi():
                        vsc = VirtioSCSICeph()
                        vsc.volume = _v
                        vsc.dev_letter = _dev_letter
                        return vsc.to_xmlobject()
    
                    if _v.useVirtioSCSI:
                        disk = ceph_virtio_scsi()
                        if _v.shareable:
                            e(disk, 'shareable')
                        return disk
    
                    if _v.useVirtio:
                        return ceph_virtio()
                    else:
                        return ceph_blk()
    
                def fusionstor_volume(_dev_letter, _v):
                    def fusionstor_virtio():
                        vc = VirtioFusionstor()
                        vc.volume = _v
                        vc.dev_letter = _dev_letter
                        return vc.to_xmlobject()
    
                    def fusionstor_blk():
                        ic = IdeFusionstor()
                        ic.volume = _v
                        ic.dev_letter = _dev_letter
                        return ic.to_xmlobject()
    
                    def fusionstor_virtio_scsi():
                        vsc = VirtioSCSIFusionstor()
                        vsc.volume = _v
                        vsc.dev_letter = _dev_letter
                        return vsc.to_xmlobject()
    
                    if _v.useVirtioSCSI:
                        disk = fusionstor_virtio_scsi()
                        if _v.shareable:
                            e(disk, 'shareable')
                        return disk
    
                    if _v.useVirtio:
                        return fusionstor_virtio()
                    else:
                        return fusionstor_blk()
    
                def volume_qos(volume_xml_obj):
                    if not cmd.addons:
                        return
    
                    vol_qos = cmd.addons['VolumeQos']
                    if not vol_qos:
                        return
    
                    qos = vol_qos[v.volumeUuid]
                    if not qos:
                        return
    
                    if not qos.totalBandwidth and not qos.totalIops:
                        return
    
                    iotune = e(volume_xml_obj, 'iotune')
                    if qos.totalBandwidth:
                        e(iotune, 'total_bytes_sec', str(qos.totalBandwidth))
                    if qos.totalIops:
                        # e(iotune, 'total_iops_sec', str(qos.totalIops))
                        e(iotune, 'read_iops_sec', str(qos.totalIops))
                        e(iotune, 'write_iops_sec', str(qos.totalIops))
                        # e(iotune, 'read_iops_sec_max', str(qos.totalIops))
                        # e(iotune, 'write_iops_sec_max', str(qos.totalIops))
                        # e(iotune, 'total_iops_sec_max', str(qos.totalIops))
    
                volumes.sort(key=lambda d: d.deviceId)
                scsi_device_ids = [v.deviceId for v in volumes if v.useVirtioSCSI]
                for v in volumes:
                    if v.deviceId >= len(Vm.DEVICE_LETTERS):
                        err = "exceeds max disk limit, it's %s but only 26 allowed" % v.deviceId
                        logger.warn(err)
                        raise kvmagent.KvmError(err)
    
                    dev_letter = Vm.DEVICE_LETTERS[v.deviceId]
                    if v.useVirtioSCSI:
                        dev_letter = Vm.DEVICE_LETTERS[scsi_device_ids.pop()]
    
                    if v.deviceType == 'file':
                        vol = filebased_volume(dev_letter, v)
                    elif v.deviceType == 'iscsi':
                        vol = iscsibased_volume(dev_letter, v)
                    elif v.deviceType == 'ceph':
                        vol = ceph_volume(dev_letter, v)
                    elif v.deviceType == 'fusionstor':
                        vol = fusionstor_volume(dev_letter, v)
                    else:
                        raise Exception('unknown volume deviceType: %s' % v.deviceType)
    
                    assert vol is not None, 'vol cannot be None'
                    # set boot order for root volume when boot from hd
                    if v.deviceId == 0 and cmd.bootDev[0] == 'hd' and cmd.useBootMenu:
                        e(vol, 'boot', None, {'order': '1'})
                    volume_qos(vol)
                    devices.append(vol)
    
            def make_nics():
                if not cmd.nics:
                    return
    
                def nic_qos(nic_xml_object):
                    if not cmd.addons:
                        return
    
                    nqos = cmd.addons['NicQos']
                    if not nqos:
                        return
    
                    qos = nqos[nic.uuid]
                    if not qos:
                        return
    
                    if not qos.outboundBandwidth and not qos.inboundBandwidth:
                        return
    
                    bandwidth = e(nic_xml_object, 'bandwidth')
                    if qos.outboundBandwidth:
                        e(bandwidth, 'outbound', None, {'average': str(qos.outboundBandwidth / 1024 / 8)})
                    if qos.inboundBandwidth:
                        e(bandwidth, 'inbound', None, {'average': str(qos.inboundBandwidth / 1024 / 8)})
    
                devices = elements['devices']
                for nic in cmd.nics:
                    interface = e(devices, 'interface', None, {'type': 'bridge'})
                    e(interface, 'mac', None, {'address': nic.mac})
                    if nic.ip is not None and nic.ip != "":
                        filterref = e(interface, 'filterref', None, {'filter':'clean-traffic'})
                        e(filterref, 'parameter', None, {'name':'IP', 'value': nic.ip})
                    e(interface, 'alias', None, {'name': 'net%s' % nic.nicInternalName.split('.')[1]})
                    e(interface, 'source', None, {'bridge': nic.bridgeName})
                    if use_virtio:
                        e(interface, 'model', None, {'type': 'virtio'})
                    else:
                        e(interface, 'model', None, {'type': 'e1000'})
                    e(interface, 'target', None, {'dev': nic.nicInternalName})
    
                    nic_qos(interface)
    
            def make_meta():
                root = elements['root']
    
                e(root, 'name', cmd.vmInstanceUuid)
                e(root, 'uuid', uuidhelper.to_full_uuid(cmd.vmInstanceUuid))
                e(root, 'description', cmd.vmName)
                e(root, 'on_poweroff', 'destroy')
                e(root, 'on_crash', 'restart')
                e(root, 'on_reboot', 'restart')
                meta = e(root, 'metadata')
                zs = e(meta, 'zstack', usenamesapce=True)
                e(zs, 'internalId', str(cmd.vmInternalId))
                e(zs, 'hostManagementIp', str(cmd.hostManagementIp))
                clock = e(root, 'clock', None, {'offset': cmd.clock})
                if cmd.clock == 'localtime':
                    e(clock, 'timer', None, {'name': 'rtc', 'tickpolicy': 'catchup'})
                    e(clock, 'timer', None, {'name': 'pit', 'tickpolicy': 'delay'})
                    e(clock, 'timer', None, {'name': 'hpet', 'present': 'no'})
                    e(clock, 'timer', None, {'name': 'hypervclock', 'present': 'yes'})
    
            def make_vnc():
                devices = elements['devices']
                if cmd.consolePassword == None:
                    vnc = e(devices, 'graphics', None, {'type': 'vnc', 'port': '5900', 'autoport': 'yes'})
                else:
                    vnc = e(devices, 'graphics', None,
                            {'type': 'vnc', 'port': '5900', 'autoport': 'yes', 'passwd': str(cmd.consolePassword)})
                e(vnc, "listen", None, {'type': 'address', 'address': '0.0.0.0'})
    
            def make_spice():
                devices = elements['devices']
                spice = e(devices, 'graphics', None, {'type': 'spice', 'port': '5900', 'autoport': 'yes'})
                e(spice, "listen", None, {'type': 'address', 'address': '0.0.0.0'})
                e(spice, "image", None, {'compression': 'auto_glz'})
                e(spice, "jpeg", None, {'compression': 'always'})
                e(spice, "zlib", None, {'compression': 'never'})
                e(spice, "playback", None, {'compression': 'off'})
                e(spice, "streaming", None, {'mode': cmd.spiceStreamingMode})
                e(spice, "mouse", None, {'mode': 'client'})
                e(spice, "filetransfer", None, {'enable': 'no'})
                e(spice, "clipboard", None, {'copypaste': 'no'})
    
            def make_usb_redirect():
                if cmd.usbRedirect == "true":
                    devices = elements['devices']
                    e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-ehci1'})
                    e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci1', 'multifunction': 'on'})
                    e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci2'})
                    e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci3'})
    
                    chan = e(devices, 'channel', None, {'type': 'spicevmc'})
                    e(chan, 'target', None, {'type': 'virtio', 'name': 'com.redhat.spice.0'})
                    e(chan, 'address', None, {'type': 'virtio-serial'})
    
                    redirdev2 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
                    e(redirdev2, 'address', None, {'type': 'usb', 'bus': '0', 'port': '2'})
                    redirdev3 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
                    e(redirdev3, 'address', None, {'type': 'usb', 'bus': '0', 'port': '3'})
                    redirdev4 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
                    e(redirdev4, 'address', None, {'type': 'usb', 'bus': '0', 'port': '4'})
                    redirdev5 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
                    e(redirdev5, 'address', None, {'type': 'usb', 'bus': '0', 'port': '6'})
                else:
                    # make sure there are three default usb controllers, for usb 1.1/2.0/3.0
                    devices = elements['devices']
                    e(devices, 'controller', None, {'type': 'usb', 'index': '0'})
                    if not IS_AARCH64:
                        e(devices, 'controller', None, {'type': 'usb', 'index': '1', 'model': 'ehci'})
                        e(devices, 'controller', None, {'type': 'usb', 'index': '2', 'model': 'nec-xhci'})
    
            def make_video():
                devices = elements['devices']
                if IS_AARCH64:
                    video = e(devices, 'video')
                    e(video, 'model', None, {'type': 'virtio'})
                elif cmd.videoType != "qxl":
                    video = e(devices, 'video')
                    e(video, 'model', None, {'type': str(cmd.videoType)})
                else:
                    for monitor in range(cmd.VDIMonitorNumber):
                        video = e(devices, 'video')
                        e(video, 'model', None, {'type': str(cmd.videoType)})
    
    
            def make_audio_microphone():
                if cmd.consoleMode == 'spice':
                    devices = elements['devices']
                    e(devices, 'sound',None,{'model':'ich6'})
                else:
                    return
    
            def make_graphic_console():
                if cmd.consoleMode == 'spice':
                    make_spice()
                else:
                    make_vnc()
    
            def make_addons():
                if not cmd.addons:
                    return
    
                devices = elements['devices']
                channel = cmd.addons['channel']
                if channel:
                    basedir = os.path.dirname(channel.socketPath)
                    linux.mkdir(basedir, 0777)
                    chan = e(devices, 'channel', None, {'type': 'unix'})
                    e(chan, 'source', None, {'mode': 'bind', 'path': channel.socketPath})
                    e(chan, 'target', None, {'type': 'virtio', 'name': channel.targetName})
    
                cephSecretKey = cmd.addons['ceph_secret_key']
                cephSecretUuid = cmd.addons['ceph_secret_uuid']
                if cephSecretKey and cephSecretUuid:
                    VmPlugin._create_ceph_secret_key(cephSecretKey, cephSecretUuid)
    
                pciDevices = cmd.addons['pciDevice']
                if pciDevices:
                    make_pci_device(pciDevices)
    
                usbDevices = cmd.addons['usbDevice']
                if usbDevices:
                    make_usb_device(usbDevices)
    
            def make_pci_device(addresses):
                devices = elements['devices']
                for addr in addresses:
                    if match_pci_device(addr):
                        hostdev = e(devices, "hostdev", None, {'mode': 'subsystem', 'type': 'pci', 'managed': 'yes'})
                        e(hostdev, "driver", None, {'name': 'vfio'})
                        source = e(hostdev, "source")
                        e(source, "address", None, {
                            "domain": hex(0) if len(addr.split(":")) == 2 else hex(int(addr.split(":")[0], 16)),
                            "bus": hex(int(addr.split(":")[-2], 16)),
                            "slot": hex(int(addr.split(":")[-1].split(".")[0], 16)),
                            "function": hex(int(addr.split(":")[-1].split(".")[1], 16))
                        })
                    else:
                        raise kvmagent.KvmError(
                           'can not find pci device for address %s' % addr)
    
            def make_usb_device(usbDevices):
                next_uhci_port = 2
                next_ehci_port = 1
                next_xhci_port = 1
                devices = elements['devices']
                for usb in usbDevices:
                    if match_usb_device(usb):
                        hostdev = e(devices, "hostdev", None, {'mode': 'subsystem', 'type': 'usb', 'managed': 'yes'})
                        source = e(hostdev, "source")
                        e(source, "address", None, {
                            "bus": str(int(usb.split(":")[0])),
                            "device": str(int(usb.split(":")[1]))
                        })
                        e(source, "vendor", None, {
                            "id": hex(int(usb.split(":")[2], 16))
                        })
                        e(source, "product", None, {
                            "id": hex(int(usb.split(":")[3], 16))
                        })
    
                        # get controller index from usbVersion
                        # eg. 1.1 -> 0
                        # eg. 2.0.0 -> 1
                        # eg. 3 -> 2
                        bus = int(usb.split(":")[4][0]) - 1
                        if bus == 0:
                            address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_uhci_port)})
                            next_uhci_port += 1
                        elif bus == 1:
                            address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_ehci_port)})
                            next_ehci_port += 1
                        elif bus == 2:
                            address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_xhci_port)})
                            next_xhci_port += 1
                        else:
                            raise kvmagent.KvmError('unknown usb controller %s', bus)
                    else:
                        raise kvmagent.KvmError('cannot find usb device %s', usb)
    
            # TODO(WeiW) Validate here
            def match_pci_device(addr):
                return True
    
            def match_usb_device(addr):
                if len(addr.split(':')) == 5:
                    return True
                else:
                    return False
    
            def make_balloon_memory():
                devices = elements['devices']
                b = e(devices, 'memballoon', None, {'model': 'virtio'})
                e(b, 'stats', None, {'period': '10'})
    
            def make_console():
                devices = elements['devices']
                serial = e(devices, 'serial', None, {'type': 'pty'})
                e(serial, 'target', None, {'port': '0'})
                console = e(devices, 'console', None, {'type': 'pty'})
                e(console, 'target', None, {'type': 'serial', 'port': '0'})
    
            def make_sec_label():
                root = elements['root']
                e(root, 'seclabel', None, {'type': 'none'})
    
            def make_controllers():
                devices = elements['devices']
                e(devices, 'controller', None, {'type': 'scsi', 'model': 'virtio-scsi'})
    
            make_root()
            make_meta()
            make_cpu()
            make_memory()
            make_os()
            make_features()
            make_devices()
            make_video()
            make_audio_microphone()
            make_nics()
            make_volumes()
            make_cdrom()
            make_graphic_console()
            make_usb_redirect()
            make_addons()
            make_balloon_memory()
            make_console()
            make_sec_label()
            make_controllers()
    
            root = elements['root']
            xml = etree.tostring(root)
    
            vm = Vm()
            vm.uuid = cmd.vmInstanceUuid
            vm.domain_xml = xml
            vm.domain_xmlobject = xmlobject.loads(xml)
            return vm
    

    显然,上述逻辑是在组装一份xml,便于之后的libvirt使用。

    然后是

     vm.start(cmd.timeout)
    

    可以看到,这里是直接调用了libvirt的sdk。

    这仅仅是一个调用流程。而在很多地方,来自MN的请求会直接调用linux的shell命令,详情见linux.py。(获取云盘大小、主存储容量等)。

    问题

    在基于扩展ZStack的Agent时,如果是一个全新的功能模块,可能并不会造成和原有代码的深度耦合。但如果在原有功能上的增强, 对原有代码进行修改可能会导致我们的业务逻辑和Utility的上游代码耦合。而在没有足够人力来维护、开发ZStack时,我们会将目标定为能够及时跟上发布版本。 因此,我们要尽量减少冲突。

    举个例子:我们要对启动vm的逻辑进行增强,添加一个自己的配置写入xml。这段代码如果写进了vm_plugin.py,那么就是一个耦合。耦合多了以后,跟上发布版本就会很困难。

    解决方案

    这是一个参考方案:

    如果是引入一个全新的功能模块,建议重写一个项目。无论是代码规范还是自动化测试,都可以有一个很好的实践。

    如果是基于Utility的扩展,比如对于扩展的api——APIStartVmInstanceExMsg。由上游发送http request时,将指定v2版本的agent。比如原有start vm会发送至path:AGENT_IP:7070/vm/start;而如果我们增强了这部分逻辑,将这段代码copy至vm_plugin_ex.py,并注册一个path,ex/vm/start。当然port也要重新注册一个,就像这样::AGENT_IP:7071/ex/vm/start

    同样的,对linux.py扩展时,复制一个linux2.py来存放属于我们自己的扩展逻辑。

    相关文章

      网友评论

        本文标题:ZStack源码剖析之二次开发——在Utility上堆代码

        本文链接:https://www.haomeiwen.com/subject/ekyujftx.html