美文网首页
多进程共享的优先级队列

多进程共享的优先级队列

作者: ThomasYoungK | 来源:发表于2018-07-30 07:48 被阅读46次

    PriorityQueue是多线程共享的,怎么让它变成多进程共享呢?
    原有的PriorityQueue可以继续使用,但是,通过managers模块把PriorityQueue通过网络暴露出去,就可以让其他机器的进程(或本地机子的其他进程)访问Queue了。

    事实上,普通的变量也都可以进程间共享:
    https://blog.csdn.net/Imagine_Dragon/article/details/77689194

    Managers provide a way to create data which can be shared between different processes, including sharing over a network between processes running on different machines. A manager object controls a server process which manages shared objects. Other processes can access the shared objects by using proxies.

    我这边有一个类似的例子,但是不是用多台机子,而是单机多进程:

    import time
    from random import randint
    from queue import PriorityQueue
    from multiprocessing import Process
    from multiprocessing.managers import BaseManager
    
    
    class Manager(BaseManager):
        pass
    
    
    Manager.register('get_priorityQueue', PriorityQueue)
    
    
    def double(n):
        return n * 2
    
    
    def producer(q):
        count = 0
        while 1:
            if count > 5:
                break
            pri = randint(0, 100)
            print(f'put :{pri}')
            q.put((pri, double, pri))  # (priority, func, args)
            count += 1
    
    
    def consumer(q):
        while 1:
            if q.empty():
                break
            pri, task, arg = q.get()
            print(f'[PRI:{pri}] {arg} * 2 = {task(arg)}')
            q.task_done()
            time.sleep(0.1)
    
    
    m = Manager()
    m.start()
    q = m.get_priorityQueue()   # q是个代理(proxy),等于是用manager构造了一个多进程可以共享的PriorityQueue, 而本来PriorityQueue只能线程间共享, 等于是用manager构造了一个多进程可以共享的PriorityQueue, 而本来PriorityQueue只能线程间共享
    
    t = Process(target=producer, args=(q,))
    t.start()
    time.sleep(1)
    t = Process(target=consumer, args=(q,))
    t.start()
    t.join()
    
    

    相关文章

      网友评论

          本文标题:多进程共享的优先级队列

          本文链接:https://www.haomeiwen.com/subject/uxdxvftx.html