美文网首页
多进程共享的优先级队列

多进程共享的优先级队列

作者: ThomasYoungK | 来源:发表于2018-07-30 07:48 被阅读46次

PriorityQueue是多线程共享的,怎么让它变成多进程共享呢?
原有的PriorityQueue可以继续使用,但是,通过managers模块把PriorityQueue通过网络暴露出去,就可以让其他机器的进程(或本地机子的其他进程)访问Queue了。

事实上,普通的变量也都可以进程间共享:
https://blog.csdn.net/Imagine_Dragon/article/details/77689194

Managers provide a way to create data which can be shared between different processes, including sharing over a network between processes running on different machines. A manager object controls a server process which manages shared objects. Other processes can access the shared objects by using proxies.

我这边有一个类似的例子,但是不是用多台机子,而是单机多进程:

import time
from random import randint
from queue import PriorityQueue
from multiprocessing import Process
from multiprocessing.managers import BaseManager


class Manager(BaseManager):
    pass


Manager.register('get_priorityQueue', PriorityQueue)


def double(n):
    return n * 2


def producer(q):
    count = 0
    while 1:
        if count > 5:
            break
        pri = randint(0, 100)
        print(f'put :{pri}')
        q.put((pri, double, pri))  # (priority, func, args)
        count += 1


def consumer(q):
    while 1:
        if q.empty():
            break
        pri, task, arg = q.get()
        print(f'[PRI:{pri}] {arg} * 2 = {task(arg)}')
        q.task_done()
        time.sleep(0.1)


m = Manager()
m.start()
q = m.get_priorityQueue()   # q是个代理(proxy),等于是用manager构造了一个多进程可以共享的PriorityQueue, 而本来PriorityQueue只能线程间共享, 等于是用manager构造了一个多进程可以共享的PriorityQueue, 而本来PriorityQueue只能线程间共享

t = Process(target=producer, args=(q,))
t.start()
time.sleep(1)
t = Process(target=consumer, args=(q,))
t.start()
t.join()

相关文章

网友评论

      本文标题:多进程共享的优先级队列

      本文链接:https://www.haomeiwen.com/subject/uxdxvftx.html