美文网首页
并发编程之concurrent.futures模块

并发编程之concurrent.futures模块

作者: Chaweys | 来源:发表于2021-12-29 01:49 被阅读0次

    标准库为我们提供了concurrent.futures模块,它提供了ThreadPoolExecutor和ProcessPoolExecutor两个类,
    实现了对threading和multiprocessing的进一步抽象,对编写线程池/进程池提供了直接的支持。
    注:pool.submit(func,*args,**kwargs),关于入参:
    func:    表示调用的自定义函数名
    *args:   表示该自定义函数名需要的入参:元组、单个参数形式,
              如pool.submit(func,arg1,arg2,arg3=3)
              表示该自定义函数的传参形式为:func(arg1,arg2,arg3=3)
              
    **kwargs:表示该自定义函数名需要的入参:字典参数形式
    
    
    举例1:ProcessPoolExecutor
    from concurrent.futures import ProcessPoolExecutor
    import time
    
    def task(name):
        print("name:", name)
        time.sleep(2)
    
    if __name__ == "__main__":
        time1 = time.time()
        with ProcessPoolExecutor(5) as pool:
            future_result1 = pool.submit(task, "name1")
            future_result2 = pool.submit(task, "name2")
            future_result3 = pool.submit(task, "name3")
    
            #pool.shutdown(wait=True)
    
        time2 = time.time()
        print("time2 - time1", time2 - time1)
    """
    结果:由结果看出,程序执行结果并不是累计等待2+2+2=6秒,而是并发执行
    name: name1
    name: name2
    name: name3
    time2 - time1 2.439988851547241
    
    解释:
    ProcessPoolExecutor(5)创建一个进程池,容量为5,并一共创建三个进程(future_result1,future_result2,future_result3),
    然后再线程池队列里并行执行该三个进程.
    且pool.shutdown(wait=True)表示创建的三个进程都执行完毕才执行该程序的主进程,这里使用with语句默认调用了pool.shutdown(wait=True),因此可以省略;
    且pool.shutdown(wait=True)的wait参数值默认即为True
    
    
    实际断言方法并行时用到如下方式:
    with ProcessPoolExecutor(5) as pool:
        future_result1 = pool.submit(task, "name1")  #但是submit提交时是异步的,这样就不用等待提交得到结果再去下一行提交代码
        future_result2 = pool.submit(task, "name2")
        future_result3 = pool.submit(task, "name3")
        
        future_result1.result() #同步调用
        future_result2.result() #同步调用
        future_result3.result() #同步调用
    
    """
    
    
    举例2:
    from concurrent.futures import ProcessPoolExecutor
    import time
    
    def task(name):
        print("name:", name)
        time.sleep(2)
    
    if __name__ == "__main__":
        time1 = time.time()
        pool = ProcessPoolExecutor(5)
    
        for i in range(3):
            future_result1 = pool.submit(task, "name{}".format(i))
        pool.shutdown(wait=True)
    
        print("main")
        time2 = time.time()
        print("time2 - time1", time2 - time1)
    """
    结果:由结果看出,程序执行结果并不是累计等待2+2+2=6秒,而是并发执行
    且pool.shutdown(wait=True)的参数为True,表示先执行完创建的子进程,再执行主进程 main
    name: name0
    name: name1
    name: name2
    main
    time2 - time1 2.2973177433013916
    
        
    pool.shutdown(wait=False)的结果:
    main
    time2 - time1 0.26048994064331055
    name: name0
    name: name1
    name: name2
    """
    
    
    
    举例3:使用submit同步调用(实际接口测试中使用过)
    同步调用:提交/调用一个任务,然后就在原地等着,等到该任务执行完毕拿到结果,再执行下一行代码
    from concurrent.futures import ProcessPoolExecutor
    import time
    import os
    
    def task(name):
        print("{}子进程{}:".format(name,os.getpid()))
        time.sleep(2)
        return name+str(2)
    
    if __name__ == "__main__":
        time1 = time.time()
        pool = ProcessPoolExecutor(5)
    
        results = []
        for i in range(3):
            future_result = pool.submit(task, "name{}".format(i))
            res=future_result.result()  #同步调用,等到结果后再执行下一行代码
            print(res)
    
        print("主进程{}".format(os.getpid()))
        time2 = time.time()
        print("time2 - time1", time2 - time1)
    """
    同步调用结果:由结果得知,每个进程得出了结果才去执行下一行代码
    name0子进程37512:
    name02
    name1子进程35884:
    name12
    name2子进程33600:
    name22
    主进程36168
    time2 - time1 6.2372283935546875
    """
        
        
    
    举例4:使用submit异步调用
    异步调用: 提交/调用一个任务,不在原地等着,直接执行下一行代码
    from concurrent.futures import ProcessPoolExecutor
    import time
    import os
    
    def task(name):
        print("{}子进程{}:".format(name,os.getpid()))
        time.sleep(2)
        return name+str(2)
    
    if __name__ == "__main__":
        time1 = time.time()
        pool = ProcessPoolExecutor(5)
    
        results = []
        for i in range(3):
            future_result = pool.submit(task, "name{}".format(i))
            results.append(future_result)   #异步调用,不用等到结果,直接执行下一行代码
    
        for result in results:
            print(result.result())
    
        print("主进程{}".format(os.getpid()))
        time2 = time.time()
        print("time2 - time1", time2 - time1)
        
    """
    异步调用结果:由结果得治,每个进程执行完成之后,并未等待结果,而是直接执行下一行代码
    name0子进程36116:
    name1子进程31972:
    name2子进程660:
    name02
    name12
    name22
    主进程42280
    time2 - time1 2.342352867126465
    """
    
    
    
    
    
    

    相关文章

      网友评论

          本文标题:并发编程之concurrent.futures模块

          本文链接:https://www.haomeiwen.com/subject/tzamqrtx.html