美文网首页
Python协程遐想

Python协程遐想

作者: 第八共同体 | 来源:发表于2017-08-09 11:28 被阅读0次

自己在学习协程的时候,对一些东西拿捏的不准,甚至是不解,搞出这样的两个脚本来,让自己更糊涂了,请教了很多人,估计是因为这个问题很愚蠢,所以也没人搭理,后来自己想了想,发现结果很明了,结合之前学习到的理论,有了一些认识

image.png

1.同步脚本在执行的时候,print_time函数总共耗时是3s,因为该函数总共调用3次,每次sleep1s,该sleep操作是阻塞的。
2.协程方式的print_time耗时很少,函数中的sleep操作是非阻塞的。这里结果显示的调用6次,可以以上图那么解释,其实是上下文切换了6次。

# 脚本demo2_1.py
import asyncio
import datetime
import time

@asyncio.coroutine
def print_time():
    print(datetime.datetime.now())
    yield from asyncio.sleep(1)

@asyncio.coroutine
def display_date(loop):
    end_time = loop.time() + 5.0
    while True:
        yield from print_time()
        if (loop.time() + 1.0) >= end_time:
            break
        yield from asyncio.sleep(1)
    return "Coroutine is finished..."



if __name__ =='__main__':
    import cProfile
    start_time = time.time()
    if sys.platform == "win32":
        loops = asyncio.ProactorEventLoop()
        asyncio.set_event_loop(loops)
    else:
        loops = asyncio.get_event_loop()
    # Blocking call which returns when the hello_world() coroutine is done
    try:
        cProfile.run("loops.run_until_complete(display_date(loops))")
        # print(loops.run_until_complete(display_date(loops)))
        # print('Used: ', time.time()-start_time)
    except KeyboardInterrupt as e:
        for task in asyncio.Task.all_tasks():
            print(task.cancel())
        loops.stop()
        try:
            loops.run_forever()
        finally:
            loops.close()
# 脚本demo2_2.py
import datetime
import time



def print_time():
    print(datetime.datetime.now())
    time.sleep(1)

def display_date():
    end_time = time.time() + 5.0
    while True:
        print_time()
        if (time.time() + 1.0) >= end_time:
            break
        time.sleep(1)


if __name__ == '__main__':
    import cProfile
    start_time = time.time()
    cProfile.run("display_date()")
    print('Used: ', time.time() - start_time)

脚本demo2_1.py的运行结果如下:

2017-08-09 12:14:51.292522
2017-08-09 12:14:53.293397
2017-08-09 12:14:55.293789
         421 function calls in 5.002 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    5.002    5.002 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 _weakrefset.py:38(_remove)
        1    0.000    0.000    0.000    0.000 _weakrefset.py:70(__contains__)
        2    0.000    0.000    0.000    0.000 _weakrefset.py:81(add)
        5    0.000    0.000    0.000    0.000 base_events.py:1325(_timer_handle_cancelled)
       12    0.000    0.000    5.002    0.417 base_events.py:1330(_run_once)
        2    0.000    0.000    0.000    0.000 base_events.py:1429(_set_coroutine_wrapper)
       24    0.000    0.000    0.000    0.000 base_events.py:1462(get_debug)
        1    0.000    0.000    0.000    0.000 base_events.py:176(_run_until_complete_cb)
        5    0.000    0.000    0.000    0.000 base_events.py:273(create_future)
        1    0.000    0.000    0.000    0.000 base_events.py:277(create_task)
       15    0.000    0.000    0.000    0.000 base_events.py:355(_check_closed)
        1    0.000    0.000    5.002    5.002 base_events.py:404(run_forever)
        1    0.000    0.000    5.002    5.002 base_events.py:432(run_until_complete)
        1    0.000    0.000    0.000    0.000 base_events.py:469(stop)
        1    0.000    0.000    0.000    0.000 base_events.py:514(is_running)
       26    0.000    0.000    0.000    0.000 base_events.py:518(time)
        5    0.000    0.000    0.000    0.000 base_events.py:527(call_later)
        5    0.000    0.000    0.000    0.000 base_events.py:548(call_at)
        7    0.000    0.000    0.000    0.000 base_events.py:564(call_soon)
        7    0.000    0.000    0.000    0.000 base_events.py:594(_call_soon)
        2    0.000    0.000    0.000    0.000 base_futures.py:23(isfuture)
        1    0.000    0.000    0.000    0.000 coroutines.py:268(iscoroutine)
        6    0.000    0.000    0.001    0.000 demo2_1.py:11(display_date)
        6    0.000    0.000    0.000    0.000 demo2_1.py:6(print_time)
        5    0.000    0.000    0.000    0.000 events.py:114(cancel)
       13    0.000    0.000    0.001    0.000 events.py:125(_run)
        5    0.000    0.000    0.000    0.000 events.py:147(__init__)
        5    0.000    0.000    0.000    0.000 events.py:192(cancel)
        6    0.000    0.000    0.000    0.000 events.py:621(_get_running_loop)
        2    0.000    0.000    0.000    0.000 events.py:632(_set_running_loop)
        5    0.000    0.000    0.000    0.000 events.py:666(get_event_loop)
       12    0.000    0.000    0.000    0.000 events.py:86(__init__)
        5    0.000    0.000    0.000    0.000 futures.py:344(_set_result_unless_cancelled)
        1    0.000    0.000    0.000    0.000 proactor_events.py:477(_loop_self_reading)
       12    0.000    0.000    0.000    0.000 proactor_events.py:539(_process_events)
       10    0.000    0.000    0.000    0.000 tasks.py:468(sleep)
        1    0.000    0.000    0.000    0.000 tasks.py:507(ensure_future)
       12    0.000    0.000    5.000    0.417 windows_events.py:417(select)
        1    0.000    0.000    0.000    0.000 windows_events.py:429(recv)
        1    0.000    0.000    0.000    0.000 windows_events.py:45(__init__)
        1    0.000    0.000    0.000    0.000 windows_events.py:606(_register_with_iocp)
        1    0.000    0.000    0.000    0.000 windows_events.py:616(_register)
       12    0.000    0.000    5.000    0.417 windows_events.py:660(_poll)
        5    0.000    0.000    0.000    0.000 {built-in method _heapq.heappop}
        5    0.000    0.000    0.000    0.000 {built-in method _heapq.heappush}
        1    0.000    0.000    0.000    0.000 {built-in method _overlapped.CreateIoCompletionPort}
       12    5.000    0.417    5.000    0.417 {built-in method _overlapped.GetQueuedCompletionStatus}
        1    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        1    0.000    0.000    5.002    5.002 {built-in method builtins.exec}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
       24    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        5    0.000    0.000    0.000    0.000 {built-in method builtins.max}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.print}
       12    0.000    0.000    0.000    0.000 {built-in method math.ceil}
        3    0.000    0.000    0.000    0.000 {built-in method now}
        7    0.000    0.000    0.000    0.000 {built-in method nt.getpid}
        1    0.000    0.000    0.000    0.000 {built-in method sys.get_asyncgen_hooks}
        2    0.000    0.000    0.000    0.000 {built-in method sys.set_asyncgen_hooks}
       26    0.000    0.000    0.000    0.000 {built-in method time.monotonic}
        1    0.000    0.000    0.000    0.000 {method 'WSARecv' of '_overlapped.Overlapped' objects}
        2    0.000    0.000    0.000    0.000 {method 'add' of 'set' objects}
        1    0.000    0.000    0.000    0.000 {method 'add_done_callback' of '_asyncio.Future' objects}
        1    0.000    0.000    0.000    0.000 {method 'add_done_callback' of '_asyncio.Task' objects}
       12    0.000    0.000    0.000    0.000 {method 'append' of 'collections.deque' objects}
        5    0.000    0.000    0.000    0.000 {method 'cancelled' of '_asyncio.Future' objects}
       12    0.000    0.000    0.000    0.000 {method 'clear' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'discard' of 'set' objects}
        1    0.000    0.000    0.000    0.000 {method 'done' of '_asyncio.Task' objects}
        2    0.000    0.000    0.000    0.000 {method 'fileno' of '_socket.socket' objects}
       13    0.000    0.000    0.000    0.000 {method 'popleft' of 'collections.deque' objects}
        1    0.000    0.000    0.000    0.000 {method 'remove_done_callback' of '_asyncio.Task' objects}
        1    0.000    0.000    0.000    0.000 {method 'result' of '_asyncio.Task' objects}
        5    0.000    0.000    0.000    0.000 {method 'set_result' of '_asyncio.Future' objects}

脚本demo2_2.py运行结果如下:

2017-08-09 11:32:43.821214
2017-08-09 11:32:45.822848
2017-08-09 11:32:47.823607
         22 function calls in 5.002 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    5.002    5.002 <string>:1(<module>)
        1    0.000    0.000    5.002    5.002 demo2_2.py:10(display_date)
        3    0.000    0.000    3.002    1.001 demo2_2.py:6(print_time)
        1    0.000    0.000    5.002    5.002 {built-in method builtins.exec}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.print}
        3    0.000    0.000    0.000    0.000 {built-in method now}
        5    5.002    1.000    5.002    1.000 {built-in method time.sleep}
        4    0.000    0.000    0.000    0.000 {built-in method time.time}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}


Used:  5.006526947021484

总结: 在实际应用中,当我们需要使用协程方式来提过效率时,注意,需要使用异步非阻塞的形式来实现,否则就像time.sleep(),即使我们将它放在了协程实现中,也是达不到效率提高的,退化成了同步实现
举例验证:

import time
import datetime
import tornado.ioloop
import tornado.web
import tornado.gen

class MainHandler(tornado.web.RequestHandler):

    @tornado.gen.coroutine
    def get(self):
        yield tornado.gen.Task(self.blocksleep)
        self.write("Hello, world")

    @tornado.gen.coroutine
    def blocksleep(self):
        time.sleep(20)
        return


def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()

那么,当我们确实需要异步调用阻塞形式的代码时,有什么办法呢,可以使用线程, 还是上面的例子

import time
import datetime
import tornado.ioloop
import tornado.web
import tornado.gen
import tornado.concurrent
from concurrent.futures import ThreadPoolExecutor

class MainHandler(tornado.web.RequestHandler):
    executor = ThreadPoolExecutor(10)
    @tornado.gen.coroutine
    def get(self):
        future = ThreadPoolExecutor().submit(self.blocksleep)
        yield tornado.gen.with_timeout(datetime.timedelta(10), future,
                                       quiet_exceptions=tornado.gen.TimeoutError)
        self.write("Hello, world")

    @tornado.concurrent.run_on_executor
    def blocksleep(self):
        time.sleep(20)
        return


def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])

if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()

相关文章

  • Python协程遐想

    自己在学习协程的时候,对一些东西拿捏的不准,甚至是不解,搞出这样的两个脚本来,让自己更糊涂了,请教了很多人,估计是...

  • python异步协程(aiohttp,asyncio)

    python异步协程 环境:python3.7.0 协程 协程,英文叫做 Coroutine,又称微线程,纤程,协...

  • asyncio + asyncio 异步编程实例

    协程用法 接下来,我们来了解下协程的实现,从 Python 3.4 开始,Python 中加入了协程的概念,但这个...

  • Python 协程

    仅供学习,转载请注明出处 协程 协程,又称微线程,纤程。英文名Coroutine。 协程是啥 协程是python个...

  • 协程

    1.协程 协程,又称微线程,纤程。英文名Coroutine。 1.1 协程是什么 协程是python个中另外一种实...

  • Python并发编程——协程

    摘要:Python,协程,gevent 协程基本概念 协程,又称微线程,纤程。英文名Coroutine,是Pyth...

  • 协程介绍

    协程 协程,又称微线程,纤程。英文名Coroutine。 1.协程是什么? 协程是python个中另外一种实现多任...

  • 4-7

    协程 协程,又称微线程,纤程。英文名Coroutine。 协程是啥 协程是python个中另外一种实现多任务的方式...

  • 协程

    协程 协程,又称微线程,纤程。英文名Coroutine。 协程是啥 协程是python个中另外一种实现多任务的方式...

  • Python 协程 异步资料记录

    1. Python3.5协程原理 详细说明了Python协程产生的历史. 研究明白之后,写一篇文章,解释协程,并附...

网友评论

      本文标题:Python协程遐想

      本文链接:https://www.haomeiwen.com/subject/skvfrxtx.html