A place to cache linked articles (think custom and personal wayback machine)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

hace 4 años
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440
  1. title: Making 1 million requests with python-aiohttp
  2. url: http://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html
  3. hash_url: 44abc2fd416f673b9c6ae0f5147726a9
  4. <p>In this post I’d like to test limits of <a href="http://aiohttp.readthedocs.org/en/stable/">python aiohttp</a> and check its performance in
  5. terms of requests per minute. Everyone knows that asynchronous code performs
  6. better when applied to network operations, but it’s still interesting to check this
  7. assumption and understand how exactly it is better and why it’s is better. I’m going
  8. to check it by trying to make 1 million requests with aiohttp client. How many requests per minute will aiohttp make?
  9. What kind of exceptions and crashes can you expect when you try to make such volume
  10. of requests with very primitive scripts? What are main gotchas that you need
  11. to think about when trying to make such volume of requests?</p>
  12. <h2 id="hello-asyncioaiohttp">Hello asyncio/aiohttp</h2>
  13. <p>Async programming is not easy. It’s not easy because using callbacks and thinking in terms of events
  14. and event handlers requires more effort than usual synchronous programming. But
  15. it is also difficult because asyncio is still relatively new and there are few
  16. blog posts, tutorials about it. <a href="https://docs.python.org/3/library/asyncio.html">Official docs</a>
  17. are very terse and contain only basic examples. There are some Stack Overflow questions
  18. but not <a href="http://stackoverflow.com/questions/tagged/python-asyncio?sort=votes&amp;pageSize=50">that many</a>
  19. only 410 as of time of writing (compare with <a href="http://stackoverflow.com/questions/tagged/twisted">2 585 questions tagged “twisted”</a>)
  20. There are couple of nice blog posts and articles about asyncio
  21. over there such as <a href="http://aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html">this</a>,
  22. <a href="http://www.snarky.ca/how-the-heck-does-async-await-work-in-python-3-5">that</a>, <a href="http://sahandsaba.com/understanding-asyncio-node-js-python-3-4.html">that</a> or perhaps even <a href="https://community.nitrous.io/tutorials/asynchronous-programming-with-python-3">this</a>
  23. or <a href="https://compiletoi.net/fast-scraping-in-python-with-asyncio/">this</a>.</p>
  24. <p>To make it easier let’s start with the basics - simple HTTP hello world -
  25. just making GET and fetching one single HTTP response.</p>
  26. <p>In synchronous world you just do:</p>
  27. <pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">requests</span>
  28. <span class="k">def</span> <span class="nf">hello</span><span class="p">()</span>
  29. <span class="k">return</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"http://httpbin.org/get"</span><span class="p">)</span>
  30. <span class="k">print</span><span class="p">(</span><span class="n">hello</span><span class="p">())</span></code></pre>
  31. <p>How does that look in aiohttp?</p>
  32. <pre><code class="language-python" data-lang="python"><span class="c">#!/usr/local/bin/python3.5</span>
  33. <span class="kn">import</span> <span class="nn">asyncio</span>
  34. <span class="kn">from</span> <span class="nn">aiohttp</span> <span class="kn">import</span> <span class="n">ClientSession</span>
  35. <span class="n">async</span> <span class="k">def</span> <span class="nf">hello</span><span class="p">():</span>
  36. <span class="n">async</span> <span class="k">with</span> <span class="n">ClientSession</span><span class="p">()</span> <span class="k">as</span> <span class="n">session</span><span class="p">:</span>
  37. <span class="n">async</span> <span class="k">with</span> <span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"http://httpbin.org/headers"</span><span class="p">)</span> <span class="k">as</span> <span class="n">response</span><span class="p">:</span>
  38. <span class="n">response</span> <span class="o">=</span> <span class="n">await</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
  39. <span class="k">print</span><span class="p">(</span><span class="n">response</span><span class="p">)</span>
  40. <span class="n">loop</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
  41. <span class="n">loop</span><span class="o">.</span><span class="n">run_until_complete</span><span class="p">(</span><span class="n">hello</span><span class="p">())</span></code></pre>
  42. <p>hmm looks like I had to write lots of code for such a basic task… There is “async def” and “async with” and two “awaits” here. It
  43. seems really confusing at first sight, let’s try to explain it then.</p>
  44. <p>You make your function asynchronous by using <a href="https://www.python.org/dev/peps/pep-0492/#await-expression">async keyword</a> before function definition and using await
  45. keyword. There are actually two asynchronous operations that our hello() function performs. First
  46. it fetches response asynchronously, then it reads response body in asynchronous manner.</p>
  47. <p>Aiohttp recommends to use ClientSession as primary interface to make requests. ClientSession
  48. allows you to store cookies between requests and keeps objects that are common for
  49. all requests (event loop, connection and other things). Session needs to be closed after using it,
  50. and closing session is another asynchronous operation, this is why you need <a href="https://www.python.org/dev/peps/pep-0492/#asynchronous-context-managers-and-async-with"><code class="highlighter-rouge">async with</code></a>
  51. every time you deal with sessions.</p>
  52. <p>After you open client session you can use it to make requests. This is where another asynchronous
  53. operation starts, downloading request. Just as in case of client sessions responses must be closed
  54. explicitly, and context manager’s <code class="highlighter-rouge">with</code> statement ensures it will be closed properly in all
  55. circumstances.</p>
  56. <p>To start your program you need to run it in event loop, so you need to create instance of asyncio
  57. loop and put task into this loop.</p>
  58. <p>It all does sound bit difficult but it’s not that complex and looks logical if you spend
  59. some time trying to understand it.</p>
  60. <h2 id="fetch-multiple-urls">Fetch multiple urls</h2>
  61. <p>Now let’s try to do something more interesting, fetching multiple urls one after another.
  62. With synchronous code you would do just:</p>
  63. <pre><code class="language-python" data-lang="python"><span class="k">for</span> <span class="n">url</span> <span class="ow">in</span> <span class="n">urls</span><span class="p">:</span>
  64. <span class="k">print</span><span class="p">(</span><span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">text</span><span class="p">)</span></code></pre>
  65. <p>This is really quick and easy, async will not be that easy, so you should always consider if something more complex
  66. is actually necessary for your needs. If your app works nice with synchronous code maybe there
  67. is no need to bother with async code? If you do need to bother with async code here’s how you do
  68. that. Our <code class="highlighter-rouge">hello()</code> async function stays the same but we need to wrap it in asyncio <a href="https://docs.python.org/3/library/asyncio-task.html#future"><code class="highlighter-rouge">Future</code></a> object
  69. and pass whole lists of Future objects as tasks to be executed in the loop.</p>
  70. <pre><code class="language-python" data-lang="python"><span class="n">loop</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
  71. <span class="n">tasks</span> <span class="o">=</span> <span class="p">[]</span>
  72. <span class="c"># I'm using test server localhost, but you can use any url</span>
  73. <span class="n">url</span> <span class="o">=</span> <span class="s">"http://localhost:8080/{}"</span>
  74. <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">5</span><span class="p">):</span>
  75. <span class="n">task</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">hello</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)))</span>
  76. <span class="n">tasks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
  77. <span class="n">loop</span><span class="o">.</span><span class="n">run_until_complete</span><span class="p">(</span><span class="n">asyncio</span><span class="o">.</span><span class="n">wait</span><span class="p">(</span><span class="n">tasks</span><span class="p">))</span></code></pre>
  78. <p>Now let’s say we want to collect all responses in one list and do some
  79. postprocessing on them. At the moment we’re not keeping response body
  80. anywhere, we just print it, let’s return this response, keep it in list, and
  81. print all responses at the end.</p>
  82. <p>To collect bunch of responses you probably need to write something along the lines of:</p>
  83. <pre><code class="language-python" data-lang="python"><span class="c">#!/usr/local/bin/python3.5</span>
  84. <span class="kn">import</span> <span class="nn">asyncio</span>
  85. <span class="kn">from</span> <span class="nn">aiohttp</span> <span class="kn">import</span> <span class="n">ClientSession</span>
  86. <span class="n">async</span> <span class="k">def</span> <span class="nf">fetch</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
  87. <span class="n">async</span> <span class="k">with</span> <span class="n">ClientSession</span><span class="p">()</span> <span class="k">as</span> <span class="n">session</span><span class="p">:</span>
  88. <span class="n">async</span> <span class="k">with</span> <span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">as</span> <span class="n">response</span><span class="p">:</span>
  89. <span class="k">return</span> <span class="n">await</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
  90. <span class="n">async</span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="n">loop</span><span class="p">,</span> <span class="n">r</span><span class="p">):</span>
  91. <span class="n">url</span> <span class="o">=</span> <span class="s">"http://localhost:8080/{}"</span>
  92. <span class="n">tasks</span> <span class="o">=</span> <span class="p">[]</span>
  93. <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">r</span><span class="p">):</span>
  94. <span class="n">task</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">fetch</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)))</span>
  95. <span class="n">tasks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
  96. <span class="n">responses</span> <span class="o">=</span> <span class="n">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">)</span>
  97. <span class="c"># you now have all response bodies in this variable</span>
  98. <span class="k">print</span><span class="p">(</span><span class="n">responses</span><span class="p">)</span>
  99. <span class="k">def</span> <span class="nf">print_responses</span><span class="p">(</span><span class="n">result</span><span class="p">):</span>
  100. <span class="k">print</span><span class="p">(</span><span class="n">result</span><span class="p">)</span>
  101. <span class="n">loop</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
  102. <span class="n">future</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">run</span><span class="p">(</span><span class="n">loop</span><span class="p">,</span> <span class="mi">4</span><span class="p">))</span>
  103. <span class="n">loop</span><span class="o">.</span><span class="n">run_until_complete</span><span class="p">(</span><span class="n">future</span><span class="p">)</span></code></pre>
  104. <p>Notice usage of <a href="https://docs.python.org/3/library/asyncio-task.html#asyncio.gather"><code class="highlighter-rouge">asyncio.gather()</code></a>, this collects bunch of Future objects in one place
  105. and waits for all of them to finish.</p>
  106. <h3 id="common-gotchas">Common gotchas</h3>
  107. <p>Now let’s simulate real process of learning and let’s make mistake in above script and try to debug it,
  108. this should be really helpful for demonstration purposes.</p>
  109. <p>This is how sample broken async function looks like:</p>
  110. <pre><code class="language-python" data-lang="python"><span class="c"># WARNING! BROKEN CODE DO NOT COPY PASTE</span>
  111. <span class="n">async</span> <span class="k">def</span> <span class="nf">fetch</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
  112. <span class="n">async</span> <span class="k">with</span> <span class="n">ClientSession</span><span class="p">()</span> <span class="k">as</span> <span class="n">session</span><span class="p">:</span>
  113. <span class="n">async</span> <span class="k">with</span> <span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">as</span> <span class="n">response</span><span class="p">:</span>
  114. <span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span></code></pre>
  115. <p>This code is broken, but it’s not that easy to figure out why
  116. if you dont know much about asyncio. Even if you know Python well but you dont
  117. know asyncio or aiohttp well you’ll be in trouble to figure out what happens.</p>
  118. <p>What is output of above function?</p>
  119. <p>It produces following output:</p>
  120. <pre><code class="language-python" data-lang="python"><span class="n">pawel</span><span class="nd">@pawel</span><span class="o">-</span><span class="n">VPCEH390X</span> <span class="o">~/</span><span class="n">p</span><span class="o">/</span><span class="n">l</span><span class="o">/</span><span class="n">benchmarker</span><span class="o">&gt;</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span>
  121. <span class="p">[</span><span class="o">&lt;</span><span class="n">generator</span> <span class="nb">object</span> <span class="n">ClientResponse</span><span class="o">.</span><span class="n">read</span> <span class="n">at</span> <span class="mh">0x7fa68d465728</span><span class="o">&gt;</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">generator</span> <span class="nb">object</span> <span class="n">ClientResponse</span><span class="o">.</span><span class="n">read</span> <span class="n">at</span> <span class="mh">0x7fa68cdd9468</span><span class="o">&gt;</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">generator</span> <span class="nb">object</span> <span class="n">ClientResponse</span><span class="o">.</span><span class="n">read</span> <span class="n">at</span> <span class="mh">0x7fa68d4656d0</span><span class="o">&gt;</span><span class="p">,</span> <span class="o">&lt;</span><span class="n">generator</span> <span class="nb">object</span> <span class="n">ClientResponse</span><span class="o">.</span><span class="n">read</span> <span class="n">at</span> <span class="mh">0x7fa68cdd9af0</span><span class="o">&gt;</span><span class="p">]</span></code></pre>
  122. <p>What happens here? You expected to get response objects after all processing is done, but here you actually get
  123. bunch of generators, why is that?</p>
  124. <p>It happens because as I’ve mentioned earlier <code class="highlighter-rouge">response.read()</code> is async
  125. operation, this means that it does not return result immediately, it just returns generator.
  126. This generator still needs to be called and
  127. executed, and this does not happen by default, <code class="highlighter-rouge">yield from</code> in Python 3.4 and <code class="highlighter-rouge">await</code> in Python 3.5 were
  128. added exactly for this purpose: to actually iterate over generator function. Fix to above error
  129. is just adding await before <code class="highlighter-rouge">response.read()</code>.</p>
  130. <pre><code class="language-python" data-lang="python"> <span class="c"># async operation must be preceded by await </span>
  131. <span class="k">return</span> <span class="n">await</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span> <span class="c"># NOT: return response.read()</span></code></pre>
  132. <p>Let’s break our code in some other way.</p>
  133. <pre><code class="language-python" data-lang="python"><span class="c"># WARNING! BROKEN CODE DO NOT COPY PASTE</span>
  134. <span class="n">async</span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="n">loop</span><span class="p">,</span> <span class="n">r</span><span class="p">):</span>
  135. <span class="n">url</span> <span class="o">=</span> <span class="s">"http://localhost:8080/{}"</span>
  136. <span class="n">tasks</span> <span class="o">=</span> <span class="p">[]</span>
  137. <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">r</span><span class="p">):</span>
  138. <span class="n">task</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">fetch</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)))</span>
  139. <span class="n">tasks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
  140. <span class="n">responses</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">)</span>
  141. <span class="k">print</span><span class="p">(</span><span class="n">responses</span><span class="p">)</span></code></pre>
  142. <p>Again above code is broken but it’s not easy to figure out why if you’re just
  143. learning asyncio.</p>
  144. <p>Above produces following output:</p>
  145. <pre><code class="language-python" data-lang="python"><span class="n">pawel</span><span class="nd">@pawel</span><span class="o">-</span><span class="n">VPCEH390X</span> <span class="o">~/</span><span class="n">p</span><span class="o">/</span><span class="n">l</span><span class="o">/</span><span class="n">benchmarker</span><span class="o">&gt;</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span>
  146. <span class="o">&lt;</span><span class="n">_GatheringFuture</span> <span class="n">pending</span><span class="o">&gt;</span>
  147. <span class="n">Task</span> <span class="n">was</span> <span class="n">destroyed</span> <span class="n">but</span> <span class="n">it</span> <span class="ow">is</span> <span class="n">pending</span><span class="err">!</span>
  148. <span class="n">task</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Task</span> <span class="n">pending</span> <span class="n">coro</span><span class="o">=&lt;</span><span class="n">fetch</span><span class="p">()</span> <span class="n">running</span> <span class="n">at</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">7</span><span class="o">&gt;</span> <span class="n">wait_for</span><span class="o">=&lt;</span><span class="n">Future</span> <span class="n">pending</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">Task</span><span class="o">.</span><span class="n">_wakeup</span><span class="p">()]</span><span class="o">&gt;</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">gather</span><span class="o">.&lt;</span><span class="nb">locals</span><span class="o">&gt;.</span><span class="n">_done_callback</span><span class="p">(</span><span class="mi">0</span><span class="p">)()</span> <span class="n">at</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span><span class="o">/</span><span class="n">asyncio</span><span class="o">/</span><span class="n">tasks</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">602</span><span class="p">]</span><span class="o">&gt;</span>
  149. <span class="n">Task</span> <span class="n">was</span> <span class="n">destroyed</span> <span class="n">but</span> <span class="n">it</span> <span class="ow">is</span> <span class="n">pending</span><span class="err">!</span>
  150. <span class="n">task</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Task</span> <span class="n">pending</span> <span class="n">coro</span><span class="o">=&lt;</span><span class="n">fetch</span><span class="p">()</span> <span class="n">running</span> <span class="n">at</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">7</span><span class="o">&gt;</span> <span class="n">wait_for</span><span class="o">=&lt;</span><span class="n">Future</span> <span class="n">pending</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">Task</span><span class="o">.</span><span class="n">_wakeup</span><span class="p">()]</span><span class="o">&gt;</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">gather</span><span class="o">.&lt;</span><span class="nb">locals</span><span class="o">&gt;.</span><span class="n">_done_callback</span><span class="p">(</span><span class="mi">1</span><span class="p">)()</span> <span class="n">at</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span><span class="o">/</span><span class="n">asyncio</span><span class="o">/</span><span class="n">tasks</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">602</span><span class="p">]</span><span class="o">&gt;</span>
  151. <span class="n">Task</span> <span class="n">was</span> <span class="n">destroyed</span> <span class="n">but</span> <span class="n">it</span> <span class="ow">is</span> <span class="n">pending</span><span class="err">!</span>
  152. <span class="n">task</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Task</span> <span class="n">pending</span> <span class="n">coro</span><span class="o">=&lt;</span><span class="n">fetch</span><span class="p">()</span> <span class="n">running</span> <span class="n">at</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">7</span><span class="o">&gt;</span> <span class="n">wait_for</span><span class="o">=&lt;</span><span class="n">Future</span> <span class="n">pending</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">Task</span><span class="o">.</span><span class="n">_wakeup</span><span class="p">()]</span><span class="o">&gt;</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">gather</span><span class="o">.&lt;</span><span class="nb">locals</span><span class="o">&gt;.</span><span class="n">_done_callback</span><span class="p">(</span><span class="mi">2</span><span class="p">)()</span> <span class="n">at</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span><span class="o">/</span><span class="n">asyncio</span><span class="o">/</span><span class="n">tasks</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">602</span><span class="p">]</span><span class="o">&gt;</span>
  153. <span class="n">Task</span> <span class="n">was</span> <span class="n">destroyed</span> <span class="n">but</span> <span class="n">it</span> <span class="ow">is</span> <span class="n">pending</span><span class="err">!</span>
  154. <span class="n">task</span><span class="p">:</span> <span class="o">&lt;</span><span class="n">Task</span> <span class="n">pending</span> <span class="n">coro</span><span class="o">=&lt;</span><span class="n">fetch</span><span class="p">()</span> <span class="n">running</span> <span class="n">at</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">7</span><span class="o">&gt;</span> <span class="n">wait_for</span><span class="o">=&lt;</span><span class="n">Future</span> <span class="n">pending</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">Task</span><span class="o">.</span><span class="n">_wakeup</span><span class="p">()]</span><span class="o">&gt;</span> <span class="n">cb</span><span class="o">=</span><span class="p">[</span><span class="n">gather</span><span class="o">.&lt;</span><span class="nb">locals</span><span class="o">&gt;.</span><span class="n">_done_callback</span><span class="p">(</span><span class="mi">3</span><span class="p">)()</span> <span class="n">at</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">lib</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span><span class="o">/</span><span class="n">asyncio</span><span class="o">/</span><span class="n">tasks</span><span class="o">.</span><span class="n">py</span><span class="p">:</span><span class="mi">602</span><span class="p">]</span><span class="o">&gt;</span></code></pre>
  155. <p>What happens here? If you examine your localhost logs you may see that requests are not reaching
  156. your server at all. Clearly no requests are performed. Print statement prints that
  157. responses variable contains <code class="highlighter-rouge">&lt;_GatheringFuture pending&gt;</code> object, and later it alerts that
  158. pending tasks were destroyed. Why is it happening? Again you forgot about await</p>
  159. <p>faulty line is this</p>
  160. <pre><code class="language-python" data-lang="python"> <span class="n">responses</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">)</span></code></pre>
  161. <p>it should be:</p>
  162. <pre><code class="language-python" data-lang="python"> <span class="n">responses</span> <span class="o">=</span> <span class="n">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">)</span></code></pre>
  163. <p>I guess main lesson from those mistakes is: always remember about using “await” if
  164. you’re actually awaiting something.</p>
  165. <h2 id="sync-vs-async">Sync vs Async</h2>
  166. <p>Finally time for some fun. Let’s check if async is really worth the hassle. What’s the
  167. difference in efficiency between asynchronous client and blocking client? How many
  168. requests per minute can I send with my async client?</p>
  169. <p>With this questions in mind I set up simple (async) aiohttp server.
  170. My server is going to read full html text of Frankenstein by Marry Shelley. It will
  171. add random delays between responses. Some responses will have zero delay, and some will
  172. have maximum of 3 seconds delay. This should resemble real applications, few
  173. apps respond to all requests with same latency, usually latency differs
  174. from response to response.</p>
  175. <p>Server code looks like this:</p>
  176. <pre><code class="language-python" data-lang="python"><span class="c">#!/usr/local/bin/python3.5</span>
  177. <span class="kn">import</span> <span class="nn">asyncio</span>
  178. <span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
  179. <span class="kn">from</span> <span class="nn">aiohttp</span> <span class="kn">import</span> <span class="n">web</span>
  180. <span class="kn">import</span> <span class="nn">random</span>
  181. <span class="c"># set seed to ensure async and sync client get same distribution of delay values</span>
  182. <span class="c"># and tests are fair</span>
  183. <span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
  184. <span class="n">async</span> <span class="k">def</span> <span class="nf">hello</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
  185. <span class="n">name</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">match_info</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"name"</span><span class="p">,</span> <span class="s">"foo"</span><span class="p">)</span>
  186. <span class="n">n</span> <span class="o">=</span> <span class="n">datetime</span><span class="o">.</span><span class="n">now</span><span class="p">()</span><span class="o">.</span><span class="n">isoformat</span><span class="p">()</span>
  187. <span class="n">delay</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">randint</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
  188. <span class="n">await</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="n">delay</span><span class="p">)</span>
  189. <span class="n">headers</span> <span class="o">=</span> <span class="p">{</span><span class="s">"content_type"</span><span class="p">:</span> <span class="s">"text/html"</span><span class="p">,</span> <span class="s">"delay"</span><span class="p">:</span> <span class="nb">str</span><span class="p">(</span><span class="n">delay</span><span class="p">)}</span>
  190. <span class="c"># opening file is not async here, so it may block, to improve</span>
  191. <span class="c"># efficiency of this you can consider using asyncio Executors</span>
  192. <span class="c"># that will delegate file operation to separate thread or process</span>
  193. <span class="c"># and improve performance</span>
  194. <span class="c"># https://docs.python.org/3/library/asyncio-eventloop.html#executor</span>
  195. <span class="c"># https://pymotw.com/3/asyncio/executors.html</span>
  196. <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">"frank.html"</span><span class="p">,</span> <span class="s">"rb"</span><span class="p">)</span> <span class="k">as</span> <span class="n">html_body</span><span class="p">:</span>
  197. <span class="k">print</span><span class="p">(</span><span class="s">"{}: {} delay: {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">request</span><span class="o">.</span><span class="n">path</span><span class="p">,</span> <span class="n">delay</span><span class="p">))</span>
  198. <span class="n">response</span> <span class="o">=</span> <span class="n">web</span><span class="o">.</span><span class="n">Response</span><span class="p">(</span><span class="n">body</span><span class="o">=</span><span class="n">html_body</span><span class="o">.</span><span class="n">read</span><span class="p">(),</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">)</span>
  199. <span class="k">return</span> <span class="n">response</span>
  200. <span class="n">app</span> <span class="o">=</span> <span class="n">web</span><span class="o">.</span><span class="n">Application</span><span class="p">()</span>
  201. <span class="n">app</span><span class="o">.</span><span class="n">router</span><span class="o">.</span><span class="n">add_route</span><span class="p">(</span><span class="s">"GET"</span><span class="p">,</span> <span class="s">"/{name}"</span><span class="p">,</span> <span class="n">hello</span><span class="p">)</span>
  202. <span class="n">web</span><span class="o">.</span><span class="n">run_app</span><span class="p">(</span><span class="n">app</span><span class="p">)</span></code></pre>
  203. <p>Synchronous client looks like this:</p>
  204. <pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">requests</span>
  205. <span class="n">r</span> <span class="o">=</span> <span class="mi">100</span>
  206. <span class="n">url</span> <span class="o">=</span> <span class="s">"http://localhost:8080/{}"</span>
  207. <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">r</span><span class="p">):</span>
  208. <span class="n">res</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">))</span>
  209. <span class="n">delay</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"DELAY"</span><span class="p">)</span>
  210. <span class="n">d</span> <span class="o">=</span> <span class="n">res</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"DATE"</span><span class="p">)</span>
  211. <span class="k">print</span><span class="p">(</span><span class="s">"{}:{} delay {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="n">res</span><span class="o">.</span><span class="n">url</span><span class="p">,</span> <span class="n">delay</span><span class="p">))</span></code></pre>
  212. <p>How long will it take to run this?</p>
  213. <p>On my machine running above synchronous client took 2:45.54 minutes.</p>
  214. <p>My async code looks just like above code samples above. How long will async client take?</p>
  215. <p>On my machine it took 0:03.48 seconds.</p>
  216. <p>It is interesting that it took exactly as long as longest delay
  217. from my server. If you look into messages printed by client script you can see how
  218. great async HTTP client is. Some responses had 0 delay but others got 3 seconds delay. In synchronous client
  219. they would be blocking and waiting, your machine would simply stay idle for this time.
  220. Async client does not waste time, when something is delayed it simply does
  221. something else, issues other requests or processes all other responses. You can see this clearly in logs, first there
  222. are responses with 0 delay, then after they arrrived you can see responses with 1 seconds delay,
  223. and so on until most delayed responses arrive.</p>
  224. <h2 id="testing-the-limits">Testing the limits</h2>
  225. <p>Now that we know our async client is better let’s try to test its limits and try to crash our
  226. localhost. I’m going to start with sending 1k async requests. I’m curious how many requests
  227. my client can handle.</p>
  228. <pre><code class="language-bash" data-lang="bash"><span class="gp">&gt; </span><span class="nb">time </span>python3 bench.py
  229. 2.68user 0.24system 0:07.14elapsed 40%CPU <span class="o">(</span>0avgtext+0avgdata 53704maxresident<span class="o">)</span>k
  230. 0inputs+0outputs <span class="o">(</span>0major+14156minor<span class="o">)</span>pagefaults 0swaps</code></pre>
  231. <p>So 1k requests take 7 seconds, pretty nice! How about 10k? Trying to make 10k requests
  232. unfortunately fails…</p>
  233. <pre><code class="language-python" data-lang="python"><span class="n">responses</span> <span class="n">are</span> <span class="o">&lt;</span><span class="n">_GatheringFuture</span> <span class="n">finished</span> <span class="n">exception</span><span class="o">=</span><span class="n">ClientOSError</span><span class="p">(</span><span class="mi">24</span><span class="p">,</span> <span class="s">'Cannot connect to host localhost:8080 ssl:False [Can not connect to localhost:8080 [Too many open files]]'</span><span class="p">)</span><span class="o">&gt;</span>
  234. <span class="n">Traceback</span> <span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">):</span>
  235. <span class="n">File</span> <span class="s">"/home/pawel/.local/lib/python3.5/site-packages/aiohttp/connector.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">581</span><span class="p">,</span> <span class="ow">in</span> <span class="n">_create_connection</span>
  236. <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/asyncio/base_events.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">651</span><span class="p">,</span> <span class="ow">in</span> <span class="n">create_connection</span>
  237. <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/asyncio/base_events.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">618</span><span class="p">,</span> <span class="ow">in</span> <span class="n">create_connection</span>
  238. <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/socket.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">134</span><span class="p">,</span> <span class="ow">in</span> <span class="n">__init__</span>
  239. <span class="nb">OSError</span><span class="p">:</span> <span class="p">[</span><span class="n">Errno</span> <span class="mi">24</span><span class="p">]</span> <span class="n">Too</span> <span class="n">many</span> <span class="nb">open</span> <span class="n">files</span></code></pre>
  240. <p>That’s bad, seems like I stumbled across <a href="http://www.webcitation.org/6ICibHuyd">10k connections problem</a>.</p>
  241. <p>It says “too many open files”, and probably refers to number of open sockets.
  242. Why does it call them files? Sockets are just file descriptors, operating systems limit number of open sockets
  243. allowed. How many files are too many? I checked with python resource module and it seems like it’s around 1024.
  244. How can we bypass this? Primitive way is just increasing limit of open files. But this
  245. is probably not the good way to go. Much better way is just adding some synchronization
  246. in your client limiting number of concurrent requests it can process. I’m going to do this
  247. by adding <a href="https://docs.python.org/3/library/asyncio-sync.html#asyncio.Semaphore"><code class="highlighter-rouge">asyncio.Semaphore()</code></a> with max tasks of 1000.</p>
  248. <p>Modified client code looks like this now:</p>
  249. <pre><code class="language-python" data-lang="python"><span class="c"># modified fetch function with semaphore</span>
  250. <span class="kn">import</span> <span class="nn">random</span>
  251. <span class="kn">import</span> <span class="nn">asyncio</span>
  252. <span class="kn">from</span> <span class="nn">aiohttp</span> <span class="kn">import</span> <span class="n">ClientSession</span>
  253. <span class="n">async</span> <span class="k">def</span> <span class="nf">fetch</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
  254. <span class="n">async</span> <span class="k">with</span> <span class="n">ClientSession</span><span class="p">()</span> <span class="k">as</span> <span class="n">session</span><span class="p">:</span>
  255. <span class="n">async</span> <span class="k">with</span> <span class="n">session</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span> <span class="k">as</span> <span class="n">response</span><span class="p">:</span>
  256. <span class="n">delay</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"DELAY"</span><span class="p">)</span>
  257. <span class="n">date</span> <span class="o">=</span> <span class="n">response</span><span class="o">.</span><span class="n">headers</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"DATE"</span><span class="p">)</span>
  258. <span class="k">print</span><span class="p">(</span><span class="s">"{}:{} with delay {}"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">date</span><span class="p">,</span> <span class="n">response</span><span class="o">.</span><span class="n">url</span><span class="p">,</span> <span class="n">delay</span><span class="p">))</span>
  259. <span class="k">return</span> <span class="n">await</span> <span class="n">response</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
  260. <span class="n">async</span> <span class="k">def</span> <span class="nf">bound_fetch</span><span class="p">(</span><span class="n">sem</span><span class="p">,</span> <span class="n">url</span><span class="p">):</span>
  261. <span class="c"># getter function with semaphore</span>
  262. <span class="n">async</span> <span class="k">with</span> <span class="n">sem</span><span class="p">:</span>
  263. <span class="n">await</span> <span class="n">fetch</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
  264. <span class="n">async</span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="n">loop</span><span class="p">,</span> <span class="n">r</span><span class="p">):</span>
  265. <span class="n">url</span> <span class="o">=</span> <span class="s">"http://localhost:8080/{}"</span>
  266. <span class="n">tasks</span> <span class="o">=</span> <span class="p">[]</span>
  267. <span class="c"># create instance of Semaphore</span>
  268. <span class="n">sem</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">Semaphore</span><span class="p">(</span><span class="mi">1000</span><span class="p">)</span>
  269. <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">r</span><span class="p">):</span>
  270. <span class="c"># pass Semaphore to every GET request</span>
  271. <span class="n">task</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">bound_fetch</span><span class="p">(</span><span class="n">sem</span><span class="p">,</span> <span class="n">url</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">i</span><span class="p">)))</span>
  272. <span class="n">tasks</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">task</span><span class="p">)</span>
  273. <span class="n">responses</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">gather</span><span class="p">(</span><span class="o">*</span><span class="n">tasks</span><span class="p">)</span>
  274. <span class="n">await</span> <span class="n">responses</span>
  275. <span class="n">number</span> <span class="o">=</span> <span class="mi">10000</span>
  276. <span class="n">loop</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">get_event_loop</span><span class="p">()</span>
  277. <span class="n">future</span> <span class="o">=</span> <span class="n">asyncio</span><span class="o">.</span><span class="n">ensure_future</span><span class="p">(</span><span class="n">run</span><span class="p">(</span><span class="n">loop</span><span class="p">,</span> <span class="n">number</span><span class="p">))</span>
  278. <span class="n">loop</span><span class="o">.</span><span class="n">run_until_complete</span><span class="p">(</span><span class="n">future</span><span class="p">)</span></code></pre>
  279. <p>At this point I can process 10k urls. It takes 23 seconds and returns some exceptions but overall
  280. it’s pretty nice!</p>
  281. <p>How about 100 000? This really makes my computer work hard but suprisingly
  282. it works ok. Server turns out to be suprisingly stable although
  283. you can see that ram usage gets pretty high at this point, cpu usage is around
  284. 100% all the time. What I find interesting is that my server takes significantly less cpu than client.
  285. Here’s snapshot of linux <code class="highlighter-rouge">ps</code> output.</p>
  286. <pre><code class="language-python" data-lang="python"><span class="n">pawel</span><span class="nd">@pawel</span><span class="o">-</span><span class="n">VPCEH390X</span> <span class="o">~/</span><span class="n">p</span><span class="o">/</span><span class="n">l</span><span class="o">/</span><span class="n">benchmarker</span><span class="o">&gt;</span> <span class="n">ps</span> <span class="n">ua</span> <span class="o">|</span> <span class="n">grep</span> <span class="n">python</span>
  287. <span class="n">USER</span> <span class="n">PID</span> <span class="o">%</span><span class="n">CPU</span> <span class="o">%</span><span class="n">MEM</span> <span class="n">VSZ</span> <span class="n">RSS</span> <span class="n">TTY</span> <span class="n">STAT</span> <span class="n">START</span> <span class="n">TIME</span> <span class="n">COMMAND</span>
  288. <span class="n">pawel</span> <span class="mi">2447</span> <span class="mf">56.3</span> <span class="mf">1.0</span> <span class="mi">216124</span> <span class="mi">64976</span> <span class="n">pts</span><span class="o">/</span><span class="mi">9</span> <span class="n">Sl</span><span class="o">+</span> <span class="mi">21</span><span class="p">:</span><span class="mi">26</span> <span class="mi">1</span><span class="p">:</span><span class="mi">27</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="nb">bin</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span> <span class="o">./</span><span class="n">test_server</span><span class="o">.</span><span class="n">py</span>
  289. <span class="n">pawel</span> <span class="mi">2527</span> <span class="mi">101</span> <span class="mf">3.5</span> <span class="mi">674732</span> <span class="mi">212076</span> <span class="n">pts</span><span class="o">/</span><span class="mi">0</span> <span class="n">Rl</span><span class="o">+</span> <span class="mi">21</span><span class="p">:</span><span class="mi">26</span> <span class="mi">2</span><span class="p">:</span><span class="mi">30</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="nb">bin</span><span class="o">/</span><span class="n">python3</span><span class="o">.</span><span class="mi">5</span> <span class="o">./</span><span class="n">bench</span><span class="o">.</span><span class="n">py</span></code></pre>
  290. <p>Overall it took around 5 minutes before it crashed for some
  291. reason. It generated around 100k lines of output so it’s not that easy
  292. to pinpoint traceback, seems like some responses are not closed, is it
  293. because of some error from my server or something in client?</p>
  294. <p>After scrolling for couple of seconds I found this exception in client logs.</p>
  295. <pre><code class="language-python" data-lang="python"> <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/asyncio/futures.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">387</span><span class="p">,</span> <span class="ow">in</span> <span class="n">__iter__</span>
  296. <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">result</span><span class="p">()</span> <span class="c"># May raise too.</span>
  297. <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/asyncio/futures.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">274</span><span class="p">,</span> <span class="ow">in</span> <span class="n">result</span>
  298. <span class="k">raise</span> <span class="bp">self</span><span class="o">.</span><span class="n">_exception</span>
  299. <span class="n">File</span> <span class="s">"/usr/local/lib/python3.5/asyncio/selector_events.py"</span><span class="p">,</span> <span class="n">line</span> <span class="mi">411</span><span class="p">,</span> <span class="ow">in</span> <span class="n">_sock_connect</span>
  300. <span class="n">sock</span><span class="o">.</span><span class="n">connect</span><span class="p">(</span><span class="n">address</span><span class="p">)</span>
  301. <span class="nb">OSError</span><span class="p">:</span> <span class="p">[</span><span class="n">Errno</span> <span class="mi">99</span><span class="p">]</span> <span class="n">Cannot</span> <span class="n">assign</span> <span class="n">requested</span> <span class="n">address</span></code></pre>
  302. <p>I dont really know what happens here. My initial hypothesis is that test server went down for some split second,
  303. and this caused some client error that was printed at the end. <a href="https://news.ycombinator.com/item?id=11557672">One of the readers</a>
  304. suggests that this exception may be caused by OS running out of free ephemereal ports. I added semaphore
  305. earlier so number of concurrent connections should be maximum 1k, but some sockets may still be
  306. closing and not available for kernel to assign.</p>
  307. <p>Overall it’s really not bad, 5 minutes for 100 000 requests? This makes around 20k
  308. requests per minute. Pretty powerful if you ask me.</p>
  309. <p>Finally I’m going to try 1 million requests. I really hope my laptop is not going
  310. to explode when testing that. For this amount of requests I reduced delays from server to range between 0 and 1.</p>
  311. <p>1 000 000 requests finished in 52 minutes</p>
  312. <pre><code class="language-bash" data-lang="bash">1913.06user 1196.09system 52:06.87elapsed 99%CPU <span class="o">(</span>0avgtext+0avgdata 5194260maxresident<span class="o">)</span>k
  313. 265144inputs+0outputs <span class="o">(</span>18692major+2528207minor<span class="o">)</span>pagefaults 0swaps</code></pre>
  314. <p>so it means my client made around 19230 requests per minute. Not bad isn’t it? Note that
  315. capabilities of my client are limited by server responding with delay of 0 and 1 in this
  316. scenario, seems like my test server also crashed silently couple of times.</p>
  317. <h2 id="epilogue">Epilogue</h2>
  318. <p>You can see that asynchronous HTTP clients can be pretty powerful. Performing
  319. 1 million requests from async client is not difficult, and the client performs really well in comparison
  320. to synchronous code.</p>
  321. <p>I wonder how it compares to other languages and async frameworks? Perhaps in some
  322. future post I could compare <a href="https://github.com/twisted/treq">Twisted Treq</a> with
  323. aiohttp. There is also question how many concurrent requests can be issued by
  324. async libraries in other languages. E.g. what would be results of benchmarks
  325. for some Java async frameworks? Or C++ frameworks? Or some Rust HTTP clients?</p>
  326. <h3 id="edits-24042016"><em>EDITS (24/04/2016)</em></h3>
  327. <ul>
  328. <li>improved code sample that uses Semaphore</li>
  329. <li>added comment about using executor when opening file</li>
  330. <li>added link to HN comment about EADDRNOTAVAIL exception</li>
  331. </ul>