html/async.html

   1
   2
   3 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   4   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   5
   6
   7 <html xmlns="http://www.w3.org/1999/xhtml">
   8   <head>
   9     <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  10
  11     <title>Primer to Asynchronous Applications &mdash; Bottle 0.12-dev documentation</title>
  12
  13     <link rel="stylesheet" href="_static/bottle.css" type="text/css" />
  14     <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
  15
  16     <script type="text/javascript">
  17       var DOCUMENTATION_OPTIONS = {
  18         URL_ROOT:    '',
  19         VERSION:     '0.12-dev',
  20         COLLAPSE_INDEX: false,
  21         FILE_SUFFIX: '.html',
  22         HAS_SOURCE:  true
  23       };
  24     </script>
  25     <script type="text/javascript" src="_static/jquery.js"></script>
  26     <script type="text/javascript" src="_static/underscore.js"></script>
  27     <script type="text/javascript" src="_static/doctools.js"></script>
  28     <link rel="shortcut icon" href="_static/favicon.ico"/>
  29     <link rel="top" title="Bottle 0.12-dev documentation" href="index.html" />
  30     <link rel="next" title="Recipes" href="recipes.html" />
  31     <link rel="prev" title="Tutorial: Todo-List Application" href="tutorial_app.html" />
  32     <link rel="shortcut icon" type="image/x-icon" href="_static/favicon.ico" />
  33     <link rel="image_src" type="image/png" href="_static/logo_reddit.png" />
  34     <script type="application/javascript" src="_static/default.js"></script>
  35
  36
  37
  38   </head>
  39   <body>
  40     <div class="related">
  41       <h3>Navigation</h3>
  42       <ul>
  43         <li class="right" style="margin-right: 10px">
  44           <a href="genindex.html" title="General Index"
  45              accesskey="I">index</a></li>
  46         <li class="right" >
  47           <a href="py-modindex.html" title="Python Module Index"
  48              >modules</a> |</li>
  49         <li class="right" >
  50           <a href="recipes.html" title="Recipes"
  51              accesskey="N">next</a> |</li>
  52         <li class="right" >
  53           <a href="tutorial_app.html" title="Tutorial: Todo-List Application"
  54              accesskey="P">previous</a> |</li>
  55     <li><a href="/">Project Home</a> &raquo;</li>
  56
  57         <li><a href="index.html">Bottle 0.12-dev documentation</a> &raquo;</li>
  58
  59       </ul>
  60     </div>
  61
  62     <div class="document">
  63       <div class="documentwrapper">
  64         <div class="bodywrapper">
  65           <div class="body">
  66
  67
  68   <p style='font-size: 0.75em; color: darkred'><b>Warning:</b> This is a preview for <b>Bottle-0.12-dev</b>, which is
  69     not released yet. Switch to the latest <a href="/docs/stable/"><b>stable release</b></a>?</p>
  70
  71
  72   <div class="section" id="primer-to-asynchronous-applications">
  73 <h1>Primer to Asynchronous Applications<a class="headerlink" href="#primer-to-asynchronous-applications" title="Permalink to this headline">¶</a></h1>
  74 <p>Asynchronous design patterns don&#8217;t mix well with the synchronous nature of <a class="reference external" href="http://www.python.org/dev/peps/pep-3333/">WSGI</a>. This is why most asynchronous frameworks (tornado, twisted, ...) implement a specialized API to expose their asynchronous features. Bottle is a WSGI framework and shares the synchronous nature of WSGI, but thanks to the awesome <a class="reference external" href="http://www.gevent.org/">gevent project</a>, it is still possible to write asynchronous applications with bottle. This article documents the usage of Bottle with Asynchronous WSGI.</p>
  75 <div class="section" id="the-limits-of-synchronous-wsgi">
  76 <h2>The Limits of Synchronous WSGI<a class="headerlink" href="#the-limits-of-synchronous-wsgi" title="Permalink to this headline">¶</a></h2>
  77 <p>Briefly worded, the <a class="reference external" href="http://www.python.org/dev/peps/pep-3333/">WSGI specification (pep 3333)</a> defines a request/response circle as follows: The application callable is invoked once for each request and must return a body iterator. The server then iterates over the body and writes each chunk to the socket. As soon as the body iterator is exhausted, the client connection is closed.</p>
  78 <p>Simple enough, but there is a snag: All this happens synchronously. If your application needs to wait for data (IO, sockets, databases, ...), it must either yield empty strings (busy wait) or block the current thread. Both solutions occupy the handling thread and prevent it from answering new requests. There is consequently only one ongoing request per thread.</p>
  79 <p>Most servers limit the number of threads to avoid their relatively high overhead. Pools of 20 or less threads are common. As soon as all threads are occupied, any new connection is stalled. The server is effectively dead for everyone else. If you want to implement a chat that uses long-polling ajax requests to get real-time updates, you&#8217;d reach the limited at 20 concurrent connections. That&#8217;s a pretty small chat.</p>
  80 </div>
  81 <div class="section" id="greenlets-to-the-rescue">
  82 <h2>Greenlets to the rescue<a class="headerlink" href="#greenlets-to-the-rescue" title="Permalink to this headline">¶</a></h2>
  83 <p>Most servers limit the size of their worker pools to a relatively low number of concurrent threads, due to the high overhead involved in switching between and creating new threads. While threads are cheap compared to processes (forks), they are still expensive to create for each new connection.</p>
  84 <p>The <a class="reference external" href="http://www.gevent.org/">gevent</a> module adds <em>greenlets</em> to the mix. Greenlets behave similar to traditional threads, but are very cheap to create. A gevent-based server can spawn thousands of greenlets (one for each connection) with almost no overhead. Blocking individual greenlets has no impact on the servers ability to accept new requests. The number of concurrent connections is virtually unlimited.</p>
  85 <p>This makes creating asynchronous applications incredibly easy, because they look and feel like synchronous applications. A gevent-based server is actually not asynchronous, but massively multi-threaded. Here is an example:</p>
  86 <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">gevent</span> <span class="kn">import</span> <span class="n">monkey</span><span class="p">;</span> <span class="n">monkey</span><span class="o">.</span><span class="n">patch_all</span><span class="p">()</span>
  87
  88 <span class="kn">from</span> <span class="nn">time</span> <span class="kn">import</span> <span class="n">sleep</span>
  89 <span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">route</span><span class="p">,</span> <span class="n">run</span>
  90
  91 <span class="nd">@route</span><span class="p">(</span><span class="s">&#39;/stream&#39;</span><span class="p">)</span>
  92 <span class="k">def</span> <span class="nf">stream</span><span class="p">():</span>
  93     <span class="k">yield</span> <span class="s">&#39;START&#39;</span>
  94     <span class="n">sleep</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
  95     <span class="k">yield</span> <span class="s">&#39;MIDDLE&#39;</span>
  96     <span class="n">sleep</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
  97     <span class="k">yield</span> <span class="s">&#39;END&#39;</span>
  98
  99 <span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s">&#39;0.0.0.0&#39;</span><span class="p">,</span> <span class="n">port</span><span class="o">=</span><span class="mi">8080</span><span class="p">,</span> <span class="n">server</span><span class="o">=</span><span class="s">&#39;gevent&#39;</span><span class="p">)</span>
 100 </pre></div>
 101 </div>
 102 <p>The first line is important. It causes gevent to monkey-patch most of Python&#8217;s blocking APIs to not block the current thread, but pass the CPU to the next greenlet instead. It actually replaces Python&#8217;s threading with gevent-based pseudo-threads. This is why you can still use <tt class="docutils literal"><span class="pre">time.sleep()</span></tt> which would normally block the whole thread. If you don&#8217;t feel comfortable with monkey-patching python built-ins, you can use the corresponding gevent functions (<tt class="docutils literal"><span class="pre">gevent.sleep()</span></tt> in this case).</p>
 103 <p>If you run this script and point your browser to <tt class="docutils literal"><span class="pre">http://localhost:8080/stream</span></tt>, you should see <cite>START</cite>, <cite>MIDDLE</cite>, and <cite>END</cite> show up one by one (rather than waiting 8 seconds to see them all at once). It works exactly as with normal threads, but now your server can handle thousands of concurrent requests without any problems.</p>
 104 <div class="admonition note">
 105 <p class="first admonition-title">Note</p>
 106 <p class="last">Some browsers buffer a certain amount of data before they start rendering a
 107 page. You might need to yield more than a few bytes to see an effect in
 108 these browsers. Additionally, many browsers have a limit of one concurrent
 109 connection per URL. If this is the case, you can use a second browser or a
 110 benchmark tool (e.g. <cite>ab</cite> or <cite>httperf</cite>) to measure performance.</p>
 111 </div>
 112 </div>
 113 <div class="section" id="event-callbacks">
 114 <h2>Event Callbacks<a class="headerlink" href="#event-callbacks" title="Permalink to this headline">¶</a></h2>
 115 <p>A very common design pattern in asynchronous frameworks (including tornado, twisted, node.js and friends) is to use non-blocking APIs and bind callbacks to asynchronous events. The socket object is kept open until it is closed explicitly to allow callbacks to write to the socket at a later point. Here is an example based on the <a class="reference external" href="http://www.tornadoweb.org/documentation#non-blocking-asynchronous-requests">tornado library</a>:</p>
 116 <div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">MainHandler</span><span class="p">(</span><span class="n">tornado</span><span class="o">.</span><span class="n">web</span><span class="o">.</span><span class="n">RequestHandler</span><span class="p">):</span>
 117     <span class="nd">@tornado.web.asynchronous</span>
 118     <span class="k">def</span> <span class="nf">get</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
 119         <span class="n">worker</span> <span class="o">=</span> <span class="n">SomeAsyncWorker</span><span class="p">()</span>
 120         <span class="n">worker</span><span class="o">.</span><span class="n">on_data</span><span class="p">(</span><span class="k">lambda</span> <span class="n">chunk</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">chunk</span><span class="p">))</span>
 121         <span class="n">worker</span><span class="o">.</span><span class="n">on_finish</span><span class="p">(</span><span class="k">lambda</span><span class="p">:</span> <span class="bp">self</span><span class="o">.</span><span class="n">finish</span><span class="p">())</span>
 122 </pre></div>
 123 </div>
 124 <p>The main benefit is that the request handler terminates early. The handling thread can move on and accept new requests while the callbacks continue to write to sockets of previous requests. This is how these frameworks manage to process a lot of concurrent requests with only a small number of OS threads.</p>
 125 <p>With Gevent+WSGI, things are different: First, terminating early has no benefit because we have an unlimited pool of (pseudo)threads to accept new connections. Second, we cannot terminate early because that would close the socket (as required by WSGI). Third, we must return an iterable to conform to WSGI.</p>
 126 <p>In order to conform to the WSGI standard, all we have to do is to return a body iterable that we can write to asynchronously. With the help of <a class="reference external" href="http://www.gevent.org/gevent.queue.html">gevent.queue</a>, we can <em>simulate</em> a detached socket and rewrite the previous example as follows:</p>
 127 <div class="highlight-python"><div class="highlight"><pre><span class="nd">@route</span><span class="p">(</span><span class="s">&#39;/fetch&#39;</span><span class="p">)</span>
 128 <span class="k">def</span> <span class="nf">fetch</span><span class="p">():</span>
 129     <span class="n">body</span> <span class="o">=</span> <span class="n">gevent</span><span class="o">.</span><span class="n">queue</span><span class="o">.</span><span class="n">Queue</span><span class="p">()</span>
 130     <span class="n">worker</span> <span class="o">=</span> <span class="n">SomeAsyncWorker</span><span class="p">()</span>
 131     <span class="n">worker</span><span class="o">.</span><span class="n">on_data</span><span class="p">(</span><span class="k">lambda</span> <span class="n">chunk</span><span class="p">:</span> <span class="n">body</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="n">chunk</span><span class="p">))</span>
 132     <span class="n">worker</span><span class="o">.</span><span class="n">on_finish</span><span class="p">(</span><span class="k">lambda</span><span class="p">:</span> <span class="n">body</span><span class="o">.</span><span class="n">put</span><span class="p">(</span><span class="ne">StopIteration</span><span class="p">))</span>
 133     <span class="k">return</span> <span class="n">body</span>
 134 </pre></div>
 135 </div>
 136 <p>From the server perspective, the queue object is iterable. It blocks if empty and stops as soon as it reaches <tt class="docutils literal"><span class="pre">StopIteration</span></tt>. This conforms to WSGI. On the application side, the queue object behaves like a non-blocking socket. You can write to it at any time, pass it around and even start a new (pseudo)thread that writes to it asynchronously. This is how long-polling is implemented most of the time.</p>
 137 </div>
 138 <div class="section" id="finally-websockets">
 139 <h2>Finally: WebSockets<a class="headerlink" href="#finally-websockets" title="Permalink to this headline">¶</a></h2>
 140 <p>Lets forget about the low-level details for a while and speak about WebSockets. Since you are reading this article, you probably know what WebSockets are: A bidirectional communication channel between a browser (client) and a web application (server).</p>
 141 <p>Thankfully the <a class="reference external" href="http://pypi.python.org/pypi/gevent-websocket/">gevent-websocket</a> package does all the hard work for us. Here is a simple WebSocket endpoint that receives messages and just sends them back to the client:</p>
 142 <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">bottle</span> <span class="kn">import</span> <span class="n">request</span><span class="p">,</span> <span class="n">Bottle</span><span class="p">,</span> <span class="n">abort</span>
 143 <span class="n">app</span> <span class="o">=</span> <span class="n">Bottle</span><span class="p">()</span>
 144
 145 <span class="nd">@app.route</span><span class="p">(</span><span class="s">&#39;/websocket&#39;</span><span class="p">)</span>
 146 <span class="k">def</span> <span class="nf">handle_websocket</span><span class="p">():</span>
 147     <span class="n">wsock</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">&#39;wsgi.websocket&#39;</span><span class="p">)</span>
 148     <span class="k">if</span> <span class="ow">not</span> <span class="n">wsock</span><span class="p">:</span>
 149         <span class="n">abort</span><span class="p">(</span><span class="mi">400</span><span class="p">,</span> <span class="s">&#39;Expected WebSocket request.&#39;</span><span class="p">)</span>
 150
 151     <span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
 152         <span class="k">try</span><span class="p">:</span>
 153             <span class="n">message</span> <span class="o">=</span> <span class="n">wsock</span><span class="o">.</span><span class="n">receive</span><span class="p">()</span>
 154             <span class="n">wsock</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="s">&quot;Your message was: </span><span class="si">%r</span><span class="s">&quot;</span> <span class="o">%</span> <span class="n">message</span><span class="p">)</span>
 155         <span class="k">except</span> <span class="n">WebSocketError</span><span class="p">:</span>
 156             <span class="k">break</span>
 157
 158 <span class="kn">from</span> <span class="nn">gevent.pywsgi</span> <span class="kn">import</span> <span class="n">WSGIServer</span>
 159 <span class="kn">from</span> <span class="nn">geventwebsocket</span> <span class="kn">import</span> <span class="n">WebSocketHandler</span><span class="p">,</span> <span class="n">WebSocketError</span>
 160 <span class="n">server</span> <span class="o">=</span> <span class="n">WSGIServer</span><span class="p">((</span><span class="s">&quot;0.0.0.0&quot;</span><span class="p">,</span> <span class="mi">8080</span><span class="p">),</span> <span class="n">app</span><span class="p">,</span>
 161                     <span class="n">handler_class</span><span class="o">=</span><span class="n">WebSocketHandler</span><span class="p">)</span>
 162 <span class="n">server</span><span class="o">.</span><span class="n">serve_forever</span><span class="p">()</span>
 163 </pre></div>
 164 </div>
 165 <p>The while-loop runs until the client closes the connection. You get the idea :)</p>
 166 <p>The client-site JavaScript API is really straight forward, too:</p>
 167 <div class="highlight-python"><pre>&lt;!DOCTYPE html&gt;
 168 &lt;html&gt;
 169 &lt;head&gt;
 170   &lt;script type="text/javascript"&gt;
 171     var ws = new WebSocket("ws://example.com:8080/websocket");
 172     ws.onopen = function() {
 173         ws.send("Hello, world");
 174     };
 175     ws.onmessage = function (evt) {
 176         alert(evt.data);
 177     };
 178   &lt;/script&gt;
 179 &lt;/head&gt;
 180 &lt;/html&gt;</pre>
 181 </div>
 182 </div>
 183 </div>
 184
 185
 186
 187           </div>
 188         </div>
 189       </div>
 190       <div class="sphinxsidebar">
 191         <div class="sphinxsidebarwrapper">
 192             <p class="logo"><a href="index.html">
 193               <img class="logo" src="_static/logo_nav.png" alt="Logo"/>
 194             </a></p>
 195   <h3><a href="index.html">Table Of Contents</a></h3>
 196   <ul>
 197 <li><a class="reference internal" href="#">Primer to Asynchronous Applications</a><ul>
 198 <li><a class="reference internal" href="#the-limits-of-synchronous-wsgi">The Limits of Synchronous WSGI</a></li>
 199 <li><a class="reference internal" href="#greenlets-to-the-rescue">Greenlets to the rescue</a></li>
 200 <li><a class="reference internal" href="#event-callbacks">Event Callbacks</a></li>
 201 <li><a class="reference internal" href="#finally-websockets">Finally: WebSockets</a></li>
 202 </ul>
 203 </li>
 204 </ul>
 205
 206   <h4>Previous topic</h4>
 207   <p class="topless"><a href="tutorial_app.html"
 208                         title="previous chapter">Tutorial: Todo-List Application</a></p>
 209   <h4>Next topic</h4>
 210   <p class="topless"><a href="recipes.html"
 211                         title="next chapter">Recipes</a></p>
 212
 213
 214   <h3>This Page</h3>
 215   <ul class="this-page-menu">
 216     <li><a href="https://github.com/defnull/bottle/blob/master/docs/async.rst" rel="nofollow">Show Source @GitHub</a></li>
 217   </ul>
 218
 219
 220 <h3>Like it?</h3>
 221 <ul>
 222   <li>
 223     <form action="https://www.paypal.com/cgi-bin/webscr" method="post">
 224
 225       <a href="http://flattr.com/thing/21888/Bottle-A-Python-Web-Framework" target="_blank">
 226         <img src="http://api.flattr.com/button/flattr-badge-large.png" alt="Flattr this" title="Flattr this" border="0" />
 227       </a>
 228
 229       <iframe style="border: 0; margin: 0; padding: 0;"
 230         src="https://www.gittip.com/defnull/widget.html"
 231         width="48pt" height="20pt">
 232       </iframe>
 233
 234       <input type="hidden" name="cmd" value="_s-xclick">
 235       <input type="hidden" name="hosted_button_id" value="10013866">
 236       <input type="image" src="_static/paypal.png" border="0" name="submit" alt="Donate with PayPal!">
 237       <img alt="" border="0" src="https://www.paypal.com/de_DE/i/scr/pixel.gif" width="1" height="1">
 238     </form>
 239
 240   </li>
 241 </ul>
 242 <div id="searchbox" style="display: none">
 243   <h3>Quick search</h3>
 244     <form class="search" action="search.html" method="get">
 245       <input type="text" name="q" />
 246       <input type="submit" value="Go" />
 247       <input type="hidden" name="check_keywords" value="yes" />
 248       <input type="hidden" name="area" value="default" />
 249     </form>
 250     <p class="searchtip" style="font-size: 90%">
 251     Enter search terms or a module, class or function name.
 252     </p>
 253 </div>
 254 <script type="text/javascript">$('#searchbox').show(0);</script>
 255         </div>
 256       </div>
 257       <div class="clearer"></div>
 258     </div>
 259     <div class="related">
 260       <h3>Navigation</h3>
 261       <ul>
 262         <li class="right" style="margin-right: 10px">
 263           <a href="genindex.html" title="General Index"
 264              >index</a></li>
 265         <li class="right" >
 266           <a href="py-modindex.html" title="Python Module Index"
 267              >modules</a> |</li>
 268         <li class="right" >
 269           <a href="recipes.html" title="Recipes"
 270              >next</a> |</li>
 271         <li class="right" >
 272           <a href="tutorial_app.html" title="Tutorial: Todo-List Application"
 273              >previous</a> |</li>
 274     <li><a href="/">Project Home</a> &raquo;</li>
 275
 276         <li><a href="index.html">Bottle 0.12-dev documentation</a> &raquo;</li>
 277
 278       </ul>
 279     </div>
 280     <div id="disqus_thread" style="margin: 2em 0;"></div>
 281     <script type="text/javascript">
 282       var disqus_shortname = 'bottlepy';
 283       var disqus_identifier = 'docs_async';
 284       var disqus_title = 'Primer to Asynchronous Applications';
 285       //var disqus_url = 'http://example.com/permalink-to-page.html';
 286       (function() {
 287         var dsq = document.createElement('script');
 288         dsq.type = 'text/javascript';
 289         dsq.async = true;
 290         dsq.src = 'http://zodbbook.disqus.com/embed.js';
 291         document.getElementsByTagName('head')[0].appendChild(dsq);
 292       })();
 293     </script>
 294     <div class="footer">
 295     &copy; <a href="index.html#license">Copyright</a> 2009-2012, Marcel Hellkamp - <a href="contact.html">Contact</a><br />
 296     Last updated on Nov 11, 2012. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3.<br />
 297     Powered by Bottle 0
 298     </div>
 299
 300   </body>
 301 </html>