Python Secret Weblog for 2011 August 2 (entry 0)

(7) Tue Aug 02 2011 15:59 WSGI: Bringing web frameworks closer together:

Recently I have seen a sentiment espoused by some people in the Python web community that WSGI has failed to live up to one of this promises: bringing Python web frameworks closer together.

Let's look at history and see whether this is true.

Here's a snapshot of the Python web framework landscape in 2005:

• Zope had its own way of hooking up to web servers. Several ways. FastCGI, SCGI, Apache Rewrite rules, ProxyPass.
• Django had just emerged, with its own way of hooking up to web servers. Several ways too; I can find references to using mod_python, FastCGI, SCGI and AJP.
• TurboGears had also just emerged. TurboGears 1.x used CherryPy as its web server. A quick search turns up references to hooking it up to Apache using mod_python, and using mod_rewrite.

Let's look at the web framework landscape in 2011:

• Zope (in all its incarnations) can be hooked up to web servers to WSGI, and this is generally the preferred method.
  • Zope 2 comes with native WSGI support.
  • Grok has native WSGI support and uses paster as its development server.
  • BlueBream has native WSGI support and uses paster as its development server.
• Django has WSGI as the preferred method to hook up to web servers.
• TurboGears is now built around WSGI and uses paster as its development server.
• Both Pylons the web framework as well as Pyramid are built around WSGI, and both use paster as its development server.

[no claim is made that this represents all Python web frameworks out there]

Has this brought web frameworks closer together? The answer is clearly a resounding YES: if you know how to hook up a WSGI application to your web server of choice, you have basic knowledge to deal with all of these web frameworks. You can use Apache mod_wsgi with all of them, for instance.

In addition, only Zope 2 and Django do not use paster as their default development web server. We see a major component being shared between Grok, BlueBream, TurboGears, Pylons and Pyramid. This knowledge transfers between web frameworks.

So this sentiment is clearly wrong. Thank you to everybody who has helped create and push WSGI!

Other arguments have been made, for instance that WSGI middleware in particular isn't bringing Python web frameworks closer together. I believe that's wrong too, but I will leave defending that point to someone more familiar with that topic.

---

Comments:

Posted by Rene Dudfield at Tue Aug 02 2011 17:57

Cherrypy does wsgi nicely too. It's a wsgi server, and was the first with the mod_wsgi apache module to support python 3.

mod_wsgi is the leading python apache module now. nginx supports the uWSGI project now too.

I've plugged wsgi apps into zope, django, and cherrypy projects just fine. Some crazy people have even integrated plone and django together into a single web app.

Posted by Martijn Faassen at Tue Aug 02 2011 18:15

Thanks for the info Rene.

I mentioned CherryPy as of 2005 as its WSGI support must've been pretty raw back then - and TurboGears I think was probably not yet using it for that purpose.

Posted by Achim Domma at Tue Aug 02 2011 18:36

I fully agree! I thought about moving to Ruby on Rails for web development, because I was always frustrated about getting python web apps up and running on servers. RoRs Passenger module works incredibly well and simple. But on Europython 2011 I saw how the python web development stuff has evovled and that WSGI is now "usable". So I'm very happy that I don't have to switch to Ruby! ;-)

Posted by PJ Eby at Tue Aug 02 2011 18:57

What you're describing is my stage 1 goal for WSGI; the stage 2 goal (see PEP 333's Rationale section) is:

"""If middleware can be both simple and robust, and WSGI is widely available in servers and frameworks, it allows for the possibility of an entirely new kind of Python web application framework: one consisting of loosely-coupled WSGI middleware components. Indeed, existing framework authors may even choose to refactor their frameworks' existing services to be provided in this way, becoming more like libraries used with WSGI, and less like monolithic frameworks. This would then allow application developers to choose "best-of-breed" components for specific functionality, rather than having to commit to all the pros and cons of a single framework."""

It is this second-stage goal that hasn't worked out so well as yet.

Posted by Martijn Faassen at Tue Aug 02 2011 19:14

PJE, thanks for commenting.

I think it's worked in part; there are such components out there (as middleware and endware), and it does happen that more than one web framework adopts them, or at least application developers that use the frameworks do. So it works in so much that it's now possible and that it happens.

The "ultimate" end point of the second-stage goal is difficult not only because it can be difficult to write correct middleware - it's also difficult because extracting a sensible middleware isn't always possible, because frameworks have legitimately different approaches, and because much of it would need to be endware with more communication needed than what WSGI provides, and thus more coupling and buy-in.

But most importantly, spreading "best of breed" components around requires those components to be presented as attractive Python libraries - useful functionality, good API, good documentation, responsive community, etc. This is just plain hard, WSGI or not.

But what WSGI does do is enhance the possible scope of such libraries - now you can write a library that does something with the web that can be plugged into just about any web framework.

Posted by PJ Eby at Tue Aug 02 2011 20:10

"""more communication needed than what WSGI provides"""

Right - that's why WSGI Lite includes a binding protocol for that communication.

Right now I'm exploring whether we should also add a simple filtering mechanism to the binding machinery, so that you can do things like have an authorized-user argument depend on a session argument, and have the session argument be able to add a set-cookie output header, at the same time as the authorized-user argument may want to redirect the request to a login page.

The real trick is going to be doing this sort of thing in such a way that it doesn't turn into a new framework of its own, such that we just end up with N+1 frameworks. :-(

Posted by Aroldo Souza-Leite at Wed Aug 03 2011 08:17

Hey Martijn, hey folks, thanks for this discussion. This also helps non-technical people involved in web software production management.