There is a a strain of thought in the Python web development community these days that considers web frameworks a bad idea. Even beginners are sometimes told: why do you need a framework anyway? Just build your app from scratch with WSGI! Or just compose your own out of existing libraries and tools! I'm going to argue that frameworks are useful. I'm going to argue that we should normally not be telling beginners to avoid frameworks. And I'm going to argue that experienced developers should carefully consider whether their perspective isn't warped a little by their experience, and that frameworks can be useful for them too. I want to do something that should be simple, and would be simple if I just did it at a low level. But because the framework forces a certain way of working on me, it becomes needlessly difficult. I have to create workarounds, and it sucks. Frameworks can get into your way, as they've made certain choices. How much they are constraining depends on the framework, your experience with it, and what you're trying to accomplish. But if you picked the right framework for the job, regularly the task you're trying to perform is something that a framework makes very easy. The first time around you use a framework for something that it is good at, you're going to be impressed at how easy it was to accomplish your task. Before long however, you might become so used to the benefits that a framework brings that you won't even notice what the framework is doing for you anymore; it's now in the background. And humans are very attuned to pain, so the pain points are going to remain. You're going to remember the ugly workarounds better than the times when things just worked. I think frameworks get in the way for beginners less than they do for experienced developers. Beginners need to learn the basics, and having a framework can be a useful guide to gain an understanding of the basics and how they go together. An experienced developer will understand all that already, and is more likely to work on more challenging projects where the choices that the framework made are not the right ones. But does that mean that the framework is wrong for all tasks, and that we should tell a beginner not to use them at all? And does it really mean that experienced developers shouldn't use frameworks at all anymore, either? What about those tasks where the pain is minimal? And what if you are working in a group that includes beginners? And may it be that your time is more productively spent improving the framework to make the pain go away, instead of avoiding it entirely? A developer might argue to the last point: no, I am a lot more productive if I build something from scratch, and list a whole range of things they've built up in a short amount of time. And of course building something from scratch can feel very liberating, and sometimes it is the right way to go. But frequently you will only feel more productive - if you're spending time to reimplement the features of the framework you discarded, you might be productive in implementing those features, but what about the task you're actually trying to accomplish? What about the web app you're trying to build? And how much code that only you understand will you end up trying to maintain? There are cases where you may feel a lot more productive, because you're having more fun, when you actually aren't. We all know programmers like to reinvent wheels. Reusing existing code may be more painful, but it isn't necessarily less productive, although of course it can be. And seriously, do we really think beginners without much experience of frameworks have enough knowledge to be able to build their web apps from scratch and do it well? Lots of people would agree with me that building things from scratch is not the way to go. But, they'll argue, people can just assemble the components they need for their application themselves, instead of using a framework. And it's true: there are a lot of building blocks available these days, and that's good. There are libraries and middleware and components and so on to do everything from interfacing with a web server to talking to a database. Instead of picking a ready-made pre-assembled framework, you can instead pick and choose the best of breed components that are out there and use those to develop your application. Since you are doing it, it may be better suited to your requirements than any pre-assembled framework can offer. You'll understand it better too. I believe there is a lot of value in this approach. The Python web development world has been moving in this direction for a while; there's the emphasis on WSGI, and there is the emphasis on easily distributing and combining libraries. I'll note that the Python web framework development has been moving in this direction too, because web framework developers are also aware of the benefit of sharing useful components with each other. But does this mean we should recommend to everyone all the time to just ignore frameworks and assemble the bits themselves? No. The obvious case where this is a bad idea is in the case of beginners. A beginner will have little understanding of how components can go together. A beginner will have no way to evaluate components. A beginner is usually much better off to use a pre-assembled framework where these choices have already been made. The beginner can find information in a one-stop shop: they'll learn about the template language and the approach to databases of the framework in one place, and they'll see not just descriptions of the bits and pieces but also how they go together. This means that they won't have to do this integration themselves anymore; they'll see examples. There's also a community where they can ask questions. These reasons for a beginner to use a pre-assembled framework also frequently apply to more experienced developers. You don't always have time to go out and assemble components together, and figure out how to make it work. Sure, what you might end up with might be slightly better suited to a particular task than a pre-assembled framework, but is it really always worth the effort? Do you really want to make all those choices each time, for each app you build? And you'll still be stuck with the maintenance burden - the glue code you wrote of course, but also the assembly itself: what if there are newer releases of the components you are using? Finally, these smaller components are often quite complex frameworks too - and if you integrate them yourself there might be less unity of vision than a good framework can offer, and the whole might end up harder to understand. The burden of assembling and integrating best of breed components can be shared: that's what the developers of a framework do. And if you base your work on a pre-assembled framework, it's likely to be less work for you to upgrade, because the framework developers will have taken care that the components in the framework work together in newer versions. There is also a larger chance that people will write extensions and documentation for the this same assembly, and that is very likely something you may benefit from in the future. So it often makes sense to share the assembling and integration of components in an open source fashion just like it makes sense to share the components themselves. An assembly is not just a collection of loose parts, it can be a new thing, with a vision of it own. All the points above are true: frameworks can get into your way, using a framework (especially when wrestling with it) can feel less productive than building something new from scratch, and reusing components yourself is often more flexible. But frameworks can also take care of a lot of issues for you, even though after a while you only feel it when they are in the way. Frameworks make choices for you so you don't have to make them all the time. Frameworks can save time. Frameworks offer an integrated whole so you won't have to worry about the rest of the world for a while. Good frameworks will also be flexible enough to handle a huge amount of tasks more than adequately. So, there is a place for web frameworks. It makes sense to recommend web frameworks to beginners. It also makes sense for experienced developers to consider using a web framework. It's not always the right tool for the job, but it often is. By all means let's discuss the particular pain points of frameworks, doing things without a framework and assembling components yourself. But let's not forget the benefits that web frameworks bring to many of us.
![[Comments]](/blog-content//themes/Default/comments.png "10 comments")
(10) Thu Apr 15 2010 19:22 web frameworks considered useful:
Frameworks get in your way
Building from scratch feels more productive
Don't use a framework, just reuse existing building blocks
Conclusion
- Comments:
Posted by Dave R. at Thu Apr 15 2010 20:48
Agree completely regarding beginners and web applications. The biggest matter is security. There are a ton of really important, really tricky and pretty obscure things that you have to do to ensure a web-based application (or script--anything that accepts input) is safe.Best to start with a framework, then learn exactly what it's doing and why.
Posted by Vasudev Ram at Fri Apr 16 2010 00:26
Excellent and thought-provoking post. I agree that there are lot of pros and cons to this whole issue, as you describe. One good point that you bring out, I think (apart from the many other good observations you make in the post), is that a framework is just that - a piece of (extensible) software that implements a set of choices (by its designer) on how to handle a particular software problem (or domain - using domain in the wider sense of any "area", be it tech or biz, not just "business domain"). I think people tend to forget that sometimes, hence the talk about frameworks being limiting, etc. The point is that they ARE limiting - intentionally so; in order to be specialized to, and more useful for, that particular area, they have to limit what they can handle or do. But by putting those constraints, they (at least the good ones) tend to give you more power and productivity for development in the area for which they are designed.- VasudevPosted by Graham Dumpleton at Fri Apr 16 2010 02:10
Part of the problem is that beginners when asking about what to use quite often have a preconceived notion that they don't want to use a full stack framework, and instead want to do everything from scratch. Sometimes this can be put down to delusions of what they are capable of, but either way, it doesn't give you much wiggle room. Often the best you can manage is to direct them to Paste and Werkzeug, or some of the emerging micro frameworks. Suggesting a full framework usually just falls on deaf ears.
Posted by Wu at Fri Apr 16 2010 09:32
"I think frameworks get in the way for beginners less than they do for experienced developers. Beginners need to learn the basics, and having a framework can be a useful guide to gain an understanding of the basics and how they go together."Well, imho "beginners" that have a lot of trouble understanding the basics of today's frameworks will probably develop a poor-quality solution from scratch, and I don't think that is a good idea. I mean, someone that is able to create a great app from scratch probably would be able to start doing things really quickly in web development frameworks (even quicker because of the good documentation and community support they have).If you think of it, using a web framework could be even a better idea, as you will learn (for sure) a lot of things.
Posted by Malthe Borch at Fri Apr 16 2010 09:37
I recently started a new project and I find myself very productive using: - WSGI
- Paste
- WebOb
- SQLAlchemy
- a routes-like URL dispatcher (in my case, Otto)
- a template engine (in my case, Chameleon)Some of these are frameworks (WSGI, Paste)---but they're not the kind of frameworks you talk about.I think there's a value in telling users: you can write just about any application using the above concoction, perhaps with the inclusion of a form framework. I would use - repoze.formapibecause it's simple and gets out of your way.As for security, disregarding attacks, there's definitely a value in using a security framework. I quite like BFG's view security.Note that it's very easy to transition from the stack I've outlined above to something like BFG. But it's not the starting point I would use. I might end there. I don't think I'll end up transitioning to one of the larger frameworks. My feeling is that it invariably ends in tears.Posted by Martijn Faassen at Fri Apr 16 2010 13:51
Malthe, the above concoction, should you document it and tell other people about it and try to get a group of people to use it, is a framework. And just pieces of that are enormously frameworky already, like SQLAlchemy.Also, clearly, obviously, using a "larger" framework (for whatever meaning of "larger") doesn't invariably end in tears. That's a rather silly sweeping statement that's so easily disproved by just my personal experience alone that I'm not going to do it.If you want to convince beginners, either in web development or Python development, and also many non-beginners, that your concoction is a good idea that they should also use, you'll simply do a better job if you produce coherent documentation about it that demonstrates how it works and why it is nice, produce a list of dependencies, and probably some glue code. And then I'll argue you're presenting a framework, even if you're saying it's not.Someone like you, who composes frameworks (Concoction) themselves and writes major framework components (Otto, Chameleon), such as you, is simply not a typical case. I'm like you, but we shouldn't start pretending everybody is like us and has the same needs and interests.
Posted by j_king at Fri Apr 16 2010 16:50
Interesting post.On the one hand I think "beginners" should know how to build a framework from the bottom up. Otherwise they could end up treating the framework as a black-box. They may not have the skill to build a reliable, maintainable, fast, and secure framework themselves but at least they could make educated decisions when they choose one.On the other hand, they don't know how to build a reliable, maintainable, fast, and secure framework. They would also end up reinventing the proverbial wheel. Which could be bad because they might invent a poor substitute.But these are extremes. I think a healthy middle road is required.For me, that's Pylons. It is a framework, but not a "full stack" one like Django. It couples together popular Python modules. It's pretty easy to hook in your own modules or code where you need to. You can learn a lot about good framework design just by working with it.Great post. :)
Posted by Martin Aspeli at Sun Apr 18 2010 05:49
If nothing else, a framework is a place to hang best practice and documentation.Pylons is a good example. Not very much of it is really "Pylons-specific" in that it re-uses Paste, WebOb and so on. But the value is that someone's tested and documented how those pieces fit together, and there's a mailing list where you can talk to other people using the same stack in roughly the same way to get help - not just with specific bugs or issues, but with conceptualising what you're working with.It's completely unreasonable to expect someone who isn't a Python insider with years and years of experience to even *know about*, let alone gain proficiency in, the various pieces that could make up a web development stack. Almost everyone relies on someone else having put together a compelling proposition (which often comes down to documentation and reference implementations) and trusting that this will be "good enough".Some people (myself included) find writing frameworks fun. Some people like reductionism in code, breaking everything down to minimal dependencies and understanding every atomic part of the application.However, an awful lot of people have clients and deadlines and other interests and cannot afford (or do not want to) become experts before they become productive. I'd wager "Copy The Example" is still the most common style of programming in the world today. ;)Martin
Posted by Malthe Borch at Sun Apr 18 2010 09:17
I think I've been misunderstood; I do not expect anyone to pick up the stack of software I outlined and immediately become productive without prior knowledge.And Martin is quite right in saying that it's in particular the community around a framework that has value, e.g. mailing-lists, IRC channels, events, etc.That said, I wouldn't want to "ship" my stack, as is popular these days with frameworks that are essentially meta-packages. And for a community, the Repoze community (www.repoze.org) works for me as an all-round web-centric Python community. It has an IRC channel (#repoze) and a pretty active mailinglist.Maybe what's needed in terms of documentation is a set of recipes to get you going (e.g. how to wire in templates, what about authentication, etc).As a closing remark, I run ARCH Linux on my laptop; it's a kind of bare-bones distribution in which you have to configure things manually. At first it seems a little contrived, but after a couple of months, I greatly appreciate that I run simply "Linux" and that the generic documentation actually applies. The same approach can work with Python; if you have just "Python", then it's a lot easier to use the generic documentation.
Posted by Martijn Faassen at Mon Apr 19 2010 00:57
Sorry if I misunderstood you, Malthe.I'm fine if you have a stack of components that work well for you and that you're not particularly interested in having others use it as well in that configuration. That's fine and I'm sure you can do great with it. It's just not an approach I'd like to recommend to everybody, certainly not beginners.I also understand the attractiveness of bare-bones where you can see the moving parts very clearly. Then again, I also understand the attractiveness of user interface which hide the moving parts.
