Log in

entries friends calendar profile Scribbles in the Dark Previous Previous Next Next
The Olestra of the Web - Please Visit http://glyph.twistedmatrix.com/ - This Blog Is Closed.
Sorry about the ads, I can't turn them off.
The Olestra of the Web
I suppose if you do not know what Python is or what the Web is, maybe you don't know this, but Guido has been asking about web frameworks.

I wish I had something more substantial to throw into the discussion here, but if you're interested in my work on the subject, you know where to find it.

Python's web situation is a mess, there's no denying it. Even if there were some totally awesome framework which everyone should be using that was head and shoulders above everything else, it would take a while to figure that out at this point, and to achieve some consensus, just by the sheer amount of work required to get everyone's attention, and once the right people had found it, to focus the larger community on it. Unfortunately I think it's clear that no such silver bullet exists, and that there is unlikely to be one any time soon. Even if there were, larger projects like Mantissa and Zope would still exist, because it's not likely that most people's criterea for "web framework silver bullet" includes "integrated digital telephony".

The general frustration with (and bewilderment at) this situation has been breeding a poisonous feeling among Python developers though, I fear. I'm calling it "disenframeworkentarianism", and it is best typified by this post (unrelated, but linked-to by Guido's discussion). There is a strong desire for all the benefits of frameworks without any of the overhead. "Frameworks are too fattening for my code!", we exclaim, "Let's come up with something that has all the delicious flavor but none of the calories." This is the same line of reasoning which lead to Olestra, and various other food-substitutes.

While superficially the idea of a food that tastes great and has no impact on your weight is a great idea, it turns out that there is a word for things that you ingest which don't do anything for your body: poison. For example, I discovered a wonderful article recently: Why No One Should Eat Products That Contain Olestra.

Frameworks with zero "fat", that is, overhead imposed on your code, are probably not actually doing anything for you. It looks like they're "lighter-weight" than other frameworks, because they don't provide any of the "fat", that is, the cognitive overhead required to learn the framework, or as Guido puts it, "lavish quantities of kool-aid". By not providing a consistent view of the world from the framework's perspective, you end up having to do more of that work yourself. You have saved time learning someone else's vision of the world, but instead of actually saving time overall, you've had to construct your own island of functionality, totally isolated from other developers and integration opportunities. It may well be that you needed to do that work, because the requirements of your project are sufficiently bizarre and integration-heavy that you are really going to have to tie things together yourself anyway - that doesn't mean it's not more work.

I'm not saying this as a defense of super-heavy-weight frameworks either. The opposite of Olestra is not a bucket of fried lard, it's a balanced diet, with plenty of low-calorie things like fresh leafy vegetables. J2EE is the Big Mac here; plenty of calories, but most of them empty. I think this metaphor could be extended well into absurdity, with the requirements of your project being its "metabolism", unit tests being "exercise"... but let me get back to the actual point here.

No project has come along yet that really tries as hard as it can to be completely modular; i.e. let you plug in your own solution for any layer of any stack with no overhead. There's good reason - whether it's a good idea or not, it's just too hard. When frameworks do try to let you choose, it turns out that somebody has to come along and pick up the pieces, and do the work of getting the disparate pieces that you want to work together, to actually work together. The trendy thing these days is to call this a "megaframework", but I think it's a symptom for this frameworks-proliferation disease, not the cure. It's largely a matter of perspective. In some cases aggregation works wondefully - Ubuntu is the mega-est megaframework of them all, and it's fantastic. At some level it just falls apart though - nobody tries to cobble together a word processor from "best of breed" word-wrap algorithms and icons; you use AbiWord or you use OpenOffice, and you don't complain that you can't use AbiWord's tables feature in OpenOffice documents - they're just different tools, and you have to decide on one or the other.

I think that the community would mostly be better served right now by some straightforward competition - try to create the best full-stack experience, and maybe eventually it will become clear that project X really does have the best ORM and project Y has the best templating system, and the user communities can exert some pressure on the development teams to merge, but until then we need good, full, complete experiences for developers trying to build sites, whether they're megaframeworks or kiloframeworks or gigaframeworks, not half-finished chunks of functionality you are expected to finish yourself. It's much easier to extract just the part that you need from a polished, working solution than it is to create a polished, working solution from a handful of parts that by themselves do nothing useful, even if they're well-built parts.

Since it probably needs to be said... While I'm a "framework" guy, and I obviously have my own axe to grind here, note that I'm not commenting on Guido's blog (or responding to it here) and saying "Use Nevow!!!!" (Although, it seems like there are enough people doing that for me ;-)) There are serious problems with all current web frameworks, Nevow included. I think Nevow could be really awesome, but there are parts of it which really need to be removed and streamlined, cleaned up, and hidden from view. Let's face it, Nevow guys - the sequence renderer is a totem of Nevow's failure, at least from the perspective of comprehensibility to new users. Also, as Tim Peters famously said, the "it's surely too much for a lesser being" argument is pretty weak - I trip over sequence so regularly that I've stopped bothering to use it.

So, I've got to say, if you're in the market for one more tool in your build-your-own-web-framework toolbox, Nevow is probably not for you. If you're looking at a larger system, with object publishing, multi-protocol support, sessions, authentication, modular AJAX, and a variety of other features, it is a good tool to use, though. Especially the AJAX thing - most of Nevow's unnecessary complexity is worth wading through just to get to Athena, the AJAX-plus-javascript-modules-plus-pluggable-page-fragments Web 2.0 bonanza feature. There aren't enough examples, but I really haven't seen anything like it before. You can extend that even further with Mantissa, to include an integrated database, sign-up, account maintenance, administrative tools, per-user plug-ins, navigation, and so on - but Mantissa and Nevow are separated for good reason; Mantissa is still much more heavily in development, and is a much larger and more ambitious project. I am really looking forward to it stabilizing so that it is a more robust example to show other framework developers some of my ideas.
13 comments or Leave a comment
From: mfcoyle Date: February 1st, 2006 06:21 am (UTC) (Link)

Web Frameworks: All fun and games until you need to expand beyond HTTP

I threw my $.02 on web frameworks and Twisted in the following blog entry:


(Deleted comment)
From: corydodt Date: February 1st, 2006 05:25 pm (UTC) (Link)

Re: Web Frameworks: All fun and games until you need to expand beyond HTTP

Commenting here on mfcoyle since that blog doesn't appear to have comments: Very nice piece. I think the key point is that the Twisted environment comes with a vision that goes far beyond putting up some stupid little web page, and it's an accurate vision, and the parts of the dev world that can't see it themselves will be stuck in a dead end sooner or later.

Also, you meant to say "complement".
(Deleted comment)
anarkystic From: anarkystic Date: February 1st, 2006 03:28 pm (UTC) (Link)
"Some people live to dis whatever the flavor of the month is, and sometimes pass it off as something significant." I find this somewhat insulting but will chalk it up to you not having actually read any other content I have written and hope you might take the chance to do so. Should you be so inclined, I threw my various framework-related posts into a category, http://an9.org/devdev/taxonomy/term/4
(Deleted comment)
(Deleted comment)
jcalderone From: jcalderone Date: February 1st, 2006 01:59 pm (UTC) (Link)

Athena Tutorial

There's a brief tutorial on the Divmod wiki, for anyone interested in Athena. Hopefully more to come soon.
From: bitjuggler Date: February 1st, 2006 02:45 pm (UTC) (Link)
> Frameworks with zero "fat", that is, overhead imposed on your code, are probably not actually doing anything for you.

I think that's the point - I don't want a framework to do anything for me *unless I ask it to*. I want to feel like I'm wrapping my application around the framework, not trying to make it 'fit inside'. The spatial metaphors may be a matter of a point of view, but there's a very different feel, to me at least, between trying to write an application that 'fits into' a large framework and writing an application that 'sits on top of' a library layer like cgi.py or BaseHTTP*.
jcalderone From: jcalderone Date: February 1st, 2006 03:26 pm (UTC) (Link)
I could stick with your spatial metaphor and say that when you stack things up they fall over, and when you nest things inside of other things the result is stable.

But metaphors are just a way of lying, right? :)

Metaphors aside, you say that you want to write software in a particular way, that you want to feel a certain way while you write it. Okay. Desires and feelings might be expressions of a valid argument, but they are not valid by themselves. Can you explain why you want this, and why you feel this way?

I'm tempted to frame the question a little by explaining why I prefer to write most software to a framework, but I think I'll marshal those thoughts a little more carefully and make them into a real blog entry.
From: bitjuggler Date: February 1st, 2006 04:48 pm (UTC) (Link)
When write code using a library, my code controls the library. When I write code in a framework, it feels more like my code is being controlled by the framework. There's a lot more uncertainty, in my head at least, about the context in which my code is being called and how much control it has. If I call a library, well, I control those things, and so I have a better mental model of what's going on. This is also why 'thin' frameworks are more tolerable than 'megaframeworks'... a thin framework has a more-understandable mental model. I can tolerate calling into a 'black box' library that does stuff that I want, because I'm in control and I know what the constraints are, but I dislike having to write code that executes *inside* a 'black box' because I don't have a good idea of the environment my code is running in, and understanding that environment is 'learning the framework' aka 'drinking the koolaid', which is more effort than I feel should be necessary. Also, it's effort that has to be undertaken before I feel comfortable writing even one line of code... whereas it's more comfortable to use a library because I only have to learn about the parts that I use when I want or need to use them.
glyf From: glyf Date: February 2nd, 2006 07:48 am (UTC) (Link)
What are your feelings on GUI libraries? Is there a difference in your mental model between executing in the event-handler for a button being pressed by a user and in an event-handler for some data being received from the network?
From: zooko Date: February 1st, 2006 07:30 pm (UTC) (Link)

what is everyone talking about?

I'm still not quite sure when something is a "framework" and when it is a "library". Glyph seems to be talking about a wide range of functionality that is tested and integrated together, but most other people seem to be talking about whether the typical control flow is that your code invokes the other code or that the other code invokes yours. Those people seem to be confusing "control flow" with "control".
13 comments or Leave a comment