Log in

entries friends calendar profile Scribbles in the Dark Previous Previous Next Next
Let's Get Pumped - Please Visit http://glyph.twistedmatrix.com/ - This Blog Is Closed.
Sorry about the ads, I can't turn them off.
Let's Get Pumped
I want to be excited about the python 3000 effort.  Every programmer loves a green field project; there's none of that icky legacy stuff holding you back, and you can have a beautiful, graceful new creation that exceeds the limitations of its hobbled and mis-designed ancestors.  Python 3000 could fix all of Python's warts, giving us a clean, simple language with more power and flexibility.

However, the "icky legacy stuff" happens to include every program I've written in the last 5 years, as well as really important functionality like GTK bindings, database support, and compatibility with applications (like GIMP, Gaim and Blender) which embed Python themselves. so I need something more substantial to get excited about.  Something to make it worth the rather onerous effort of upgrading the Twisted codebase, and simultaneously breaking support for millions of existing Python installations.  Some of the "substantial" new features I've seen, like the new "iostack" library, seem to be controversial.  I haven't done a thorough code review myself, but a few comments on the mailing list, like "8k is a perfect buffer size for any application", suggest that while there may be improvements, these changes are far from problem-free.  (Not to mention the fact that a large portion of what iostack is trying to accomplish is the sort of thing that would be features in Twisted anyway, so it's likely we won't be using much of that code...)

PyPy brings some of this effort along with it as well, but PyPy's advantages are much clearer: it will be about a zillion times faster, it will make writing bindings to existing native functionality fundamentally easier, and it might be possible to add core language interpreter features, like restricted execution, without having to patch the core itself.  Also, PyPy is currently targeting fundamentally the same language as Python 2.4, whereas Python 3000 is intentionally incompatible, so it will be possible to support Python 2.4 and PyPy, although PyPy may require a lot of conditional blocks to work right in the real world.

This is all a high-level understanding gathered from listening to rumors, perusing mailing list archives, reading a few websites, and attempting to read between the lines.  I could be wrong about both projects.  With my current understanding though, the plan for the Twisted project is to support the 2.x Python series and PyPy as soon as it's feasible, but ignore Py3k until there is a compatibility layer which would allow us to migrate gradually rather than in one fell swoop.

Lots of Python fans seem to read this blog, so maybe you can help me out.  What new features or idioms should I be really excited about in Python 3000?  Am I missing something fundamental about what it's trying to achieve?

(Those of you who don't have an OpenID login can feel free to answer this question by sending email to glyph@divmod.com rather than posting a comment.)
7 comments or Leave a comment
From: jerub Date: September 11th, 2006 02:28 am (UTC) (Link)
We've already talked on IRC. It worries me that no one has responded yet. I felt like responding in a big rant in my own blag, but on balance I feel like I should say something here.

Py3k represents an effort to make a new programming language that isn't Python. It's not python at all. I have a friend who is a programmer, who caused a bit of a hoo hah on the python-dev list with his distaste about changes in python.

What wasn't mentioned in his rant, or any of the follwoing material was exactly what kind of system or what its vintage was that had been broken twice by python upgrades.

The previous breakage that caused a problem was FCNTL, the module containing all of the constants for the fcntl module being folded into fcntl.

Greg went around and put all his installations on python2.3 and changed all the hashbang lines so that they explicitly referenced the python2.3 executable. Except on one system. That system was running python1.2, and he ecided not to upgrade it just yet.

Yep, 1.2. not 1.5. 1.2.

Python is an old language. Over a decade now, with legacy going back so far that people don't even think about it. On IRC I refer to python2.3 as being archiaic and obsolete, but there's code that runs on pythons much older than that.

So. That brings me to python3000. Python 3000 represents a new language. This language that is being discussed isn't python. It's Python 3000. Python 1 has a logicial continuation through all the way to python2.5. And in that time, a programmer has been bitten by *2* backwards compatbility problems. FCNTL going to fcntl, and someone changing time.strftime's behaviour.

Yep. 2.


Now. Python 3000 is suggesting dozens, if not hundreds of backwards compatiblity breaking changes. So many that automated tools are already in the works to take care of the simple changes, but we all know that there will be dodgy corner cases that no AST transformation will be able to help with.

Python. CPython. Python the language. Python that isn't Py3k. Python. Python must be, and must always be, maintained as a language. /usr/bin/python can never point at /usr/bin/py3k.
glyf From: glyf Date: September 11th, 2006 03:41 am (UTC) (Link)
Thanks for the comment, jerub...

I agree with most of your points. I assume (correctly, I hope) that the developers engaged in the py3k project, or those outside who are excited about it, have at least considered these concerns and believe that sufficient value is being added to warrant the massive changes.

I don't expect to ever be convinced that this kind of massive, risky, shoot-the-moon upgrade is a good idea, mind you, but I'd at least like to hear some reasons on the other side of the scale. If I were convinced, I might have enough time to work on a compatibility layer so that Twisted can support both runtimes at once, since I'm sure the neophiles who used decorator syntax before it was even out of beta will be all over py3k when it's released too... and, I hope, some of them will want to use Twisted.

So far, though, those neophiles, while not exactly silent, tend to take their viewpoint as obvious. The only argument I've heard in favor is the implicit "breaking compatibility would let us do such neat stuff" - the "neat stuff" seems to be obvious to everyone talking about it. Of course, the emperor -- or dictator, I suppose -- might just be naked, but (sarcastic comments aside) I have more confidence in the community than that.

This is where Python is going, after all, for better or worse. The, uh, vibrant development landscape around perl 5.8 shows what happens when a language's maintainers stop maintaining it. I echo your concerns about /usr/bin/python meaning "python" forever and ever, amen - but I don't have anywhere near the time or energy for even half of my own projects, let alone maintaining a fork of the Python codebase. So, until someone else does, if I have to live with the choices of python-dev I'd rather be happy about them!
From: ianbicking Date: September 11th, 2006 05:46 am (UTC) (Link)


The IO redesign seems like a horribly difficult but essential change if we get rid of 8-bit strings (replaced with bytes). It's a huge and particularly difficult change, but I think it's important and useful.

A lot of the other things aren't such a big deal. That .keys() is a view is a subtle one, for instance, but when used improperly it should cause an exception not just tricky behavior. Hopefully that's how many changes will look.

Then I also hope that any syntactic changes will start to appear in the 2.x series as __future__ features, so that it can be possible to support both versions (e.g., "except Exception as e" can only be supported in both 2 and 3 via __future__, since I don't think 3 will support "except Exception, e").

Hopefully other features can appear in 2.x as modules, like bytes and the IO system. In many cases these are just nice features that have nothing to do with 3k, only with advancing Python.

Anyway, that's my take on it, though I'm also unsure about how hard it's going to be to deal with this. Conversion tools have to create source that keeps working in *some* version of Python 2 (even if not Python 2.5), because most of us will be using Python 2 for a long time.
glyf From: glyf Date: September 11th, 2006 09:36 am (UTC) (Link)

Re: Features

None of the things that you just described seem like "features" to me. I already use str/unicode to distinguish between bytes and text. It'd be nice if other libraries did that too, but it hardly seems like a difficult or huge change to emphasize that distinction further. What's the thing that I can do with Python now that I couldn't before? Don't get me wrong, I'm not trying to make a point about turing completeness or anything silly like that, I would certainly accept that certain things can be possible but fundamentally easier with some different syntax. I've been trying to write Java code this week, and while the equivalent of "for x, y in zip(range(3), 'abc')" is certainly possible there, it's a rigamarole that takes several classes, an interface or two, and a dozen declarations to make work right. The same distinction isn't true of, say, the transformation of "," to "as".

Is there some snippet using a py3k feature that might produce an "a-ha"? Again, after slogging through Java I'm less enthusiastic about the absence of "byte-strings"; "no byte strings" means "no byte string literals", which means tedious decoding of common protocol elements into ISO-8859-1 everywhere. It's no fun to type [82, 69, 84, 82] when you really just mean "RETR", or perhaps b"RETR". Again, I'm extrapolating, because Py3k is also proceeding with no clear language specification or direction for implementation; the PEPs are vague and incomplete.
smitty1e From: smitty1e Date: September 11th, 2006 09:19 am (UTC) (Link)
My take, from the occasional glance at the 3k mailing list, is that the rewrite is deliberately not a perl6 über-rewrite so much as a removal of weakness (and I can't google that famous saying, alas).
glyf From: glyf Date: September 11th, 2006 09:38 am (UTC) (Link)
I'm aware of the fact that it's not going to be a complete rewrite. If it were, I wouldn't be worried that we would have to contend with it within the next few years, I'd be comfortable that the project would eventually be scrapped and I could ignore it forever :).
leonardo_m From: leonardo_m Date: September 12th, 2006 01:23 am (UTC) (Link)
I share your pain, because I too have a lot of Python code that I'll have to update for Python 3.0. But you need a balance between legacy support and "cleaning"/updating the language. And we have to remember that there is more than old code, there are new future users too, and they may appreciate a tidier language like Python 3.0 is supposed to be.
7 comments or Leave a comment