How Python 3 Should Have Worked
As a workaday Python developer, it’s hard to shake the feeling that the Python 2 to 3 transition isn’t working. I get occasional requests to make my libraries work in 3 but it’s far from clear how to and when I try to look it up I find all sorts of conflicting advice, none of which sounds very practical.
Indeed, when you see new 3.x versions rolling off the line and no one using them, it’s hard to shake the feeling that Python might die in this transition. How will we ever make it across the chasm?
It seems to me that in all the talk about Python 3000 being a new, radical, blue-sky vision of the future, we neglected the proven methods of getting there. In the Python 2 era, we had a clear method for adding language changes:
In Python 2.a, support for from __future__ import new_feature was added so you could use the new feature if you explicitly declared you wanted it.
In Python 2.b, support was added by default so you could just use it without the future declaration.
In Python 2.c, warnings begun being issued when you tried to use the old way, explaining you needed to change or your code would stop working.
In Python 2.d, it actually did stop working.
It seems to me this process worked pretty well. And I don’t see why it couldn’t work for the Python 3 transition. This would mean mainly just:
- A Python 2.x release that added support for
from __future__ import python3.
Putting this at the top of a file would declare it to be a Python3 file and allow the interpreter to parse it accordingly. (I realize behind the scenes this would mean a lot of work to merge tr 2 and 3 interpreters, but honestly it would always have been better to have a unified codebase to maintain.)
Then if I wanted my Python 2 program to use some 3 modules, I just need to make sure those modules have the import line at the top. If I want to do a new release of my module that works on Python 3, I just need to declare that it only works in Python 2.x and higher and release a version that’s been run through 2to3 (with the new import statement). If my project is big, I can even port files to 3 one at a time, leaving the rest as 2 until someone gets around to fixing the rest. Most importantly, I can start porting to Puhon 3 without waiting for all my dependencies to do the same, parallelizing what until now has been a rather serial process.
Users know they can safely upgrade to 2.x since it won’t break any existing code. Developers know everyone will eventually upgrade to 2.x so they can drop support for earlier versions. But since 2.x supports code that also runs in 3, they can start writing and releasing code that’s future-compatible as well. Eventually the vast major code will work in 3 and users can upgrade to 3. (2.x will issue warnings to the remaining stragglers.) Finally, we can drop support for 2.x and all live happily having crossed the bridge together.
This isn’t a radical idea. It’s how Python upgrades have always worked. And unless we use it again, I don’t see how we’re ever going to cross this chasm.
You should follow me on twitter here.
March 9, 2012
Hi Aaron,
Been a while, mate. Anyway I think one can accept your points as valid without despairing about “crossing the chasm.” The transition from Python 1.x to 2.x was similarly slow, and I’m pretty sure 1.52 was still more heavily used by the time 2.2 emerged. I think the fact that it takes a while is actually a testament to Python’s success.
—Uche
posted by Uche Ogbuji
on March 10, 2012 #
Some of these thoughts have occurred to me as well recently, except I’m a bit more pessimistic than you. I’ve debated writing a “Python’s Dark Age?” piece. It’s not just py3k, but pypy, and numpy+scipy that have me a bit gloomy with all the fragmentation of focus. In some ways I think Py3K wasn’t a big enough change if they were going to break backwards compatibility. They should have added much more thorough functional programming underpinnings, etc.. Implementation wise, CPython is terrribly slow, especially when compared to Javascript these days, which, if anything, is even more dynamic than Python, and has been approaching C-like speeds with the new JITs. And with cross-compiled languages like Coffeescript, even Python’s syntax is looking clunky in some ways.
PyPy is very promising but it is not numpy, scipy compatible, which are the main reasons I stick around in Python these days. PyPy is also 2.7 compat. I believe, so Python 3K is truly sitting out there all alone. Though I consider myself a Python guru, and use it quite a lot, there are a couple of other languages I would consider switching too for evertyhing if I could: CoffeeScript/JS + node, Lua + LuaJIT, Haskell, Mono C# even… have you seen the awesome new async features being added to C#? Sigh.
posted by Craig
on March 10, 2012 #
Aaron, python will not die out because of 3x and i will explain why. I’m a senior in high school, i started coding 2 years ago on python and everybody suggested i start with 2.7 because there is no real “community” or books for 3x beginners yet. I then read an article dated 1999! about the early adapters of 2x when 1.5 was being phased out. It spoke about new coders or beginners being taught in 2x. How educational institutions, teaching blank slate students will give them a head start by early adapting 2x.guess what? thats exactly what happened, and who uses 1.5.2 now? that being said, 2x was backwards compatible with 1.5.2. The point is though, this year our compsci classes started coding in 3.2.2. a lot of teachers are starting to teach in 3.2.2. the younger guys are going to do the porting for popular 2x modules, its gonna take a little time but it will happen. you will start to see a lot of people in a few years who never used 2x but are pro coders in 3x. plus the problem i had 2 years ago is no longer a problem, there are plenty of great beginners books and on-line tutorials available now for 3x. don’t get left behind buddy, adapt!
posted by Mike
on March 11, 2012 #
I have no doubt that Python 3 will win the day in the end. It is taking a while but it is no Perl 6.
I am a huge fan of Mono as well and the async stuff is going to be great. You cab even use IronPython on Mono and just use C# with async/await where it makes sense. Or you can use F# for key parts. The ability to mix and match languages in a single codebase is one the great strengths of Mono.
posted by Justin
on March 12, 2012 #
You can also send comments by email.
Comments
Hi Aaron,
Been a while, mate. Anyway I think one can accept your points as valid without despairing about “crossing the chasm.” The transition from Python 1.x to 2.x was similarly slow, and I’m pretty sure 1.52 was still more heavily used by the time 2.2 emerged. I think the fact that it takes a while is actually a testament to Python’s success.
—Uche
posted by Uche Ogbuji on March 10, 2012 #
Some of these thoughts have occurred to me as well recently, except I’m a bit more pessimistic than you. I’ve debated writing a “Python’s Dark Age?” piece. It’s not just py3k, but pypy, and numpy+scipy that have me a bit gloomy with all the fragmentation of focus. In some ways I think Py3K wasn’t a big enough change if they were going to break backwards compatibility. They should have added much more thorough functional programming underpinnings, etc.. Implementation wise, CPython is terrribly slow, especially when compared to Javascript these days, which, if anything, is even more dynamic than Python, and has been approaching C-like speeds with the new JITs. And with cross-compiled languages like Coffeescript, even Python’s syntax is looking clunky in some ways.
PyPy is very promising but it is not numpy, scipy compatible, which are the main reasons I stick around in Python these days. PyPy is also 2.7 compat. I believe, so Python 3K is truly sitting out there all alone. Though I consider myself a Python guru, and use it quite a lot, there are a couple of other languages I would consider switching too for evertyhing if I could: CoffeeScript/JS + node, Lua + LuaJIT, Haskell, Mono C# even… have you seen the awesome new async features being added to C#? Sigh.
posted by Craig on March 10, 2012 #
Aaron, python will not die out because of 3x and i will explain why. I’m a senior in high school, i started coding 2 years ago on python and everybody suggested i start with 2.7 because there is no real “community” or books for 3x beginners yet. I then read an article dated 1999! about the early adapters of 2x when 1.5 was being phased out. It spoke about new coders or beginners being taught in 2x. How educational institutions, teaching blank slate students will give them a head start by early adapting 2x.guess what? thats exactly what happened, and who uses 1.5.2 now? that being said, 2x was backwards compatible with 1.5.2. The point is though, this year our compsci classes started coding in 3.2.2. a lot of teachers are starting to teach in 3.2.2. the younger guys are going to do the porting for popular 2x modules, its gonna take a little time but it will happen. you will start to see a lot of people in a few years who never used 2x but are pro coders in 3x. plus the problem i had 2 years ago is no longer a problem, there are plenty of great beginners books and on-line tutorials available now for 3x. don’t get left behind buddy, adapt!
posted by Mike on March 11, 2012 #
I have no doubt that Python 3 will win the day in the end. It is taking a while but it is no Perl 6.
I am a huge fan of Mono as well and the async stuff is going to be great. You cab even use IronPython on Mono and just use C# with async/await where it makes sense. Or you can use F# for key parts. The ability to mix and match languages in a single codebase is one the great strengths of Mono.
posted by Justin on March 12, 2012 #
You can also send comments by email.