Handwriting on the Sky - Fused Silicon and Free Software

April 2008

Username:	Password:
Forgot your password?	Remember Me

Something's wrong in Free Software.

Supposedly I use a Free-as-in-Freedom desktop and operating system. According to the Free Software Foundation, that means I have the "right to use, study, copy, modify, and redistribute" all the software on my computer. That's absolutely fantastic. As a professional programmer, it means I'm not beholden to another company to make my living. As a user, it means that I am not at the mercy of huge corporations, I don't have to pay a tax simply to use my computer. Those rights, especially in combination, are empowering because I can make my computer behave as I need it to, not as some cabal of programmers think I need it to.

Or... am I?

For example, let's take this program I'm using right now, "drivel". As I mentioned previously, I'm a professional programmer so I know my way around. So, let's go through a checklist here. Use? Check. I'm using it. Study? Let's see:

glyph@legion:/usr/bin% file drivel
drivel: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux
2.2.0, dynamically linked (uses shared libs), for GNU/Linux 2.2.0, stripped

Hmm... nope, looks like that's not really going to be easy to study in any meaningful way. Let's see about the vaunted Source Code I keep hearing so much about.

glyph@legion:~/Scratch/Build% apt-get source drivel
Reading package lists... Done
Building dependency tree... Done
Need to get 950kB of source archives.
Get: 1 http://archive.ubuntu.com dapper/universe drivel 2.0.2-5ubuntu1 (dsc) [883B]
Get: 2 http://archive.ubuntu.com dapper/universe drivel 2.0.2-5ubuntu1 (tar) [931kB]
Get: 3 http://archive.ubuntu.com dapper/universe drivel 2.0.2-5ubuntu1 (diff) [18.6kB]
Fetched 950kB in 4s (210kB/s)
dpkg-source: extracting drivel in drivel-2.0.2
dpkg-source: unpacking drivel_2.0.2.orig.tar.gz
dpkg-source: applying ./drivel_2.0.2-5ubuntu1.diff.gz
(...)
glyph@legion:~/Scratch/Build/drivel-2.0.2% ./configure --prefix ~/Scratch/drivel
configure: error: Library requirements (...) not met; consider adjusting the PKG
_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix
so pkg-config can find them.

Yeah, uh, I sure don't need that cabal of programmers to figure out what's going on here...

For the record, I know exactly what I need to do to fix this. I have, in fact, built and modified this exact application on no fewer than five previous occasions. However, unless I am already a Free Software expert, how can I meaningfully "study" or "modify" this software without some kind of documented way to progress from the software I see on my system, some way to stumble my way towards an understanding of what's going on?

Also, this complexity really does explode to the point where it's a problem even for experts. I am aware of how to get individual applications to build and install into alternate locations for testing, but setting up a working environment where I can meaningfully modify my operating system or desktop is an exercise in futility. I know what's involved in setting up a gnome development environment, vaguely, but I'm not sure how I would effectively develop it without at least 2 computers, one for the debugger and one for the main desktop. Surely something like a rectangle with buttons on it that start programs like Gnome Panel should not require a six month training program and a 12-hour build process to get started on modifying.

For a better example of how this might work, let's have a look at a program with similar functionality to Drivel, "gnome-blog-poster". If I open up /usr/bin/gnome-blog-poster, instead of seeing a mass of binary crud, I see some words which vaguely resemble english. "from", "import", "class", "self". I might not know what these words mean, but I can see one phrase from the user interface: "Post Blog Entry". If I open up that file in an editor, and modify the words between the quotes, then run the program again... wow! The program is different now!

Knowing to look at a file in /usr/bin/ and launch a program like "gedit" on it is hardly a seamless, intuitive experience (nor is editing source code) but experiences like the one I just described really are important. Free software tools rarely realize the potential that they represent to users; at best, it's just like Windows but cheap; at worst, it's like Windows, but missing all the useful scripting tools like VBA.

Let me indulge myself for a moment in a metaphor. Supposedly this is to clarify the point I'm trying to make clearer, but really I just love metaphors.

Imagine if car manufacturers started welding car-hoods shut and legally prohibiting opening them. This could be a serious problem; they could charge drivers obscene rates for basic service like oil changes. In response - the Open Hood movement; a massive undertaking, costing billions of hours of volunteer labor, heroically laboring against the oppressive regime which seeks to prevent regular folks from fixing their cars when they break, or choosing their own mechanic.

After two decades of work and dozens of half-finished mopeds, go-karts and engine parts, the first fully Open Hood car is released. To the automotive industry's collective surprise, It's dramatically cheaper and gets better gas mileage than the economy cars from each manufacturer.

However, beleaguered consumers are in for a surprise. Joe Average buys himself an Open Hood car, and after a few thousand miles, he decides he's really going to take advantage of the spirit of the movement which created it, saving himself a few bucks in the process, and change his oil himself. He gets his dipstick, and some motor oil, and expectantly pops open the hood to find...

... a solid block of fused silicon.

Now, to an automotive engineer in this metaphorical universe (which has metaphorical physics - a lot more fun and easier to get an A in than actual physics) this may make perfect sense. In fact, it might be obvious. "Avoiding moving parts improves aerodynamics", "exposing the driver to complex parts wouldn't be user-friendly". Nevertheless, the movement's promise is abandoned, and it instead simply a way to get cheaper cars, which can still only be serviced at the dealer. It still decreases the cost of service, because now any licensed dealer can service a car, as long as they have the $10M machine required to safely and temporarily crack the fused shell around the engine.

Wasn't that trip into metaphor-land fun? If it was hard to follow, I'll break it down.

C is the fused silicon of software. There are technical reasons to use it in free software: it's fast, it's ubiquitous, it's UNIX's history, it's what all the infrastructure is written in. And sure, in some places, like the kernel, that makes sense. In today's real cars, there's a big difference between changing your oil and actually trying to fully disassemble the combustion engine. You need a lot of tools for the latter.

The programs on a Linux system aren't themselves objects that you can study and modify - you have to take the authors' word for it that the code you're running is the same code they're running. Sure, you could use a distribution like Gentoo, but that just moves the burden of the build process onto your machine (and I would estimate 95% of the folks who use gentoo don't understand the gibberish that scrolls by when they emerge something). The final products are still opaque lumps with no obvious way to modify them; the intermediary process is not designed to help the user. Finally, even if it were, the multi-hour feedback loop necessary to see the impact of anything modified in GNOME (and the absolutely disastrous impact if you get anything wrong) is going to be a turn-off.

When I was growing up, as young as 8 years old, I would change things around on my computer just to see what happened. I modified arexx scripts, APL programs, HyperCard stacks, batch files, anything I could get my hands on. Then I ran them. Sometimes they exploded violently. Sometimes not.

It might be easy to dismiss this as simple nostalgia, but it's only sometimes that these sorts of experiments lead to children learning to program. Other times, they lead to more "serious" results like improvements in office efficiency. Writing scripts with Visual Basic for Applications is a very similar experience (although markedly less positive or fun) than messing around with working HyperCard stacks.

Lots of people write programs to teach people how to program, and write "scriptable" applications with absolutely no functionality implemented as scripts. That might teach some people to program, or get some tasks automated, but in general people can learn faster, and get things done more effectively, by modifying full, working examples that are personally relevant to them and that they can see some immediate feedback if they modify.

I would like to close with two different points for two different audiences.

Python developers, especially core python developers: remember CP4E? That was a great dream. It saddens me that most of the work on Python these days is adding syntax and libraries to the core language for web programmers. Sure, that's a niche where it's useful today, but what about the future? What about the desktop users of tomorrow? The only real way to make Python into a language that's useful for everyone is to evangelize the hell out of it so that everyone has at least one application they use on a daily basis that's written in python, and methodically hammered into consistent and documented shape so that users can approach it at an odd angle. Novices never show up in the orderly way you expect. Let's stop asking what the people who are using Python today need: we are already using Python, clearly we have what we need. What about the people who aren't using python? What do they need? How can we get Python to be the lingua franca of end-user applications?

Free software C programmers - most especially desktop applications programmers: holy crap, what's wrong with you. It's 2006, people, what are you doing. You will get your applications done in maybe 1/10 as much time if you use a dynamic language. Don't make your applications "scriptable", let your users get right into the guts and change whatever they want. Your application will be fast enough, trust me. Rhythmbox, a hulking monster of C++ code, is visibly sluggish and painful to use on my computer; Quod Libet, a similar application written in Python, is snappy and has a lot more features.

Fine, you hate whitespace, you don't use newlines in your code, you were bitten by a snake as a small child, you hate British comedians, whatever. It doesn't have to be Python. As long as there is an accepted convention in the language for re-compiling at runtime and including the source with the application, some users will figure it out.

(A brief aside to the Mono guys: even though compilation is a required step, you're really close to this. Python compiles everything to bytecode before executing it, after all. Can you just tweak the deployment conventions somehow so you can easily run from a smattering of .cs files on the filesystem, and package things that way?)

There are a ton of options available today. Use ruby, use lua, use tcl or pike or csch or guile or clisp or javascript or perl or icon or sbcl or ... you know, whatever. Look around and find something you like.

(But don't use PHP if you can avoid it.)

Current Mood: thoughtful

	From: nerdbeard	Date: April 18th, 2006 11:02 am (UTC)		(Link)
	Sqeak

Squeak is freakin' awesome from an educational/empowerment point of view. You can examine and modify anything, be it application or OS, even while its running. It is, however, even more oddball than Python. Not so much the language (though it's not like you see smalltalk every day) or the size of the userbase, but the very environment. Squeak mandates a standard set of virtual hardware. Every Squeak user has essentially the same PC. (Their own instance of that PC, but projects like Crochet seem to aim to make it one BIG universal PC.) It's a nice PC, sure, it's got audio, 3D graphics, colour-coded mouse buttins, network access, etc. But it's a wee bit to get used to. If you're not used to anything anyway, no problem! If nothing else, it's a lot of fun, which I think is the most important feature of a language.

	From: glyf	Date: April 18th, 2006 01:33 pm (UTC)		(Link)
	Re: Sqeak

Squeak is great, except, its isolationist design makes it immediately revolting to too many people. Not only is it nigh-impossible to interface with any programs on your system unless you already know C, (and I will bet you a dollar none of the programs you actually use are written in Squeak) it also looks deliberately alien, so people interested in business automation or kids wanting to make applications for their friends will avoid it out of silly aesthetic concerns.

I have big hopes for Croquet Project (not "crochet"), because at least that has a familiar metaphor: it's moving towards looking like an online game, which is at least a familiar mimetic space to some people. If anyone ever manages to make an interactive environment with it that people want to actually play in, rather than just refer to as a promising research project, it might deliver on some of the promise I describe here.

	From: piman	Date: April 21st, 2006 01:26 am (UTC)		(Link)
	Re: Sqeak

Squeak's license is unfortunately also not free; it's basically a Java trap, but worse because it's even more tied to implementation rather than a standardized platform.

From: jdahlin	Date: April 18th, 2006 02:49 pm (UTC)		(Link)
apt-get

I must point out that you're first point is irrelevant, it would have failed even if the application was written in python.
The command you are looking for is: apt-get build-dep drivel, which will install all required build dependencies for a specific package.
Secondly rhythmbox is written in C, relatively minor.
Thirdly, more importantly the interface of quod libet is absolutely horrible, one of the worst one I ever seen. User interfaces are not about features, they're about usefulness.

But I could not agree more with you. Desktop applications should be written in a high level language. If that language is Java, C#, Ruby or Python is less important. But stay away from C/C++.

	From: glyf	Date: April 18th, 2006 05:18 pm (UTC)		(Link)
	Re: apt-get

As I said in the article,

For the record, I know exactly what I need to do to fix this. I have, in fact, built and modified this exact application on no fewer than five previous occasions.

The point I was making was that it is possible to have a little knowledge of how to get source code, configure and compile it, and still be missing a vast sea of lore necessary to actually produce a working copy. Once you've gotten past build-dep, what about --prefix? How do you deal with the fact that certain configure scripts are buggy and don't honor installation locations (or require installation in the same location as some pile of infrastructure like gimp or openoffice)? The list goes on and on.

As far as Rhythmbox, I inferred its implementation language using this admittedly unscientific technique:

glyph@legion:~% ldd /usr/bin/rhythmbox | grep c++
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb6eb6000)

Something in there must be written in c++ but I guess it's not the whole application.

From: paddy3118	Date: April 19th, 2006 11:17 pm (UTC)		(Link)
Open-source Car analogy.

Don't think you got the analogy correct. An open-sourced car would allow the Joe Blogs garage mechanics to, for example, study the engine management systems full schematics and software and tweak it for their customers without fear of lawyers from Global Car Corp. shutting them down. Not every user will have the expertise to teak the engine management system as it is a skilled job, but some users would wish that the engine management system came with an easier interface that at least allowed some tinkering with minimal effort ;-)

- pad.

	From: glyf	Date: April 24th, 2006 03:18 am (UTC)		(Link)
	Re: Open-source Car analogy.

You're saying I didn't get the analogy correct because I didn't cite every possible advantage of open source in the "open hood" car? I think you're missing the point.

I am trying to draw attention to the emotional experience of open source. Once you understand what's going on, it can be very powerful. However, for most users this epiphany is delayed, or never happens, because they do not practically have enough time to learn how to get at the code, even if they do have enough to learn how to program.

If you're interested in getting laypeople to better understand the real value of free software, then it's an important thing to keep in mind. If you're happy the way things are, then never mind! :)

From: paddy3118	Date: April 24th, 2006 06:45 am (UTC)		(Link)
Re: Open-source Car analogy.

You're saying I didn't get the analogy correct because I didn't cite every possible advantage of open source in the "open hood" car? I think you're missing the point.

No, I am saying that some things are easy, some things are hard. You can sit and wait for someone to make those hard things easier, or you can train yourself/ go on courses.

I, (more privately), complain about the high price of getting a car fixed in the UK as well as the variable quality level. I haven't however done a thing about learning about a cars innards. If a garage wiped my old air filter, put it back, then charged me for a new one, then they would get away with it.

From: mattcampbell80	Date: September 22nd, 2006 03:22 am (UTC)		(Link)
How about a Gaim replacement in Python using Twisted?

One obvious category of desktop app that many people use dailiy is an instant messenger. So one way to put Python to use on a lot of desktops would be to write a multi-protocol instant messenger, with functionality and UI similar to Gaim, in Python using Twisted? Make it available for both GNU/Linux and Windows, and give it enough bells and whistles that non-geeks will be interested (especially kids/teens who might want to get their feet wet with programming). I'd suggest wxWidgets as the GUI toolkit, since it has completely native look and feel on all major desktop platforms. I know about Instance Messenger; has any work been done on that lately?

It also occurs to me that open-source Python apps made available for Windows which are intended to be easily studied and modified shouldn't be packaged using py2exe, at least in its current form, since it gets rid of the source, thus defeating the purpose.

Any thoughts?

	From: glyf	Date: September 22nd, 2006 02:47 pm (UTC)		(Link)
	Re: How about a Gaim replacement in Python using Twisted?

One obvious category of desktop app that many people use dailiy is an instant messenger. So one way to put Python to use on a lot of desktops would be to write a multi-protocol instant messenger, with functionality and UI similar to Gaim, in Python using Twisted?

YES.

I've wanted this for years. I wrote it, even, twice :). I just don't have time to maintain it. Then, Moe Aboulkheir wrote it again - there's a project called "Raygun" in the Divmod repository somewhere.

Make it available for both GNU/Linux and Windows, and give it enough bells and whistles that non-geeks will be interested (especially kids/teens who might want to get their feet wet with programming). I'd suggest wxWidgets as the GUI toolkit, since it has completely native look and feel on all major desktop platforms.

I agree on all points except one: use GTK rather than WX. WX is slow, full of bugs, interacts poorly with Twisted, and is considerably harder to use interactively than GTK. GTK feels better on Linux, is faster on Windows, and even looks native if it's set up right.

I know about Instance Messenger; has any work been done on that lately?

Nope. Nobody else ever had the time or inclination when so much else needed to be fixed in Twisted (just have a look at our tracker). If you wanted to take up that mantle though, I'm sure you'd find lots of support.

It also occurs to me that open-source Python apps made available for Windows which are intended to be easily studied and modified shouldn't be packaged using py2exe, at least in its current form, since it gets rid of the source, thus defeating the purpose.

Agreed.

From: mattcampbell80	Date: September 22nd, 2006 03:26 pm (UTC)		(Link)
Re: How about a Gaim replacement in Python using Twisted?

I suggested wx over GTK for one reason: accessibility on Windows. While GTK is quite well known for its claimed accessibility under Unix as part of GNOME, accessibility on Windows has apparently been neglected. GTK widgets aren't exposed to assistive technology through any API that I'm aware of -- certainly not through the standard accessibility API on Windows, Microsoft Active Accessibility. So GTK apps are quite useless to blind Windows users right now. On the other hand, I know of one wxPython-based app that quite a few blind people are using on Windows right now, the Juice podcast receiver (formerly iPodder). Of course, this GTK accessibility problem can be solved, but I don't know of anyone who has the time and inclination to do it, so I figured it was better to suggest wx.

	From: glyf	Date: September 22nd, 2006 03:41 pm (UTC)		(Link)
	Re: How about a Gaim replacement in Python using Twisted?

Hmm. That's unfortunate. Unfortunately I know almost nothing about accessibility and suspect that any GUI application I would write would be completely useless to blind users regardless :-(.

However, there also exists no GTK port for the Mac, so at the very least you'd need to write a separate frontend for that as well. Applications which are written directly to the platform APIs generally behave better anyway, so an architecture which permits multiple frontends is a must.

From: mattcampbell80	Date: September 22nd, 2006 04:02 pm (UTC)		(Link)
Re: How about a Gaim replacement in Python using Twisted?

You said, "Unfortunately I know almost nothing about accessibility and suspect that any GUI application I would write would be completely useless to blind users regardless :-("

If the GUI toolkit does its job, it's usually not that hard. Here's my three-point summary for app developers:

1. Use stock widgets as much as possible. If you must implement your own widget, you'll need to implement the toolkit's accessibility API.

2. Make sure everything can be done with the keyboard.

3. Make sure all images that aren't just decoration have text equivalents. For example, never use an icon for a button without also providing a text label.

Of course, this only works if the GUI toolkit makes itself accessible, which GTK doesn't on Windows.

Name: Glyph Lefkowitz