Yes/No/Cancel

…which I found in the blog of a german journalist living in London, writing about what happens if you have a great zest for life and are disabled at the same time.

I don’t want to describe it there. Just watch it (it’s a short movie), it’s really worth it. And think about that next time you are designing a web site.

Did you know that there is such thing as a World Rock Paper Scissors Society which does hold regular tournaments and country championships and even a world championship (where the world champion will be awarded 10,000 $)?

Maybe this way of decision making is strongly underrated.

I believe strongly that teams of software engineers should have good tools which help to manage the development process (as a minimum, bug tracking and version control of source code). We use Subversion and FogBugz, but there are lots of other good tools too. These tools get particularly useful when connected, so that it’s possible to see e.g. which changes were made to the code in order to fix a particular bug.

The usual way of connecting version control and issue tracking is that developers must enter a bug number every time they make a commit to the source code repository. And because most version control systems don’t foresee integration with a bug tracker, this is usually this is just a special string in the commit message (”BUG:12345″). A bit ugly, but works.

A neater way of doing this, for users of  Subclipse or TortoiseSVN, is to set a few magic properties in the base directory of your project. These graphical front-ends detect the presence of those properties and add a separate input box for the bug number to the commit dialog. The number entered there just gets translated into a line in the commit message, so it’s nothing magic, but it helps as a reminder to put in the bug number, and avoids having to remember the syntax for the bug reference (was it “Bug:12345″ or “Bug#12345″?).

Full details of this ‘bugtraq’ convention are described on the “svn commit ./me” blog.

Here’s what it looks like:

Screenshot of bug tracker, subversion and eclipse integration

This page has been offline for the last 24 hours because I messed up the server on which it was running (now I’ve moved it to another one). I officially hate system maintenance…

The server was running Ubuntu Feisty (7.04) which is now getting a bit old and gradually coming towards the end of its support lifetime. An upgrade would be necessary before too long anyway, so I decided to try updating it using the do-release-upgrade tool. My setup is pretty sane, using only standard packages, so I thought the tool would be able to handle the upgrade smoothly.

Unfortunately, no. At some point during the configuration of the new packages, dpkg started dying by segmentation fault, and from there on everything just went downhill. In the end I had a system with dozens of broken packages (and no way of fixing them with dpkg segfaulting), and quite a few daemons dying similarly. Probably some important shared library got replaced with an incompatible version — rubbish! Apache, MySQL and OpenVPN were dead, and I set them up on a new server; fortunately, Bind, Postfix and Dovecot somehow survived, so at least our emails are still getting through, and I could point the domain names to the new server’s IP address. Still, bloody annoying.

Why is server configuration and maintenance such a crap activity? What I really want is a sort of version control system for installed packages and configuration file changes. My wish list:

• The version control should save only modifications to config files (differences from the package maintainers’ versions) so that the same modifications can be merged into the configuration files for a new version of the packages. This also makes the setup more maintainable, because it is easier for one admin to see the configuration changes made by another admin. Elementary software engineering stuff really.
• I want to be able to generate a functionally identical system by taking a barebones install, applying the package installations and config file changes from version control, and copying in the contents of /home and /var/lib. Then I would never ever do an upgrade to a new release of the distribution; instead I would do a fresh install of the new release, configure it like the old one, test it thoroughly to make sure that it works, synchronise the contents, and switch them over in an instant. MUCH less stressful and error-prone.
• It should still be possible to edit the server’s config files like on any other, since all sorts of changes need to be made in day-to-day operation; such changes should be checked into the version control so that they are recorded and documented.

I have already built myself a tool for generating Amazon EC2 images — effectively a few Python and Shell scripts which take a barebones install and configure it completely for a particular application; I keep these scripts in version control so the build process is completely transparent and reproducible. However, if I make any modification to the image by hand, I need to remember to enter the same changes into the scripts, so that they still correctly reflect the build process. Really what I want is that manual step to go away.

I’m thinking that it oughtn’t be too hard to build such a system configuration management framework by putting /etc in a standard version control system, recording changes elsewhere in the filesystem, and making clever wrappers for a few standard maintenance tools such as apt-get (which just remembers somewhere which packages have been installed). Has nobody built something like that yet?

A few years ago, I was discussing the tensions between relativism and religion with a friend. In a vastly simplified nutshell, relativism is an understanding of the world which is founded on the principle that everything we know and perceive is relative to our own person (e.g. we perceive the whole world through our senses, which are known to be fallible) and that there can therefore be no such thing as absolute truth. Think of the artificial reality of The Matrix as an extreme example. (The tension between relativism and religion is a topic I continue to be interested in, but it’s so complex that I’ve come to think that understanding it is more or less a lifetime project. I’ve certainly not even scratched the surface, let alone made up my mind.)

In that context I was told that I had to read a certain article by Sir Hermann Bondi (Physicist, 1919–2005) entitled “Arrogance of certainty”, because it apparently forms the basis of every discussion of the topic. I was given a copy of the article on paper, and I liked it because of its lucid and clear writing style. I kept it, even though the paper got quite crumpled in my bag at some point. 

Recently I was looking for the article again and searched for it on Google. To my great surprise, I couldn’t find any trace of it, let alone the whole text. Therefore I want to re-publish it here so that others may find it online in future. Unfortunately I have no idea where and when it was originally published — all I have is a crumpled photocopy of a photocopy of a newspaper cutting. If you have any further information, please let me know.

I neither fully agree or disagree with this article, but I think it is well worth reading. Without any further comment, here it is.

Hermann Bondi

Arrogance of certainty

I am a non-believer in any revealed religion and a scientist. In my acquaintance with scientists I find both belief and non-belief. I know sufficient numbers of scientists of each persuasion to be willing to classify two statements as both being stupid and palpably untrue prejudices, viz, that a person, being a scientist must accordingly be a believer in a revealed religion, or the opposite statement.

Any thinking person must be struck with awe and wonder on contemplating the mystery and complexity of our universe. We scientists have somewhat enlarged our modest island of understanding that is surrounded by a huge ocean of ignorance. Some feel that there must be an intelligence, an architect of all this grandeur, an architect that may be called God, but without ascribing to this unknown entity any interest in our human affairs or in our prayers. (If I rightly understand, this was Einstein’s view.)

There are also people who believe, as a generalized feeling, that this entity, this God, in some undefined way responds to their trouble and their prayers without claiming that they have any describable or communicable knowledge of this their God. Again, there is revealed religion, the belief that God in some way, different for different religions, revealed himself in some precise communicable manner conveying some absolute truth.

I have no quarrel with the first three stages, but I regard the widespread human tendency to have firm faith in a revealed religion as one of our most negative traits. Indeed, I do not call myself an atheist, but an anti-revelationist. To call myself an atheist would mean denying an entity so differently defined by different people that the denial is meaningless. Some say God is love. I would certainly not wish to deny love. Some say God is nature. I am not so absurd as to deny nature. But the certainty involved in revelation horrifies me, and the historical record of the deeds done in the name of such revelations bears me out.

If one looks at religiosity, the immediate staggering fact is that different people believe firmly and fervently in different and, in many respects, contradictory religions. The very variety of faiths is remarkable, yet each can claim adherents of the highest integrity, sincerity and honesty, utterly convinced of their belief. How anyone can have the arrogance to think that their own belief is right and anybody who thinks differently is wrong passes my comprehension. Surely the overwhelming evidence is that the human mind has the tendency to believe firmly but incorrectly, since at most one of the many competing revealed religions can be right.

Nor am I much impressed by what some regard as threads common to different major religions as regards their theory. What has a non-theistic faith like Buddhism in common with a theistic one like Islam? Why are we to stress likenesses now when people have fought to the death over minute differences in their religions?

There is indeed a common morality among all of us humans, enshrined in the golden rule that one should do to others only as one would have done to oneself. I see this founded in our common humanity, which is why I call myself a Humanist. I see this common morality sometimes supported by religion, sometimes perverted by it. Above all we need to strengthen all that unites us with other humans, where religion so readily divides us. This division by faiths, so often pursued with the utmost cruelty, is what surely we should strive to heal, by relegating religion from the public domain to that of individual belief or non-belief.

What I abhor about revealed religion is its supposed absolute certainty. It is here that I see the real conflict between science and religion. In science we know that our understanding, our theories are only provisional, and liable to be upset by experiment and observation. On this basis, so well described by Karl Popper, science has indeed acquired universality with people of different cultures, ideologies, races etc able to cooperate. Science is so successful in this because it is attuned to the basic human characteristic of fallibility. It is the inhuman certainty a believer feels in revelation that is so obnoxious and harmful and unacceptable as a basis for morality.

Of course we must recognize the great role religion has played in history but need not support it. Being an anti-revelationist is in no way arid. It allows one to enjoy freely all that human genius has produced; it allows one to engage untrammelled in the search that is the real joy of living.

Sir Hermann Bondi, FRS, was Master of Churchill College, Cambridge.

Choosing a web application framework. Aaaaargh, the pain.

Ok, so we’re actually in a pretty lucky situation right now: We have a new, substantial web development project, which we’re designing from scratch. It’s for a feature-rich, complex web application, it has got to scale well, and it has got to be maintainable. Those are the core requirements, and I thought they didn’t sound too demanding really. Between us in the team we have experience in most of today’s common programming languages, so that wasn’t an important point; what we wanted was the web framework which was objectively the best given our requirements.

Most frameworks, it seemed, were essentially suitable only for toy projects. There is a particularly ridiculous number of open-source web frameworks, and most of them don’t appear to be widely used in real-world situations. When I say real world, I don’t mean your average blog about kittens; I mean a site with at least 100 million page views per month or so. And I wanted the framework to be reasonably widely used, so that other people will have found the fatal bugs and already fixed them before we come along.

This left me with three options which seemed to me to be worth taking seriously:

1. Java Platform Enterprise Edition (JEE)
2. Ruby on Rails
3. Microsoft ASP.NET

Of course there are plenty more (Django came into consideration, for example) but judging by the contents of the computer section of my local book shop, those others must be pretty niche. Although ASP.NET looks like a reasonably well-designed platform I had to unfortunately exclude it straight away, because I don’t want the risk of being locked into a Microsoft platform (and I’m not yet sure how reliable Mono is).

So the showdown is between Java Enterprise and Ruby on Rails, and the contenders could hardly differ more in terms of their culture. In a nutshell, Ruby on Rails focuses on fast and agile development, while Java Enterprise focuses on flexibility and integration with enterprise IT. In Ruby on Rails, the common tasks are made very very simple, and if you stick within those common tasks, life as a developer is bliss; in Java Enterprise, even the simplest jobs require you to write ugly XML configuration or auto-generate boilerplate code. Ruby on Rails is what you’ll find on the servers of a cool young Web 2.0 start-up; Java Enterprise is found in the IT nerve-centres of investment banks. In a completely clichéd world view, Ruby on Rails hackers wear Jeans and T-Shirt, have a MacBook and a Linux server, and have a lot of fun; while Java Enterprise software engineers wear suits, have a Windows laptop and a Solaris server, and act very seriously. (I exaggerate.)

The instinctive tendency here is obviously to go with the fun, informal start-up types than with the boring corporate types. But wait, I was trying to make an objective decision, and the discussions about these platforms are already quasi-religious and emotional enough.

I have tried to come up with a list of criteria by which I want to judge these to frameworks. One by one, in no particular order:

• Developer productivity and learning curve for new developers. Ruby on Rails will get you started quicker, there’s not much doubt about that. In Java, I think the worst problem for a newcomer is actually the massive choice of different libraries to use and different ways to do the same thing — there doesn’t appear to be any combination of libraries which is particularly widely used; instead every tutorial, every manual and every book you find will use a different combination of technologies and libraries, which makes it particularly hard to learn the first steps. Because there isn’t just one obviously right way of doing something in Java EE, developers can spend a lot of time sorting out niggling little details, which slows them down and can be demotivating. I believe that if there was a full standard distribution for Java EE (e.g. Icefaces+JSF+Seam+EJB3+JPA+Hibernate+Glassfish, to name just one possible API stack out of thousands of different combinations), people would write a lot more good documentation for that particular stack, and it would become a lot easier for more developers to start using it effectively. With the right tools and good documentation, productivity could potentially be about the same for Rails and Java, but at the moment Java is shooting itself in the foot in this regard.
• Ease of recruiting good and motivated developers. I’ve not yet worked out how the two frameworks compare in this point. There are not many people who know Ruby, but those two do tend to be pretty passionate about it, so there’s an increased chance they will do a good job. Just about every computer science student learns Java these days (except at a few misguided institutions where they still teach C++), so there are plenty of Java developers around, although it’s not clear how many of them are actually really good.
• Standardisation of platform. This point is a combination of the last two points. If a lot of people use a platform, it is likely to be more stable, less buggy, better documented, better designed, faster, more supported in the long term, more interoperable with other systems, and so on. However, you can’t just judge by number of users — so many developers worldwide use plain PHP, the poor souls, and probably don’t even realise that although their LAMP platform is a quasi-standard, it’s complete rubbish for developing non-trivial applications.
• Manageability of a complex code base. This is the point where I personally really appreciate statically typed programming languages (call me old-fashioned). With Java in Eclipse, you can immediately search for the place where something is defined, you can refactor class and method names without too much fear (except when you use Java names in XML files), you get decent code completion, you know at compile-time which methods an object has (rather than calling something speculatively which may or may not be there), and so on. Dynamically typed scripting languages are great for prototyping and writing things quickly, but I find they start getting a bit scary and tricky to handle once you go past a few thousand lines of code.
• Testability and robustness — i.e. “if I change something here, how likely is it that I will break something at the other end of the application?”. Fortunately, both frameworks offer reasonable support for automated regression testing; Ruby on Rails probably a bit more so, because it relies primarily on automated tests (rather than a type system) to ensure things don’t fall apart horribly.
• Scalability, stability and reliability. This is a pretty important one, since any outage immediately gives a very bad impression to your users, and can potentially cost a lot of money. However, it’s hard to get hold of accurate reports on how well the frameworks behave in a harsh production environment, with a large number of concurrent users (hundreds of page views per second) and a large database. I’m inclined to attribute better stability to Java because it’s almost certainly used in corporates for more critical applications than Ruby is. Ruby on Rails, on the other hand, has had some pretty bad press concerning its scalability. The biggest RoR deployment at the moment is apparently Yellow Pages; rather embarrassingly for both them and for Rails, this site has actually been down for at least the last 12 hours as I write this post.
• Internationalisation and Unicode. There are translation features in both frameworks, and they are both ok, although none really strikes me as brilliant. The bigger issue is with Unicode support. I agree fully with Joel Spolsky’s opinions in this matter — there is no such thing as 8-bit plain text. And yet, I could hardly believe my eyes, Ruby treats strings just as an array of bytes, 8 bits per character, unspecified encoding. You can put UTF-8 in them, but then the standard string editing methods will rip them up and make them invalid. For heaven’s sake, guys, we live in the 21st century! Do you not want to sell your software worldwide or what? Ok, there are some UTF-8-safe methods, but you’ve got to remember to call them explicitly. At least Java encourages you to specify an encoding when you convert between byte streams and strings (although in my opinion it still isn’t radical enough — all conversion methods without an explicit charset parameter ought to be marked as deprecated, to make it really clear to the developer what they are doing).
• Toolchain support. Java has been around for a long time, and some pretty powerful tools to support it have evolved in that time (we are using the commercial MyEclipse and it’s doing a good job for us). Ruby on Rails is younger, but is actually quite well supported too due to its active developer community — Aptana is pretty neat, for example.
• Libraries for additional functionality. Since Java is so common, libraries for it are also very common — pretty much whatever you want to do, you can download a jar file to do it for you. With Ruby we’re not that far yet, but more and more libraries are being ported, so this is not likely to be a very limiting factor.
• Ease of integration with back end. We’ve got some algorithmic, number-crunching applications running in the background handling all the really clever technology. These are written in Java, and the web framework has got to be able to communicate with them. If the web tier is also in Java, that is easy; if it’s not, we have to write an explicit interface, but it’s not the end of the world either.
• Integrating with and impression on customers. Who’s your customer? If your application is targeted at general consumers who will use it over the web, they don’t care what framework you use as long as it works well. However, if the software is actually going to be licensed to an enterprise where the IT department have to integrate it with their own systems, then the technology matters. Not so much because of real difficulties (most integrations probably happen on the database level anyway, so the application server is pretty irrelevant) but because they are going to ask you what technology you use; if it’s different from what they use themselves, they will suspect that it’s going to be more effort to integrate, which might become their excuse not to buy from you.
• Long-term support. In 3–5 years’ time, what will have happened to the software? It may be that our application is still running, but will we still get security updates and bugfixes for the framework? Will somebody come along and completely re-write the API so that we in turn have to invest a lot in porting the application to the new API, or else risk running on an unsupported platform? This is crystal ball gazing of course, but to be honest: Java Enterprise is backed by Sun, and if there is one thing which big corporations tend to be good at, it’s keeping things the way they are. With Rails I fear more about the changeability of the framework over time. That said, the move from J2EE 1.4 to JEE 5 (two years ago today, incidentally) was pretty major, so maybe they are both equally changeable.

After all those points I’d like to finish off with a few things which I would rather not have:

• Too much flexibility in combining technologies. Abstract APIs create an illusion of portability — ok, so you think you can replace your PostgreSQL with Oracle by changing one line in an XML configuration file. Really? Not even subtle differences in query semantics? And how often are you going to make such a major change to the system like changing your database server? I don’t think it’s exactly such a common case that it has to be optimised for. It’s not bad to have those APIs, but they don’t add as much value as you might think at first. And as I mentioned above, having too much choice about the components from which you can assemble your system implies a lack of useful documentation and a slower learning curve for new developers.
• Boilerplate and duplicate code. You can create it with a code generation tool, with a bit of luck you can even keep it up-to-date with that tool, but it’s still extremely ugly and makes things harder to maintain. The main advantage of scripting languages in my mind is that things like database model objects are defined at runtime rather than compile-time, eliminating all that generated code. However, you then lose the static type system, so you can’t win on this one.

So where does that leave us? Do you have any further aspects or information which we should consider, or any specific experience with these technologies? Let us know by the comments below.

Seen this morning: PENNY, a german supermarket chain, sells typewriters.

Lunchtime! Vegetables au gratin, not bad after all. In our factory canteen you can only pay cash. We’re nearly the last one. I know of a couple of lunchrooms who do accept only chip cards which you can charge on an automat. This may have some advancements (you don’t have to mess with loose change), but after all, it’s just one step more between me and my food, isn’t it?

Spending the afternoon might become a challenge. My boss cares for the first hour with an unexpected meeting. Meeting is just another word for the collective comparison of PDAs and laptops among my troglodyte colleagues (Me have bigger club. Me leader!). I earn some disbelieving looks and return them with a Yes-I-am-using-paper-and-a-pencil-because-I-have-everything-under-control-anyway expression. — Surprisingly, this works. Even that good that my boss assigns a task to me which was scheduled to someone else in the beginning. I would never have believed that one day ragged paper and an IKEA pencil could become insignia of superiority. Question is: What do they think I want to show that way? That I care for the really important things? That I have everything in mind?

I spend the rest of the day setting up an experiment which is mainly manual work and taking notes. My colleagues are wondering why I’m always coming around instead of using the telephone. This makes me wonder which one is more disturbing: The phone ringing or someone knocking on the door? As for me, the phone causes more stress because it gives you the impression of total urgency: If you don’t pick up the receiver immediately, it will stop ringing and you will miss something important. But once you have picked up, you must start the conversation. If someone comes around, I can tell him to wait for some thirty seconds without him running away again. What do you think?

In case you didn’t know: In Cambridge, bicycles rule the roads. In the the more studenty parts of the city at least. Ok, it’s nothing like what you get in many Asian cities, but by European standards it’s not bad, as demonstrated by this video (embedded below, or follow this link to YouTube):

Of course, I do virtually all my travelling around town by bike — the traffic is congested, the bus service isn’t particularly good, everything is fairly flat and close together, so it’s by far the most sensible option. And during all this cycling to work or visiting customers, it was just a matter of time before I came to think of cycling as a general metaphor for my approach to work. So here goes. Highly tenuous, but maybe mildly amusing.

I must start with a confession: I sense a kind of simple-minded delight when I can overtake cars while on my bike. Which happens fairly regularly in some spots. The cars are all stuck in a queue, but I can put my weight on the pedals, wiggle my way past them, take short cuts via pavements and back alleys. Not only do I get to my destination in a shorter time, and don’t have to pay for parking, I also have more fun in the process.

Then there are the days where it’s cold and rainy. You get out the high-visibility jacket (praying that it’ll save you from getting run over by a lorry), waterproofs, wrap up warm, and get out there on the road nonetheless. Those are the times which put many people off cycling, and they require the greatest level of determination. But, at the risk of sounding clichéd, it’s also invigorating.

The essence of cycling is that you try to get somewhere quickly and efficiently, but completely out of your own strength. This means it’s more satisfying, more flexible and more cool than any other means of transport. Start-up business is just like that. You try to beat the big guys by being quick and agile, by knowing the short-cuts, by avoiding the traffic jams. It’s a sociable experience if you can convince a few friends to get on their bikes and come along too. And who knows, if you take it seriously enough, you might get to cycle in the Tour de France one day.

Working in a corporate, in contrast, is much more like taking the bus. It’s comfortable, but not much quicker than cycling, and it’s always the same route. If you climb the corporate ranks and get into a more senior managerial position, the experience is more like driving a car. Now you have control over some pretty strong forces, but you have to play very carefully by the rules, otherwise you cause accidents.

Driving the car of corporate careers may take you further in terms of distance, but I don’t think it holds the same level of satisfaction as cycling. Think of the Tour de France. You can still be part of it if you’re a car driver — unfortunately you will not be part of the race itself, but your job will be simply to carry the TV cameras. A sideshow, not a main actor.

I think this metaphor is working surprisingly well. Let’s see how far we can push the comparison between different career paths and different means of transport.

• Academic research is like walking. It’s definitely the best way of getting around a new and unknown place, or one with difficult terrain. You get to enjoy lots of nice flowers and other details along the way, but it’s slow — you can’t expect to travel very far.
• Corporate careers are like taking the bus when you start out, and like driving a car when you are more senior. For many people that’s the best way, it’s pretty safe, and maybe a bit unexciting.
• Start-up business is like cycling. It’s hard work, but you get to discover new and exciting places, you get there pretty quickly if it’s not too far, and you get the satisfaction of doing it out of your own strength. Also, when you’re cycling and you have a bit of spare time, it only takes an instant to become a pedestrian, so you have some of the benefits of academic research too.
• When a VC (venture capitalist) invests in a start-up business, I see it as being a bit like attaching a rocket booster to your bicycle, putting on a helmet, lighting the fuse and holding on tight. The kind of thing you might expect to see on Top Gear. With a bit of luck you will leave all the cars behind, find yourself at a garage which will transform your bike into a heavy-duty motorbike, and you can go driving around the nicest places in the world for years to come. With less luck, you will fall off and get a few bruises, but probably you will laugh at the kick you got out of it, and you’ll immediately start searching for another bike and another rocket booster to give it another try.

To conclude, I’d say that these are all good ways of getting from one place to another, and clearly some people will prefer one type over another. But you should know what the options are, and make a conscious decision. The same thing applies with work.

I’ve just worked out how you could make a 3G iPhone yourself, even adding GPS support, and still get away with a lower cost than buying a regular iPhone. The solution:

1. Get an iPod Touch (from £199).
2. Get a Symbian smartphone, such as the N95, with an internet plan (on 3 you’d pay about £34 per month over 18 months for a N95 and a tariff roughly equivalent to O2’s iPhone tariff).
3. Download JoikuSpot and install it on the N95. Use it to create an ad-hoc wireless network, and connect the iPod Touch to that network.
4. Voilà. Total cost is about £811 over 18 months (compared to the iPhone total cost of £899), you get 3G or even HSDPA, and you get a whole additional handset with Nokia’s awesome features.

JoikuSpot is still a bit limited — rather than just routing packets, it proxies HTTP traffic and doesn’t support anything else, so e.g. IMAP isn’t going to work for the time being. I hope that will get fixed soon. I tested JoikuSpot briefly for plain web traffic on my E65 and it seems to be working.

I’m not going to rush out and buy all those things now, I just find this situation curious.