Skip to content


Should you go beyond relational databases?

A few weeks ago I had the pleasure to meet the excellent Ryan Carson, the man behind the Carsonified conferences like Future of Web Apps. We got chatting about the future of databases, and what lies beyond the relational model, and in the end Ryan asked me to write an article for ThinkVitamin, a widely-read blog for web professionals.

Relational databases, such as MySQL, PostgreSQL and various commercial products, have served us well for many years. Lately, however, there has been a lot of discussion on whether the relational model is reaching the end of its life-span, and what may come after it.

Should you care? Which database technology should you be using?

Of course the answer is “it depends”, but that’s not very helpful. Let me ask you a few questions to help you figure out which technology is appropriate to your particular application. Then I can give a few pointers so that you can find out more.

I am pleased that it is now published, so please head over to ThinkVitamin for the full story, and let me know your thoughts!

Posted in software, techie notes, web. Tagged with , .

Customer testimonial: Bid for Wine

I am a bit obsessed with doing an excellent job for our clients and customers. I will bend over backwards and do anything that is required to ensure that everyone is delighted with the outcome of our projects.

As a result it is particularly satisfying when our customers recommend us warmly. I do not like bragging, but on this one occasion please let me quote Lionel Nierop from Bid for Wine — they have been our client for around a year now, and have recently written us a recommendation on LinkedIn:

I have known Martin personally since meeting him at Corpus Christi college in Cambridge in 2003. He was widely known by his peers both as a deeply intelligent and analytical person (skills which saw him regularly coming top of the Computer Science Tripos) and as a great contributor to the life of the college through music and other means.

Our professional relationship began in June 2007 when I joined Martin in setting up his current business, Ept Computing. I left the business in January 2008 for personal reasons.

In March 2008 I put out a large web application project out to tender. This was for the development of an online wine auction site which had to be low maintenance and scalable. There were also a range of detailed requirements relating to tax administration and logistics. The brief was provided to 5 businesses including some major London firms but Ept was the only company to respond in a manner which showed a real understanding of the technical side of the project as well as a broader engagement with the concept as a whole.

The project was successfully delivered on time and on budget despite several major alterations and additions to the brief during the development phase. Martin always responded to these changes in a constructive manner and worked exceptionally hard to make sure he could see their relevance to the project in the round rather than simply as a technical issue to overcome. This was key to the success of the project as it meant we did not have to explicitly detail the minutiae of every change but could rely on Martin to develop the application in a manner which considered the underlying needs of the user. His strong work ethic and personal skills made us feel that Martin was very much a member of our team, rather than simply a contractor and our commercial relationship continues to go forward in a constructive and mutually beneficial manner.

Service Category: IT Consultant; Year first hired: 2008; Top Qualities: Great Results, Personable, Creative

Thanks Lionel, it has been great to work with you, and I look forward to continuing it further!

Posted in business. Tagged with , , , .

Building Go Test It: Fun with Scala and REST APIs

Go Test It

Go Test It logo

Go Test It, the awesome new web testing product I am currently working on, was announced on Monday. I had hinted at it back in December, but now that it is taking shape it was time for me to start spreading the word.

Please see the Go Test It website for details about the product; in summary, it allows web developers and QA teams to very quickly and easily record functional tests on their web application, and plays them back in our own cloud-hosted cross-browser infrastructure. It is going to be the fastest way ever to get started with cross-browser testing, and also provide great tools for running your existing test suites. With Go Test It, there will be no more excuses for not having great coverage of automated functional tests.

On this blog I would like to share from time to time some of the technical challenges we have overcome in the process of building this product.

Today: Simple REST APIs in Scala

The Go Test It system consists of a number of different components using different programming languages, different frameworks, different infrastructures. The main user front-end is a Rails app, but not all tasks are so well suited to Ruby. In particular, the multi-threaded parts of the system which do a lot of the internal coordination are being written in Scala, a very elegant JVM-based language which lends itself well to this sort of thing.

The various components of the system need to communicate, and depending on the type of interaction needed, we’re using a combination of a message queue and REST APIs. There is good Java library support for both, and since Scala can seamlessly use Java libraries, it all fits together very nicely.

Jersey is a really neat Java (and, by extension, Scala) library for writing REST APIs. (It falls into a similar category as Sinatra in the Ruby world.) For example, a resource which returns a HTML document can be written in just a few lines of Scala code:

1
2
3
4
5
6
7
8
package com.example.restapi
import javax.ws.rs._
 
@Path("/hello")
class Hello {
  @GET @Produces(Array("text/html"))
  def doGet = "<html><body><h1>hello jersey/scala world!</h1></body></html>"
}

A resource is defined by a class, annotated with the path(s) under which it can be accessed, and has one or more methods to handle different HTTP verbs (GET, POST, PUT, DELETE), and if you want, also different methods for different content types.

Coda Hale has written up some really nice examples of using Jersey and doing nice clean error handling of input. His examples are in Java, but translating them into Scala is a very simple exercise — more or less a matter of removing the type declarations and the semicolons.

But here comes the tricky bit. Given the beautiful code snippet above, how do you actually build and run the thing? That is something I wrestled with for a while, and here I would like to reveal my solution. You can build it with Maven (for the Ruby guys, it is basically RubyGems and Rake rolled into one, intended for building Java projects with automatic dependency management). The process is a bit scary, but once it works, it is magic — it automatically downloads all the right packages, sets the right classpaths and packages up the results of your build into a war file (web archive).

But the scary bit first. You need to declare your dependencies and build properties to Maven, and the prescribed way of doing that is using the most ugly XML configuration file on the planet. I’m really sorry for the ugliness, but I think it’s worth it. However, the good news is that you only need to set it up once and never touch it again; once your build is working, you can create resources and handle them in Scala to your heart’s content. And for a bonus, see below for a Capistrano recipe which allows you to easily deploy the resulting war file to a Tomcat server, again and again.

First, put the following in a file called pom.xml in the base directory of your project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
< ?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelversion>4.0.0</modelversion>
  <groupid>com.example</groupid>
  <artifactid>restapi</artifactid>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>restapi</name>
 
  <dependencies>
    <!-- Scala standard library -->
    <dependency>
      <groupid>org.scala-lang</groupid>
      <artifactid>scala-library</artifactid>
      <version>2.7.2</version>
    </dependency>
 
    <!-- JUnit for testing -->
    <dependency>
      <groupid>junit</groupid>
      <artifactid>junit</artifactid>
      <version>3.8.2</version>
      <scope>test</scope>
    </dependency>
 
    <!-- Jersey and Servlet (for the REST API) -->
    <dependency>
      <groupid>com.sun.jersey</groupid>
      <artifactid>jersey-server</artifactid>
      <version>1.0.3</version>
    </dependency>
 
    <dependency>
      <groupid>javax.servlet</groupid>
      <artifactid>servlet-api</artifactid>
      <version>2.4</version>
      <scope>provided</scope>
    </dependency>
 
    <dependency>
      <groupid>com.sun.jersey.test.framework</groupid>
      <artifactid>jersey-test-framework</artifactid>
      <version>1.0.3</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
 
  <build>
    <plugins>
      <!-- Compile to Java 1.6 -->
      <plugin>
        <groupid>org.apache.maven.plugins</groupid>
        <artifactid>maven-compiler-plugin</artifactid>
        <configuration>
          <source>1.6</source>
          <target>1.6</target>
        </configuration>
        <executions>
          <execution>
            <phase>compile</phase>
            <goals>
              <goal>compile</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
 
      <!-- Build Scala sources -->
      <plugin>
        <groupid>org.scala-tools</groupid>
        <artifactid>maven-scala-plugin</artifactid>
        <version>2.10</version>
        <executions>
          <execution>
            <id>scala-compile-first</id>
            <phase>process-resources</phase>
            <goals>
              <goal>add-source</goal>
              <goal>compile</goal>
            </goals>
          </execution>
          <execution>
            <id>scala-test-compile</id>
            <phase>process-test-resources</phase>
            <goals>
              <goal>testCompile</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
 
      <!-- Run the application using "mvn glassfish:run" -->
      <plugin>
        <groupid>org.glassfish</groupid>
        <artifactid>maven-glassfish-plugin</artifactid>
      </plugin>
    </plugins>
 
    <pluginmanagement>
      <plugins>
        <plugin>
          <groupid>org.scala-tools</groupid>
          <artifactid>maven-scala-plugin</artifactid>
          <version>2.9.1</version>
        </plugin>
 
        <plugin>
          <groupid>org.apache.maven.plugins</groupid>
          <artifactid>maven-compiler-plugin</artifactid>
          <version>2.0.2</version>
        </plugin>
      </plugins>
    </pluginmanagement>
  </build>
 
  <!-- List of repositories where the various dependencies and plugins can be found -->
  <repositories>
    <repository>
      <id>scala-tools.org</id>
      <name>Scala-tools Maven2 Repository</name>
      <url>http://scala-tools.org/repo-releases</url>
    </repository>
 
    <repository>
      <id>maven2-repository.dev.java.net</id>
      <name>Java.net Repository for Maven</name>
      <url>http://download.java.net/maven/2/</url>
      <layout>default</layout>
    </repository>
 
    <repository>
      <id>glassfish-repository</id>
      <name>Java.net Repository for Glassfish</name>
      <url>http://download.java.net/maven/glassfish</url>
    </repository>
  </repositories>
 
  <pluginrepositories>
    <pluginrepository>
      <id>scala-tools.org</id>
      <name>Scala-tools Maven2 Repository</name>
      <url>http://scala-tools.org/repo-releases</url>
    </pluginrepository>
 
    <pluginrepository>
      <id>maven2-repository.dev.java.net</id>
      <name>Java.net Repository for Maven</name>
      <url>http://download.java.net/maven/2/</url>
      <layout>default</layout>
    </pluginrepository>
  </pluginrepositories>
</project>

That was a terrible mouthful. Urgh. But worry not, the worst is over. Just one other XML file, and this one goes in src/main/webapp/WEB-INF/web.xml (sorry, no syntax highlighting on this one due to a bug in my syntax highlighting plugin):

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
  <servlet>
    <servlet-name>Example.com REST API</servlet-name>
    <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
    <init-param>
      <param-name>com.sun.jersey.config.property.packages</param-name>
      <param-value>com.example.restapi</param-value>
    </init-param>
    <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>Example.com REST API</servlet-name>
    <url-pattern>/*</url-pattern>
  </servlet-mapping>
  <session-config>
    <session-timeout>30</session-timeout>
  </session-config>
</web-app>

And that’s all the boilerplate you need. You can now write the actual code, with Scala source files in src/main/scala and Java source files in src/main/java (if you want to use Java alongside Scala). For example, place the source for the Hello resource above in src/main/scala/com/example/restapi/Hello.scala. Tests go in src/test/scala and src/test/java respectively.

Now go and build it! (You need to have Maven 2.0.9 or newer installed.) The two most useful commands are mvn package, which compiles and packages your project, and places the result in target/restapi-1.0-SNAPSHOT.war, and mvn glassfish:run, which launches an embedded Glassfish application server and serves your new REST API on http://localhost:8080/ for local testing. The example resource which we defined above would then be accessible at http://localhost:8080/hello in your web browser.

If you don’t have the dependencies yet, Maven will go away and download them. This will take a while, but they won’t be re-downloaded once you have them. It’s a bit frightening at first to watch Maven doing its magic, but in my limited experience so far it has always done a good job.

Now for the bonus points. Just building your app locally is not enough; you also need to deploy it to a server and make it accessible to the world. In our case this is Tomcat, but any standard Java web container should do. I wanted to use Capistrano for deployment, because it gets a lot of things right and also fits well into the rest of our system which does have a lot of Ruby in it. There is an existing Capistrano recipe for deploying war files to Tomcat, but it needed a bit of improvement, so my own version is included below.

These deployment instructions assume you’re running a Debian Lenny system, with a restapi user account; for other systems, you’ll need to customise it to suit.

Create the following shell script in /etc/tomcat5.5/symlink-webapp and make it executable:

1
2
3
4
5
6
7
#!/bin/sh
# Allows an unprivileged user to deploy an application to Tomcat. Add a rule for this
# script and /etc/init.d/tomcat to /etc/sudoers. Then to deploy an application, call:
#     sudo /etc/tomcat5.5/symlink-webapp webapp-name /location/of/war/file
NAME=`echo "$1" | sed -e 's/[^a-zA-Z0-9\.-]//g'`
echo "ln -f -s \"$2\" /var/lib/tomcat5.5/webapps/$NAME"
ln -f -s "$2" /var/lib/tomcat5.5/webapps/$NAME

Next, add the following line to /etc/sudoers:

restapi ALL = NOPASSWD: /etc/init.d/tomcat5.5, /etc/tomcat5.5/symlink-webapp

With that set up, here’s the Capfile for you to put in the base directory of your project:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
# This recipe is based on:
# http://github.com/andynu/capistrano-recipes/blob/master/capfile_tomcat
 
load 'deploy'
set :application, "restapi"
 
# DEPLOYMENT SCHEME
set :scm, :none
set :deploy_via, :copy
set :repository do 
  fetch(:deploy_from)
end
 
# LOCAL
set :war_path, "#{File.dirname(__FILE__)}/target/*.war"
 
# TOMCAT SERVERS
role :webserver, "server.example.com"
set :tomcat_home, "/var/lib/tomcat5.5"
set :tomcat_ctrl, "/etc/init.d/tomcat5.5"
set :deploy_to, "/home/restapi/deploy"
 
# USER LOGIN
set :user, "restapi"
ssh_options[:keys] = File.expand_path('~/.ssh/my_private_ssh_key')
set :use_sudo, false
default_run_options[:pty] = true
 
set :deploy_from do 
  dir = "/tmp/prep_#{release_name}"
  system("mkdir -p #{dir}")
  dir
end
 
# simple interactions with the tomcat server
namespace :tomcat do
  desc "start tomcat"
  task :start do
    sudo "#{tomcat_ctrl} start"
  end
 
  desc "stop tomcat"
  task :stop do
    sudo "#{tomcat_ctrl} stop"
  end
 
  desc "stop and start tomcat"
  task :restart do
    tomcat.stop
    tomcat.start
  end
 
  desc "tail :tomcat_home/logs/*.log and logs/catalina.out"
  task :tail do
    stream "tail -f #{tomcat_home}/logs/*.log #{tomcat_home}/logs/catalina.out"
  end
end
 
# Before everything else, build the application with maven and copy
# the war file to a temporary directory.
before 'deploy:update_code' do
  cmd = "mvn clean package"
  puts cmd; system cmd or raise "\"#{cmd}\" failed"
  set :war, Dir[war_path].first
  cmd = "cp #{war} #{deploy_from}"
  puts cmd; system cmd or raise "\"#{cmd}\" failed"
end
 
# Restart tomcat at the end of deployment
namespace :deploy do
  task :restart do
    cmd = "/etc/tomcat5.5/symlink-webapp #{application}.war #{deploy_to}/current/`basename #{war}`"
    puts cmd
    sudo cmd
    tomcat.restart
  end
end
 
# Disable all the default tasks that
# either don't apply, or I haven't made work.
namespace :deploy do
  [ :upload, :cold, :start, :stop, :migrate, :migrations ].each do |default_task|
    desc "[internal] disabled"
    task default_task do
      # disabled
    end
  end
 
  namespace :web do
    [ :disable, :enable ].each do |default_task|
      desc "[internal] disabled"
      task default_task do
        # disabled
      end
    end
  end
 
  namespace :pending do
    [ :default, :diff ].each do |default_task|
      desc "[internal] disabled"
      task default_task do
        # disabled
      end
    end
  end
end

So there we go. For the first time use cap deploy:setup to create the directories on the server, and thereafter always type cap deploy. Those 11 keystrokes are all you need to fetch all the dependencies (no more troubles replicating your build environment on different development machines), build a clean copy of the entire app, ship it off to the server and run it. And your app can be all in beautiful, concise and type-safe Scala. I have survived the XML deluge and I am happy :-)

Posted in business, software, techie notes. Tagged with , , , , , , .

My talk on the Invoicing Gem 0.2

On Monday I did a talk in front of 80 or so of London’s finest Ruby developers at LRUG, to present the new v0.2 release of the Ruby invoicing gem. The main new feature of this release is a Rails generator script which can be invoked with script/generate invoicing_ledger in the root directory of a Rails project. This generates a default set of model objects, a database migration, a controller, views and some routes; enough to get started with using the gem’s features very quickly.

Note that the Invoicing Gem still isn’t Rails-specific — although the generator currently only works with Rails controllers and templates, the core of the gem depends only on ActiveRecord and can be used with framework components of your choice.

To demonstrate just quite how quickly the invoicing gem allows you to add commercial features to your application I decided that it would be a good idea to have no slides, but to spend the entire 25 minutes I was allocated to give a live demo. That is rather scary for a speaker because a million things can go wrong (and indeed several things did), but with a forgiving audience it ended up also being a lot of fun.

Skills Matter, who host the LRUG events, kindly recorded the talk and put it online. I have also embedded the video below:

It’s a bit hard to see exactly what I’m typing in that video, but you can follow the exact steps from the notes I wrote myself for that presentation, and the example application I used, cracktastic, is also on GitHub.

I have the feeling that I still need a lot of practise to improve my public speaking, but it’s not too dreadful either.

Posted in business, event report, techie notes. Tagged with , , , , .

Managing Meta: Thoughts on focus and priorities

One of the great things about being your own boss is that you decide yourself (to a large extent at least) what you do and don’t do. You are of course constrained by your customers’ wishes, your shareholders’ requirements, your contracts, the law etc., but compared to a ‘normal job’ you do have a lot of freedom.

What I find particularly often is that there is a trade-off between three different types of activity:

  • Doing things which are of immediate value now (such as working billable hours for a client). I call this type of activity immediate or short-term. It is obviously useful because it ensures you have food to eat and a house to live in, but too much of it would be to the detriment of the other two activities below.
  • Doing things which are incidental and unavoidable overheads, prerequisites for something more immediate, or work necessary to improve productivity elsewhere. These things are often not particularly interesting in their own right (such as replying to email, doing tax returns or fixing the build system). I call this type of activity meta-activity, in the sense of denoting something of a higher or second-order kind, or operating at a higher level of abstraction to quote the OED. Meta-activities are particularly questionable because it’s often unclear how much time it’s worth putting into them.
  • Doing things with a more long-term or intangible benefit (such as acquiring a new skill or writing a blog post). I call this type of activity strategic; getting the right proportion of strategic work is important — too little and people say that you’re short-sighted and lack vision, too much and people say you’re a dreamer with no sense for reality.

The total amount of time you have is fairly fixed, but you can vary the proportion of time you spend on these various activities. This freedom of choice can be a heavy burden: if you have a range of different things you could be doing, but not enough time for them all, it is solely your own responsibility to choose where to spend your time. Whether you spend five minutes surfing the web or taking a walk outdoors is unlikely to make a big difference, but added up over days and years, your choice of what to do with your time determines almost everything else: which people you know, which skills you acquire and what you are like as a person; and these things in turn determine almost everything else in life.

Generally we leave it up to our intuition and/or external influences (such as someone nagging us) to decide what to do at any given moment in time. But if this decision has such a massive impact on our future, it seems prudent to give it at least a bit of thought and analysis. We should be managing the meta, the strategic and the immediate, to come up with a consciously chosen balance and not just leave it to chance and/or whoever shouts at us loudest.

Being a scientific mind, I like to try to describe something complex in terms of a simpler model which captures the essential features. This may of course turn out to miss certain crucial aspects, but it’s worth experimenting with models, as a thought experiment at least.

Money

Time is money, or so they say, and as such it might be a suitable way of modelling the use of time. Economists will tell you that in essence, money serves three purposes:

  1. a means of measuring value,
  2. a means of storing value,
  3. and a means of transferring value between different people.

Note that this definition has nothing to do with coins or notes; it is a completely abstract concept, but it is also extremely powerful. What I am most interested in here is the aspect of measuring value. What money gives you is a fairly simple way of attaching a quantitative label to something, describing how much it means to someone. It has its limitations, but it’s an interesting first approximation.

When doing freelance/contract work for clients, the monetary value of time is a very tangible concept; you decide how much you think your time is worth per hour, and in order for a client to persuade you to work on their project, they will need to wave at least that amount of cash in front of your nose.

For simplicity’s sake, money in contracts tends to be proportional to time (i.e. the price per unit time is constant, such as a fixed hourly rate). A more accurate model would probably be hyperbolic: if I had absolutely nothing else to do, I’d probably work on something for free or very little, just to keep myself busy; but if I’m already working a 70-hour week, it will take a lot of money to convince me to give up just one of the remaining hours with my girlfriend. As my amount of free time tends to zero, the price of the remaining time tends to infinity.

In a business context, the monetary value of things is often reduced to a dependency on a small number of variables, because there is a need for simple and objective decision-making in order to avoid conflict and to allow contracts to run smoothly. In non-business life, we instinctively take many more things into account. For example, if a friend wants to borrow my bike, I will of course lend it to them for free. That doesn’t mean that my bike has no value to me (in fact, it is rather important to me), but rather that I think that asking for money would damage our friendship, and I value the friendship much more highly than a few pounds.

This opens an interesting can of worms, because you suddenly start thinking about whether you can attach a monetary value to sleep, to time with your family, to a walk in the sunshine, to education, etc. The answer is yes, you probably can, but you would need to factor in many, many different aspects in order to get any sort of vaguely useful figure. In order to value sleep, you would need to determine how much productivity you lose through lack of sleep. In order to value time with friends and family, you would need to place a value on the happiness you derive from being with them, and a negative value on the loneliness caused by their absence. This reminds me of the teenage conversations along the lines of “would you have sex with person X if you were given a million pounds?” which I always hated — it’s a completely hypothetical valuation which I would rather not even think about.

If you had such a formula, you could work out what to do with your time simply by maximising its monetary value over the range of possibilities. But to be honest, you could probably be doing better things with your time than work out how much your sleep is worth. Such as getting sleep, for example. And many people would consider it offensive to place a monetary value label on a friendship or on a holiday with your partner (even if it was a very high value), so it might be wise to steer completely clear of such discussions.

Dependencies

There is another model which captures different aspects of the things you could be doing with your time. Rather than asking how much something is worth, you ask why you are doing something, and build up a chain of reasons leading up to something desirable. In a business context, that desirable thing is generally one of two things: to increase revenue, or to reduce cost. To give a very simple example:

  1. I am spending three hours on the train because it takes that long to travel to a potential customer.
  2. I want to visit that potential customer because my visit will greatly increase the chance that they will buy our product.
  3. I want them to buy our product because that will increase our revenue.

This argument is simplistic, because it doesn’t take any values into account — if the product is a pack of chewing gum that costs 60 pence, it’s probably not worthwhile for the salesperson to go and visit the potential customer in person, even if this does convince them that this particular chewing gum is particularly wonderful. But for an expensive product, or a big reseller deal, it makes perfect sense. Leaving the numbers aside, we can focus on the reasons for doing a particular thing with our time.

The chain of reasons is not often as simple as in the example above. To give another example: I currently follow 66 people on Twitter, and I spend time reading what they write. Why do I do that?

  1. I spend time reading what my friends on Twitter write so that I find out what they are doing, for example which conferences they are attending.
  2. I want to know which conferences are most worth attending so that I can go there myself.
  3. I want to go to worthwhile conferences in order to meet interesting people.
  4. I want to know interesting people because they may one day become my customers or partners, or may refer me to new customers or partners.
  5. I want to build a pool of potential future customers because I think that it is easier to close sales with people who already know and trust me.
  6. I want to close sales because that will increase our revenue.

It is a bit like a proof of a mathematical theorem, in which you try to derive a statement (such as “I spend time reading what my friends on Twitter write”) from a set of basic principles/axioms (such as “increase our revenue”) and a list of logical deduction rules (each line of “because”, “so that”, or “in order to”). In this case I derived the statement from the axiom in a chain of six rules.

There might be more than one chain of logical deductions; in fact, with any interesting argument, there will be lots of different reasons, some more tenuous than others. For example, I can construct a completely different chain of reasoning for the same statement and the same axiom as above:

  1. I spend time reading what my friends on Twitter write because in the chat I can hear hints about which entrepreneurs are doing cool things at the moment.
  2. I want to know which entrepreneurs are doing cool things because they are people who might be suitable co-founders in a future venture.
  3. I want good co-founders in a venture because that will increase our chances of looking good in the eyes of investors.
  4. I want to look good with investors because they can give us money and contacts.
  5. I want money and contacts so that we have the resources to build a great product and market it effectively.
  6. I want to build a great product and market it effectively because that will make lots of people want to buy it.
  7. I want lots of people to buy our product because that will increase our revenue.

As you can see, the more steps there are, and the more debatable the validity of each individual step, the more tenuous the whole argument gets. But it’s also possible to reinforce an argument by adding parallel chains of reasoning. For example, I could build a short-cut across steps 3 to 5: “I want good co-founders in a venture because together we’ll have the skillset to build a great product and market it effectively.” This reduces the number of steps from 7 to 5, and avoids mentioning investors (whether that makes the argument more or less valid is up to debate).

Debating the fundamentals

Of course you may disagree with my choice of axiom above, namely increasing revenue. Didn’t we just say above that it wasn’t all about the money, and that many very important things were in fact incredibly difficult to value in monetary terms?

Absolutely right — the arguments above work in a business context, where the principle of increasing revenue or decreasing cost is indeed fundamental, but there’s more to life than business. The interesting thing with the form of argument above is that I think you can change the set of axioms and rules, and still have a valid way of analysing the world. For example, I think that “being happy” would be a good choice of axiom in a personal context. Then you can make arguments like:

  1. I am taking time off work because I want to go on holiday with my girlfriend.
  2. I want to go on holiday with my girlfriend because the time spent travelling together will strengthen our relationship.
  3. I want a strong and healthy relationship because that makes us both happy.

This is such a short and clear argument that it trumps most others. There is another potential line of argument which says that I don’t want to take time off work, because that leads to a loss of revenue or opportunities, and maybe ultimately (through some argumentative steps of which I’m unsure) to a loss of happiness… however, that other line of argument is less clear and convincing, and so the argument in favour of taking time off work wins.

Managing Meta

This brings me back to the original question of deciding what to do with your time. It’s a choice between immediate activities, meta-activities and strategic activities. And if you know what type of activity each of the various items on your ten-page-long to-do list is then at least you have some mental framework to prioritise and choose between them.

Building chains of argument and reasoning, as in the examples above, is a good exercise and helps to become more conscious and more in control of the things that need doing. And it allows you to recognise what type of activity it is:

  • Immediate activities are easy to spot, because they have very short reasoning chains, i.e. it’s pretty obvious why they need doing.
  • Strategic activities tend to be those where you learn something new, but you want to do it for fun, even if you cannot immediately see the reasoning for its long-term value. I believe that if something is interesting and fun, it will almost certainly turn out to be valuable for you in the long term, because you will find ways of using it. If you try to find a chain of reasons and find yourself thinking “one step reasoning: because it’s fun” then it’s probably strategic.
  • The remaining activities are meta. You have clear logical chains of argument for why those activities are necessary, and the longer the chains, the more meta they are. Twitter for me is partly strategic, partly meta.

Be suspicious of all types of activity: immediate because it can be short-sighted, strategic because it can be far-fetched, meta because maybe the chain of reasoning for its necessity is more tenuous than you thought. Depending on your character, you will probably have a tendency of overdoing certain activities to the detriment of others. But by being aware of your them you can consciously manage and balance your life.

Posted in business. Tagged with , , , , .

Doing a PhD

I have decided to apply to do a PhD in Cambridge.

This might come as a surprise, so please let me explain. It is something which has been tempting me for a long time. I have always loved working independently and getting deep into a project which I find cool, and a PhD (in computer science at least) seemed to me the ultimate manifestation of this independence: three years in which you can learn about and figure out an interesting topic, and invent new ways, with hardly any constraints other than that you’re supposed to write up something vaguely insightful at the end. (I’m sure this is an overly idealised notion of what a PhD entails, but please bear with my dreamworld for now.)

On the other hand, I have started a company and I’ve had an incredibly experience-packed two years so far doing that. I would be a completely different person now if I had gone directly into a PhD after graduating. Running a start-up has made me less risk-averse, more dynamic, more outgoing and confident, more pragmatic, more focussed, and has given me a much better understanding of how the world works.

You might think that returning to university is a cop-out, a return from the harsh winds of a start-up into the safe haven of academia. Let me assure you that this is not the case, for two reasons.

  • Firstly, I will keep my company going on the side. Obviously I won’t be doing it full-time any more, but I am keeping all of my active clients, and I will continue the high level of service they know from me. It won’t be a return to student lifestyle for me; if anything, my focus will get sharper.
  • Secondly, the research proposal I have written is not just any proposal. It is aimed squarely at what I (and many others) believe will be the most influential technologies of the next decade or two: technology which deals with the vast amount of data on the web, filtering, processing and mining that information such that it becomes a source of useful insight. Machine learning and computational linguistics.

We are rapidly moving towards a world where everything which can be digitised and put on the web will be. Blogs, social networking sites, Twitter and many other services increasingly become expressions of a person’s identity. Already now I find that if, for example, I receive an email from somebody I don’t know, often the first thing I do is to look up them up on LinkedIn, find their blog or Twitter username, look up their company or affiliation and find out what they do. This allows me to quickly judge the context of their enquiry, gauge the level at which I should reply, or detect whether I need to be cautious for some reason. If it is somebody I have dealt with before, I have a private database of contact history which helps when I don’t remember details of conversations months or years back. (It’s nothing particularly secret, it’s just an extended memory.)

Identity on the web further manifests itself in social interactions with others. This can be a powerful source of insight: for example, if I don’t know somebody, but I see that they publicly communicate with somebody I already know and trust, I will immediately be more inclined to trust them too. This is not a rigorous decision, but a useful first guess in the absence of other information.

However, gathering the pieces of a person’s identity from across the web is currently a time-consuming manual process.

My own digital identity, for example, is spread all over the interwebs. It manifests itself in my blog (which you are currently reading), my LinkedIn profile, my undergraduate dissertation, my open source projects, my company, my tweets, my Facebook profile, my photos and my taste in music, not to mention the many other fragments scattered about other sites, in the form of press articles about me or my projects, my archived emails to mailing lists, my comments on other people’s blogs, etc. They are all there, and Google has indexed them all (apart from the small number of things behind logins), but at the moment they do not come together to form a coherent whole. They are scraps of data, but without further analysis they don’t mean much.

In a nutshell, my PhD proposal is to gather that publicly available data together and make it useful. For me, that means to map out the graph of connections between different people, and relationships between people and topics. Who is interested in what, and who discusses what topics with which people?

Consider an example to see why this might be useful: say you are new to a particular field of specialism (whatever it may be), and you attend a conference to find out more about it. The programme of the conference is a long list of sessions with names of speakers and titles of talks. Someone who has been in the field for a while will know many of the speakers’ names and will immediately know which sessions will be worth attending and which people to talk to. But a newcomer will have no idea, and has no way to find out other than by spending years getting to know the community. Why can’t you just visualise the relationships between the various speakers and topics, so that you can immediately see who the most influential presenters are and whose interests are closest to your own? Or even discover which attendees of the event would be most worth talking to? At the moment we rely on personal referrals, serendipitous meetings and crude markers (like First Tuesday’s “green for start-up, red for investor, yellow for service provider”); why can’t we have a more direct way of finding the people we should be talking to?

There are two steps to making this work: firstly identifying which two bits of information on the web belong to the same person (even if they are on different websites, using a variant spelling of the name or pseudonym-like username, and without confusing two people who happen to share the same name), and secondly mapping out the relationships between the people and the topics they talk about.

Google’s success rests, amongst other things, on the PageRank algorithm which calculates a ‘quality’ rating for each page on the web. Their core innovation was to realise that links between pages, not just the pages’ content, were the measure which determined how useful a search result would be, and implementing PageRank allowed them to achieve much better search results than the other search engines at the time.

A lot has been said about the next big thing post-Google. I wouldn’t want to make predictions, but let’s put it this way: I would not be surprised if the next core innovation is to realise that individual people, and the connections between people, are even more powerful than pages and links between pages. The marriage of social web and semantic web, to be fully buzzword-compliant.

This is a difficult and multi-faceted problem, which is why I want to take it on within the framework of a PhD rather than try to develop it as a product straight away. There is a lot I need to learn, from the mathematical details of the best machine learning techniques to the linguistic techniques needed to extract structured information from natural language text and small clues on the web.

There is a lot of existing research on which to build. Stephen Clark, my proposed PhD supervisor, is one of the authors of the C&C parser, which is maybe the finest statistical natural language parser out there; also in the Computer Lab’s Natural Language and Information Processing Group, Simone Teufel and others’ work on citation analysis is likely to be relevant. And I hope to collaborate with the lovely people at the Cambridge Engineering department’s Machine Learning group, including Zoubin Ghahramani who is recognised as one of the top researchers worldwide in the machine learning field. Very good reasons to be in Cambridge.

Please note that this is not at all certain yet — I have applied, but I may not get accepted, I may not get funding, and the Board of Graduate Studies may lose/forget my papers. But all going well, this is the general direction in which I’d like to head.

On a final note, it will also be interesting to explore the ethical aspects of identity on the web. I believe that both open sharing of personal information and automated mining of that information will increase massively in the coming years, and exploring the ethical and social consequences, as well as protecting the rights of the individual, should be a part of the research in this area.

PS. My favourite techy buzzword so far is “maximum entropy supertagger” (one of the components of the C&C parser). Just say that out loud. Maximum Entropy Supertagger. Doesn’t it sound awesome? Before my inner eye, there is a sci-fi film of a group of heroes fighting off an alien invasion. The tentacled beasts from outer space are everywhere, but the good guys are just managing to keep them at bay. But then… ominous music in the background… a huge towering construction appears from behind a hill in the distance. Silence falls. Everybody stares at the terrifying thing brought by the aliens. The guy who later will be in charge of single-handedly saving the world turns around, and in a brief close-up shot he says to his colleagues in a perfect Hollywood manner: “Oh my God. They’ve got a Maximum Entropy Supertagger.”

Posted in Uncategorized. Tagged with , , , , , .

OAccounts – Setting your accounts data free

Open standards for data are really important. If you store your data in proprietary formats, you are at the mercy of the software vendor who developed the standard; if they go out of business or hike the prices or you find someone else who does the job much better, well, you’re a bit stuck. If you’re lucky, you might be able to find someone who can import your data, but it probably won’t be perfect.

Think of trying to open Microsoft Word files in any application other than Microsoft Word: you will probably see the text, but any more complex layout will, in all likelihood, be messed up. That’s an example of a proprietary standard. Now think of looking at websites: they work in Internet Explorer, Firefox, Safari, Opera, Chrome, and even more different browsers. There might be some minor glitches, and sometimes the developers had to expend extra effort to make it work everywhere (because of bugs in particular browsers), but by and large, it simply works. Or think of email; no matter which email application you use, it just works. Those are open standards.

I have been doing a fair bit of accounting-related work recently, both in keeping my company’s accounts and in developing the Ruby invoicing gem. And I can tell you, all accounting software packages I have seen have proprietary formats, and getting them to work together is a nightmare. Espen Antonsen wrote today on CloudAve, a widely-read blog on web-based accounting software and cloud computing, about Data Standards for Web Applications:

One example where a common standard would be very beneficial is accounting. Accounting is defined by commonly adopted principles but differ in countries in relation to reports, tax setup’s and such. But at the end of the day the data ends up in journal/transaction entries and account information. All accounting vendors take a different approach to this and importation of data must be designed for to suit the format in general or must be customized for customers. This limits the choice of accounting vendors for someone using a less popular accounting service. It also results in a lock-in for customers using applications which provide less commonly implemented data formats.

But Espen also points out that it’s not just about exporting and importing data, it’s about linking it up continually:

Taking data with you from one service to another is not just about the data in the system you are moving from but also about maintaining links to other systems. I suspect that data values that are linked to external integrated services are overlooked in many data exports.

Linked Data Standards (Image from Ted Berners-Lee's TED talk)

Linked Data Standards (Image from Ted Berners-Lee's TED talk)

Open standards for accounting data

I have started working on an open standard for accounting data called OAccounts, which is an attempt to accomplish exactly this. (The name was probably inspired by OAuth.) With the help of other interested parties I would like to draw up a specification and a reference implementation for a data format and protocol for storing, interchanging and synchronising accounting data between different software packages. Of course it will all be free and open. In particular, I want it to be easier for developers to create applications and tools which create/process transactions and do reporting (my Ruby invoicing gem of course falls into this category).

Many web-based accounting systems have APIs, either RESTful or SOAP, but they are all different, and some are woefully incomplete. If a third-party developer wants to integrate with several different accounting systems, to have the broadest possible customer base, they will have to implement each API separately. And if a user of one accounting system wants to use several third-party extensions, such as payment processing, shopping carts and advanced reporting, they will have to integrate each extension separately. The result is a complete mess:

Integration of accounting systems without OAccounts

Well, implementing APIs is going to be unavoidable if we want to integrate several different applications. But if we’re doing that, we should implement each API once, and have an open standard which provides the common language across all those integrations. This is what OAccounts sets out to achieve:

Integration of accounting systems with OAccounts

Why all of this?

Starting OAccounts for me was primarily motivated by my move from one accounting software package to another. I had been using Sage in my small business for somewhere around 18 months, but I wanted to move away from it, and I wanted to take my data with me. I preferably wanted something web-based with all the Software-as-a-Service advantages.

To my great surprise, I had difficulty finding an online accounting provider who could import my Sage data. From Xero, for example, all I got was “wait until the end of your next accounting period, then enter the latest figures from Sage as opening balance”. No, that is not acceptable; I entered all that data carefully into Sage for a reason, namely that I could go back and investigate individual transactions or examine a detailed billing history in future. I wanted all my data to be portable.

Finally, as I was about to give up and write my own parser for the Sage data files (which looked as though they were not too hard to reverse-engineer), KashFlow started offering a Sage import facility. Thank goodness, I thought. (Dear accounting software start-ups: I think the very first feature you offer should be import from your competitors. Remember, I was trying to give you money.)

Nevertheless, importing was not totally smooth. There were a few bugs at first, which KashFlow was very quick to fix (I’m impressed with the customer service!). Now I’ve got most of the data in KashFlow except for our history of VAT returns, which I somehow need to reconstruct. It has already taken hours, if not days of work to migrate the data, and I wish it was all much simpler.

For me, moving accounting systems was not a completely clear decision. I disliked Sage quite a lot, but I didn’t hate it from the bottom of my heart; on the other hand, I think Kashflow is friendlier to use, but it’s not quite the love of my life either. In a vague situation like that, more sensible companies probably wouldn’t have bothered with the migration and would have just stuck with the status quo.

What I would have ideally liked to do is to run several accounting systems in parallel for a while, and then choose which I liked best after using them each for real tasks, such as completing a tax period. Of course this would mean that a single export-import wouldn’t be sufficient; there would have to be a continuous synchronisation of the systems, the “linked data” which Tim Berners-Lee talked about. Changes I make in one application should be reflected in the other application (provided they both have the capability, but accounting data has a pretty standard structure, so that would be perfectly feasible).

I would like the OAccounts initiative to be a starting point for discussions about how we can make an open standard for storing and interchanging accounting data between different systems. I have a bunch of ideas how this can be done in a safe and scalable way without re-inventing the wheel, and I will write them up in a draft specification over the course of the next few weeks. In the meantime, if you find this interesting or even would like to contribute, please subscribe to my blog and leave a comment below. I appreciate your input.

Posted in business, software, techie notes. Tagged with , , , , , , .

Ruby Invoicing Gem released

I have just made the first public release of a new open source project — the Ruby Invoicing Framework, a bunch of tools and a structure which can be used inside commercial (web) applications to handle billing of customers, tax calculations, and other financial matters. Full announcement and background story here.

Some highlights from the announcement:

After learning the basics of accounting, I still find it dead boring. And that is really the reason why I took the Bid for Wine code in January 2009 and started extracting a general-purpose finance handling framework out of it: I wanted to find a neat representation of the data, one which made both me as a developer happy and also satisfied the accountants; and I wanted to package it up so that I wouldn’t have to think about the financial stuff any more than necessary: I wanted to make it go away, not by ignoring it, but by automating it as much as possible, and by explaining it in a way which doesn’t require you to know any accounting jargon.

What I have now released, the Invoicing gem version 0.1.0, is the combination of all the experience we gathered while developing Bid for Wine with several weeks of my full-time work. Despite the young version number, this is already a pretty solid and stable framework. I waited with the first release until I felt that the core API was well enough thought out that I could minimize the number of backwards-incompatible changes in the upcoming minor versions up to 1.0.

Please help me spread the word: take a look at the invoicing gem website, tag your tweets with #invgem, subscribe to the invoicing gem news feed, give the gem a try, browse the docs and the source, and let me know what you think!

Posted in business, techie notes. Tagged with , , .

CamCow (Cambridge Coworking) Update

Yesterday evening I went to a Refresh Cambridge event — there was an interesting talk by Matt Wood from the Sanger Institute on how they use Scrum in a very agile scientific computing environment. Matt’s slides are online.

Then, before we all went off to the pub, I gave a quick update on our progress on setting up a coworking space in Cambridge under the name of “CamCow”. Here are my slides (on SlideShare):

If you’re interested in looking around the new coworking space or want to find out more, please get in touch.

Posted in Uncategorized. Tagged with , , .

Flickr API security weakness, and thoughts about web APIs in general

I am in the process of uploading my entire digital photo archive to my Flickr account. This is a fairly big job as I have collected over 10,000 digital photos since 2001; prior to uploading they were sorted into a hierarchy of folders with copies on several different hard drives. I tried a few different applications for uploading to Flickr, but none really did what I wanted — I want to:

  • map folder names to Flickr photosets and/or tags,
  • be able to pause and resume the upload process (since it takes days and days to upload so many photos),
  • use the right sorting order,
  • deal sensibly with failures (using a retry schedule, giving up up after a number of unsuccessful attempts).

Eventually I came to the conclusion that I will just have to write my own Flickr uploader (uploadr) script (not yet finished; currently based on Cameron Mallory’s uploadr.py, although maybe I should be using Python Flickr API kit instead) and make the source available so that I (and others with specialist upload requirements) can just customise it as we wish.

Anyway, that’s why I’ve been playing around with the Flickr API. I reckon the API has contributed significantly to Flickr’s success, because it makes it feasible for third parties to provide a huge variety of tools, all of which add value to Flickr as a platform (and being a popular platform for developers is incredibly valuable). And at first I thought it was a really neat API: you don’t need to give out your password (which would put you at risk of someone else storing it insecurely); you can choose to which applications you want to grant access your data, and revoke that permission later; you can even choose between different levels of permission (read-only, read-write without delete, read-write with delete).

Good authentication of access is incredibly important. Imagine somebody managed to break into my account (which contains only harmless photos of friends, family and holidays) and went on to post child pornography pictures. I don’t even want to think about it. If they were technically competent, they would maybe do so from a public wireless network and fake their network card’s MAC address. Then it would be hard for me to prove that it wasn’t me, and that would mean serious trouble. Therefore access to my account must be safe.

Flickr Uploadr is Flickr’s official desktop application for uploading photos. I stopped using it a while ago (since it lacks the features described above), but I imagine it must be quite popular. So today, while playing around with the Flickr API, I wondered how Uploadr authenticated itself. Of course, when you first launch it, you get a page in your web browser on which you need to confirm that you want to let Uploadr upload pictures to your account.

How does Flickr know that a particular file upload is coming from the real Uploadr application? That is ensured by signing each API request with a secret key. The secret key is a 64-bit number, usually written as 16 hex digits. And where is that secret key stored? Naturally, somewhere within the Flickr Uploadr application itself. If you download Uploadr, that secret is somewhere on your own hard drive.

Curious as I am, I searched for hex strings within the application binaries:

cd /Applications/Flickr\ Uploadr.app/
for file in `find . -type f`; do
    strings $file | grep -Ei '[0-9a-f]{16}'
done

In the output you can easily spot the application’s API key (a5c4c07991a15c6b56b88005a76e7774) — a 128-bit number which also uniquely identifies the application, but isn’t secret. (Extra points if you can guess correctly how many times the string ‘0123456789abcdef’ is found. ;-) ) There’s no 64-bit number which looks like it might be the secret. However, with a small amount of experimentation I was also able to find the secret in the binaries. I won’t give it away here, but it should be a simple exercise for anybody who knows regular expressions.

The secret could have easily been hidden better (e.g. by XORing with another binary string), but that’s not the point. You could still find it with a debugger, it might just take a few hours instead of a few minutes. If somebody wants to, they can find the secret. And this applies to any other downloadable application just as much as it does to Uploadr.

I am not the first person to discover the secret key in Flickr Uploadr — Ian McKellar independently discovered it a few weeks ago, using a different method. (Also read the comments below, where Richard and Cal from Flickr have replied.)

Knowing a popular application’s secret doesn’t automatically give you access to the accounts to everybody who has authorised that application, but it does bring you a lot closer. What you would still need to do is to trick users into authorising a user token (a ‘frob’ in Flickr lingo) for that application. For example, this could be done with a fake email appearing to come from Flickr, containing a link to the authorisation page, or by redirection from some other site. On the surface it wouldn’t look much like a typical phishing email: the link would genuinely point to flickr.com and Flickr would simply be asking you whether you want to authorise Flickr Uploadr. Users may pause and wonder for a moment, but if they have previously already authorised Uploadr, there’s hardly a reason why they should smell a rat. (At least that’s what I expect. I have not tried it on other people, and I’m not going to since that would be wrong.)

Alternatively, if you can sniff the traffic from another user using the real Uploadr, you can just steal their previously authenticated user token, since it is transmitted in the clear. However, if you are in a position to sniff traffic, all bets are off anyway (you could simply steal their session cookie).

The question is, of course, how to fix it?

  1. Ian proposed an approach to authorisation in which users explicitly authenticate actions, not applications; any changes made by an application will first be placed in a kind of ‘quarantine’ area, and will only be published if authorised by the user through a trusted channel (the website). In theory I think this is a good idea, but it has some shortcomings: an additional step for the user, making it impractical for fast operations (such as a Twitter update) and harming the user experience; and a significant amount of additional engineering required on the site’s side to implement the ‘quarantine’ area.
  2. A variant of Ian’s idea would be to allow easy undoing of changes made by a particular application during a particular time period. It would require a similar kind of versioning of data. This isn’t so much useful for preventing attacks, but it may be a good answer to buggy third-party applications which might mess up your data. For example, a while ago I tried synchronising contacts from my phone with Highrise using Soocial. It all went horribly wrong, and it would have been very useful if I could have just rolled back the changes inflicted on my data by Soocial.
  3. My first suggestion to fix the Flickr security it is a fairly simple one, and doesn’t require major changes on Flickr’s side: instead of coding the secret key into the application, application developers should require that users apply for their own key, and ask them to copy that key into the application’s configuration. This effectively turns the application key into a user key. It’s not quite the way Flickr intended API keys to be used; applicants for API keys are currently asked to write a paragraph or two of text to justify why they want one, and commercial users may be unable to get a key. However, unless the current frob authentication mechanism is switched off, users remain vulnerable to the same phishing risk as before. The user experience isn’t great either.
  4. If you are designing a new API, I would suggest a simpler authentication mechanism: it should not be possible for applications to request tokens themselves; authentication tokens should be produced only at the explicit request of a user, only via a trusted channel (the website), and explicitly copied and pasted into the application by the user. I think this is acceptable from the user experience perspective: it has a physical counterpart (e.g. giving your friend a key to your house: an explicit transfer of key), it’s familiar from serial numbers, and may make people think for a moment longer about the permissions they are passing on to third parties. I would still allow multiple keys for multiple different applications, to make them revocable.
  5. My final suggestion is more of a comment about security awareness. Many computer users, even the not fully tech literate, are learning to watch out for phishing emails; today’s lesson is that the reality is even more complicated. Even if the link points to the right domain, even if it’s using SSL, and even if all looks right on the page, it still doesn’t meen that it’s safe.

Posted in software, user experience. Tagged with , , , , .