How do you user test this?

Most of the blogging I’ve done in the past has been political and thus I really haven’t had an opportunity to try to post much in the way of source code in a blog.  Once I got Nephandus up and running on WordPress, however, I thought I’d have a shot at it and thus posted a short article on my experiences with C#’s serialization quirks.

To illustrate a specific point I included a brief snippet of C# code which WordPress promptly turned into an illegible mess.  The web is a notoriously difficult place to display source code and thus I set off in search of a WordPress plug-in that would allow me to do so without too much thought.  Several days, a dozen plug-ins, and a string of curse words that would make a sailor blush with shame, I have a solution.

There is no shortage of syntax highlighting plug-ins available, but the support for their instantiation is practically non-existent.    Nephandus is running a number of plug-ins, none of which required much more than a few mouse-clicks to install and configure yet this particular task proved more difficult and involved than anything else I’ve done with WordPress.

The more I thought about this frustration the more I realized that this is an ongoing problem in the software development industry.

When developers buid a word processor or an email application they spend (or they should spend) a fair bit of time working out where users will intuitively look for certain basic functionality.  Tools, options, and commands should be where the user expects them to be; that saves time and cuts down on support costs.  We can test and evaluate that design by asking people who are totally unfamiliar with the product to find those sorts of functions and watching what they do.  It’s a system that works well because, familiar with the product or not, everyone has a basic grasp of certain tasks like writing or arithmetic.

But how do you do that kind of testing with an IDE?

The people who use IDEs are, pretty much by definition, super-users.  At the very least we can expect them to have a high level of familiarity, if not with the interface itself than with the thing the interface manipulates – code in a programming language they know.  These users have very specific ideas about where things ought to be which are based, not on intuition but where they were in some other IDE that they learned.  Because the population of knowledgeable coders is small and the number of IDEs they develop on is also small, it’s likely that they’ll bring some of their preconceived notions into user testing and that those notions will persist through the testing process.

Neophyte users aren’t terribly helpful because while they’re learning the interface they’re also learning the language and thus don’t know enough to evaluate the more powerful aspects of a product.

How then do you test the usability of a new interface when that interface’s intended users are a small, niche population often with preconceived, conflicting, and sometimes nearly religious notions of how the interface should look, feel, and act?

That isn’t a problem that WordPress plug-in authors really need to worry too much about – at least not if they’re giving their work away for free (beggars can’t be choosers) – but those of us who make specialty products for high-end niche professionals need to find a way to tackle the problem lest we find ourselves unable to change a 30 year old catastrophe of a design because it’s what everyone knows and expects.