Semantic Werks

I previously mentioned my doubts about general purpose information visualization techniques. Too often these seem to make a pretty picture for a conference, where the focus is the novelty of the visualization — e.g., a new way to display a fisheye view — rather than a sober focus on how that picture aids understanding. It is the cognitive support an infoviz offers that is its raison d’etre. It should show you something about the data you didn’t know before, some hidden pattern. But too often revealing that pattern requires knowing about it, a paradox. How do we reveal without knowing what we are looking for?

There’s a newish sub-discipline in computing called visual analytics. From a key resource:

Technologies are needed that will support the application of human judgment to make the best possible use of this information… [we need to] define a long-term research and development (R&D) agenda for visual analytics to address the most pressing needs in R&D to facilitate advanced analytical insight.

But why presuppose visual analysis is the most important/cutting edge thing to do? I speak with the experience of three years working with a visual tool for ontology understanding (analysis, if you will). My experience was that the visual aspect of the tool got in the way of answering the questions. Now, granted, the tool may not have been professionally designed, it had bugs, it used Swing, etc. But I haven’t been convinced by the later pretty pictures from other tools. What we are about is answering questions. I worry that when we come at this from a visualization perspective, we are essentially carrying a giant hammer around looking for nails.

I think the first thing to do is start with good, old-fashioned user centered design. Ask them what they want. Then brainstorm some solutions. Come up with some new interfaces. Iterate, allowing their reactions to the prototype to inform any new questions they might have. Don Norman casts some doubt on the capacity for this process to produce innovative design, but I think that’s fine in this context: we are answering existing questions. I agree that UCD can be held back by what Vicente calls the task-artifact cycle. Nonetheless, it is more important to identify what to look for (particularly given the high failure rate of innovative designs).

The problem is that to be effective, a cognitive aid has to be able to get out of the way. It should be completely unobtrusive. This is, for me, what Apple is so good at doing. You start off complaining about their design choices, but almost always it turns out to be really effective.

Here’s an example I was given once. Every police radio car has a laptop now, and it is here the officer punches in your licence and registration when he or she stops you. It queries the database and returns any information it might have on you — outstanding warrants, prior arrests, who knows. Well, a friend of mine told me that this screen is nearly always filled with useless information. Most of the time it is either empty, as you are not “known to police”, or it comes back with reams of data. If he stops someone he knows is a ‘bad guy’, it will report arrests back many many years. But that isn’t what he wants to know. He wants to know a) can I arrest him for anything outstanding b) what exactly has he been up to (recently). And right now that’s all they can find out. But I can think of many other questions: who has he been arrested with? Has he got any recent weapons offences? When was he stopped before but not arrested? And so on.

The challenge then is to

1. answer the standard questions officers ask of the machine now;
2. then look for other questions they didn’t know they had.

It would really surprise me if a well-tailored SQL report wasn’t sufficient at this point. To paraphrase a famous quote, if the answer to your analysis question is “use my infovis tool” now you have two problems. You still need to answer the original question, but also to learn this tool.

First, must-read ‘contrarian’ posts on infoviz, in my semi-expert opinion:

“Big Fat Graphs” by mc schraefel and David Karger (why graphs are not the best way to visualize relational data).

Information visualization and art (PDF) by Stephen Few (how pretty pictures are not information visualization).

One popular way of understanding what information visualizations ought to support is Ben Shneiderman‘s information-seeking mantra: “Overview first, then zoom-and-filter, then details on demand”. This axiom proposes a fairly rigid set of tasks that a visualization must support. More generally, his mantra is applicable to visual interfaces that aim to be all things to all people.

However, my Masters thesis showed that this set of actions isn’t particularly useful unless you are new to a dataset. For example, ontology editors want to see their area of the model, not the overview. It’s like being a worker on the Toyota assembly line and starting each day walking through everyone else’s work station before getting to yours. There is some use, of course, to seeing what else is happening, but in general, the overview is hard to visualize and not particularly relevant to them. They need “task-specific visualization” as defined in Lloyd Treinish‘s work.

The obvious downside to task-specific visualization is that you need to know a priori what tasks users will be interested in. The traditional way has been to support some form of query and history mechanism, like bookmarks or signposts to indicate relevant views. And the downside to this has been the inability of most users to formulate these queries in the language of the system. SQL, for example, was intended as a natural way to query data, and yet the complexity — essential mathematical complexity — of relation algebra prevents all but the simplest queries.

So how should we best support task-specific information visualization? And is infoviz even useful for end users, or is it condemned to be a pretty marketing picture?