Asking your users, part 2

To get the most out of a user survey, make sure your users can give you answers which are measurable and actionable. This is, in my experience, the key to a good user survey.

Make your survey measurable, so you can quantify who among your users requires what from your documentation. This ensures you get a clear picture of the current state of both, users and documentation. It also means you can repeat the survey with little or no changes in a year or two. To see how your demographics have changed and how your documentation has improved.

Make your answers actionable, so you can connect insights from the survey to specific tasks and their priorities. For example, you might find that marketing pushes for a quick start guide, but even novice users don’t really demand it, because most of them find their way around the product using existing documentation or other means.

Today, I’ll look at questions that can help you serve your users better while keeping you efficient and focused. (This comes on the heels of last week’s post where I looked at obvious, yet unfortunate questions.)

Segment users

Segmenting your users is important to distinguish newbies from power users, private from corporate users or whatever groups your product targets and serves. This can help you to prioritize the results and actions. You can ask users to identify with one or several options of segments. For example, ask for which purpose they mainly use the product (if several), how often or how long.

I recommend to respect the users’ anonymity since I think it increases their trust and honesty in the survey. For transparency’s sake, I suggest to start with these profiling questions, so participants already know what they’ve revealed of themselves before they give their opinions.

Survey effectiveness

This means to gauge whether or not your documentation helps users to get stuff done. Bad scores in this area indicate poor coverage: The documentation doesn’t cover (all) the right topics. Be as specific as you need to with questions such as these:

  • Does the documentation help you achieve your task or a step in that task?
  • Does the documentation offer sufficient background or contexts for the instructions it provides?
  • Is the documentation clear, complete and easy to follow?
  • Does the documentation help you to recover from occurred errors?

Survey efficiency

This means to rate how well and quick your documentation helps users to get stuff done. Poor grades suggest poor usability: Users have a hard to time to find or apply the documentation. To survey efficiency, consider questions such as:

  • Can you find the information you need quickly and easily?
  • Does the documentation you find apply to your task or question?
  • Does the documentation “speak your language”?
  • Can you find related information, such as similar topics, quickly and easily?

How others do it

To get a second opinion or if you find that the effective vs. efficient distinction doesn’t work for you, check out:

Your turn

Which strategies have you used to survey users? Feel free to share any tips for what works and what doesn’t!

Advertisements

Content is a service, not a product

A couple of days ago, an intriguing tweet taught me a lesson about content – and the search for its source taught me another lesson about the benefits of twitter. It all started with this:

Content is a service, not a product. For consumers, less a thing they buy, more an experience.

This tweet by Scott Abel on Feb 23, 2010 made sense to me immediately. That day, I wondered how much content I had actually produced, compared to serving existing content to people who need it: I had converted a legacy document to a new template. A colleague had needed a use case which I told him could be found in the release notes. And some existing documentation needed to be made available in a new channel. So my readers needed me to supply content, not to create more of it.

Then I noticed Scott’s tweet RT’ed a less telegraphic tweet by Aptara from Feb 17, 2010. Attached to it was a link…

… which led to a blog post by NewFiction.com from Feb 9, 2010 that ended in a link to its source…

… which was an article by Kevin Keller in Business Week from Feb 7, 2010, “What Murdoch Still Doesn’t Get About the Internet”: Content isn’t a product anymore, it’s a service. Because for consumers, content is less and less a thing they buy and more a thing they experience. It turns out the sentence paraphrased and quoted…

… a blog post by Andrew Savikas from July 13, 2009, “Content is a Service Business”: … what you’re selling as an artist (or an author, or a publisher for that matter) is not content. What you sell is providing something that the customer/reader/fan wants. … media companies are in the service business, not the content business. An update at the end of the post mentioned…

… a talk by Jim Lichtenberg at O’Reilly’s TOC conference on Feb 8, 2008: Book publishing is moving from bringing physical commodities to market, to offering services that delivers content in a variety of modalities based on consumer choice. Jim mentioned as his inspiration…

… the seminal article by Tim O’Reilly, “Publishing Models for Internet Commerce” from June 19, 1995: The actual content is valuable–but far more valuable [in publishing] is the relationship with the people…

I had read O’Reilly’s article years ago, but I couldn’t remember that point as clearly as Scott Abel made it. The phrasing is, of course, different, but most likely, the timing or the context wasn’t right either.

Then I noticed how the very idea proved itself: Here was content that was available and essentially known to me, like the unused coffee-maker at the back of my pantry. But it took Scott’s service to make it valuable as a new and fresh insight that summed up my experiences of that day.

Certainly, Scott didn’t plan to teach me a lesson with his tweet. But the scattershot distribution is part of twitter’s design and success, so my insight is not an isolated accident, but intended. In my case, twitter was effective because the tweet was applicable and good, not because it was a new idea.

What do you value more in twitter and blogs: What’s fast and new or what fits for you?

And, to come full circle, what’s your experience: Do you feel like you create or curate content?

P.S. To read more about good vs. new, check out Scott Berkun’s first article for Business Week, “Good beats innovative nearly every time“.

Familiar is easy is ‘true’

The success of easy-to-read fonts and familiar sentences may have psychological reasons – and far-reaching consequences for professional writers.

The Boston Globe recently ran a piece by Drake Bennett called “Easy = True“: According to psychologists, people are more likely to perceive something as true when it’s familiar and hence easy to think about. The underlying measure is called “cognitive fluency”. (Now, I’ve never heard of that, and it seems so incomprehensible, I don’t think it’s true… 😉 ). It’s not so much the principle that’s the big news, but rather its effects.

Clear and Easy

For the written word, this means:

… when presenting people with a factual statement, manipulations that make the statement easier to mentally process – even totally nonsubstantive changes like writing it in a cleaner font or making it rhyme or simply repeating it – can alter people’s judgment of the truth of the statement, along with their evaluation of the intelligence of the statement’s author and their confidence in their own judgments and abilities.

Of course, writing that’s clear and easy to understand has long been a standard objective of technical writers. It’s simply more efficient and lets our users get on with their tasks.

But apparently, there are additional effects: Documentation can shape users’ opinion of a product and its manufacturer (my employer, that is). ‘Clear and easy’ documentation is seen as truthful, the product as reliable and the manufacturer as professional. The inverse seems true, too: As the medium is the message, readers struggling with obscure fonts “unwittingly transfer that sense of difficulty onto the topic they’re reading about.”

Confident and Alert

‘Clear and easy’ documentation can also increase user confidence in product which can in turn contribute to a product’s success. (For more on boosting user onfidence, check out Kathy Sierra’s brilliant “Kick Ass Curve” that shows how to overcome the “Suck Threshold”.)

Studies also make a case for texts that are not clear and easy: If you want readers to be alert and attentive and “to prevent them from making silly mistakes, make them work to process the question: make the font hard to read, the cadence awkward, and the wording unfamiliar.” – The example relies on answers to inconsequential test questions, so I’m not ready to apply this to writing obscure warning and danger notes just yet…

What do you think? Is this a new and applicable insight – or merely science catching up with industry practice?

Grace, class and subtlety in a manual

A really nice review of impressive, if idiosyncratic technical writing is David Carkeet’s “The Muse of Mopar”:

I have fallen in love. I have found grace, class, and subtlety, along with an intuitive appreciation for who I am, in a place where I would never have expected to find it. I am speaking, of course, of The New Dodge Caravan Owner’s Manual…

The article originally appeared in the St. Louis magazine. It was reprinted in “The Vocabula Review” of March 2001, where it’s still available for subscribers. The rest of us can find it here.

Do you know any appreciations, serious or ironic, of user manuals? Please leave a comment if you do!