Asking your users, part 2

To get the most out of a user survey, make sure your users can give you answers which are measurable and actionable. This is, in my experience, the key to a good user survey.

Make your survey measurable, so you can quantify who among your users requires what from your documentation. This ensures you get a clear picture of the current state of both, users and documentation. It also means you can repeat the survey with little or no changes in a year or two. To see how your demographics have changed and how your documentation has improved.

Make your answers actionable, so you can connect insights from the survey to specific tasks and their priorities. For example, you might find that marketing pushes for a quick start guide, but even novice users don’t really demand it, because most of them find their way around the product using existing documentation or other means.

Today, I’ll look at questions that can help you serve your users better while keeping you efficient and focused. (This comes on the heels of last week’s post where I looked at obvious, yet unfortunate questions.)

Segment users

Segmenting your users is important to distinguish newbies from power users, private from corporate users or whatever groups your product targets and serves. This can help you to prioritize the results and actions. You can ask users to identify with one or several options of segments. For example, ask for which purpose they mainly use the product (if several), how often or how long.

I recommend to respect the users’ anonymity since I think it increases their trust and honesty in the survey. For transparency’s sake, I suggest to start with these profiling questions, so participants already know what they’ve revealed of themselves before they give their opinions.

Survey effectiveness

This means to gauge whether or not your documentation helps users to get stuff done. Bad scores in this area indicate poor coverage: The documentation doesn’t cover (all) the right topics. Be as specific as you need to with questions such as these:

• Does the documentation help you achieve your task or a step in that task?
• Does the documentation offer sufficient background or contexts for the instructions it provides?
• Is the documentation clear, complete and easy to follow?
• Does the documentation help you to recover from occurred errors?

Survey efficiency

This means to rate how well and quick your documentation helps users to get stuff done. Poor grades suggest poor usability: Users have a hard to time to find or apply the documentation. To survey efficiency, consider questions such as:

• Can you find the information you need quickly and easily?
• Does the documentation you find apply to your task or question?
• Does the documentation “speak your language”?
• Can you find related information, such as similar topics, quickly and easily?

How others do it

To get a second opinion or if you find that the effective vs. efficient distinction doesn’t work for you, check out:

• Ann Wiley’s excellent STC intercom article “Customer Satisfaction Measurement” (via the EServer TC Library)
• IBM came up with a comprehensive “Approach to Quality Technical Information” that centers around nine quality characteristics. I think these traits are valuable in the whole documentation process, from design via writing to reviewing and editing, so this goes beyond user surveys. To learn more about it, check out:
  • The presentation “An Approach to Quality Technical Information: Outlining Nine Quality Characteristics” by Michelle Corbin and Fran DeRespinis, available as PDF from the STC Carolina Chapter’s wiki page on the Tri-Doc 2005 Conference Proceedings
  • Lori Fisher’s article “Nine Quality Characteristics and a Process to Check for Them“
  • The book Developing Quality Technical Information: A Handbook for Writers and Editors by IBM Press

Your turn

Which strategies have you used to survey users? Feel free to share any tips for what works and what doesn’t!

Advertisement