The best Key Performance Indicators (KPIs) in tech comm are aligned to measure the success of your documentation strategy.
That’s some advance insight I got from Rachel Potts who will run a workshop about “Developing KPIs” for tech comm at TCUK in Bristol in a few weeks.
Measuring performance
KPIs are “a type of performance measurement to evaluate success… often success is simply the repeated, periodic achievement of some level of operational goal (e.g. zero defects, 10/10 customer satisfaction, etc.). Accordingly, choosing the right KPIs relies upon a good understanding of what is important to the organization.” (Wikipedia, “Performance indicator“)
But KPIs can be tricky! Says Business Administration professor H. Thomas Johnson: “Perhaps what you measure is what you get. More likely, what you measure is all you get. What you don’t (or can’t) measure is lost.” (Quoted and explained in a Lean Thinker blog post)
KPIs in tech comm
Some KPIs in tech comm are also deceptive. To pick a glaring example, measuring grammatical and spelling errors per page is comparatively easy and will probably help to reduce that figure. But one very fast way to improve this KPI is by changing the page layout, so there’s less text per page. Fewer words and more pages lead to fewer mistakes per page – without correcting a single word. Also, the measure won’t improve documentation that’s out of date or incomplete or incomprehensible.
Rachel advised me: “It depends on strategy and purpose: What’s right for one team is completely wrong for another. Measuring errors on the page is only a valuable KPI if the number of errors on a page relates closely to the purpose of your documentation. If there is a close relationship, then that’s a useful KPI!”
Strategic KPIs
So what would be alternative KPIs, depending on particular tech comm strategies?
If your strategy is to make customer support more cost-effective, you can measure (expensive) support calls against (cheaper, self-service) documentation traffic, while trying to align your documentation topics, so they can effectively answer support questions.
If your strategy is to improve your net promoter score and customer retention, you can measure users’ search terms for documentation, number of clicks and visit time per page, while trying to optimize content for findability and relevance to users’ search terms.
If your strategy is to improve content reuse and topic maintenance, you can measure redundant content to drive down the number of topics that have mixed topic-type content:
- As long as you still have abundant conceptual information in task topics, you probably have redundant content. (Though a couple of sentences for context can be acceptable and helpful!)
- As long as you have window and field help reference information in task or concept topics, you propbably have redundant content.
What do you think? What KPIS are helpful? Which are you using, if any?
Filed under: conferences, managing, metrics, topic-based authoring | Tagged: KPIs, Rachel Potts, TCUK13 |
Kai's Tech Writing Blog 
Interesting. Measuring the efficacy of tech comm is so important and so difficult to do well. We need to pay a lot of attention to KPIs.
A couple of thoughts on redundancy as a KPI.
1. The point of reuse is to *create* redundancy in your content. If content is reused, it appears, redundantly, in more than on place in our content. So reuse is not about reducing redundancy, but about increasing it.
2. The reason for creating redundancy is that no one reads your entire content set. They read only what they need to read (plus any other cruft that the presentation of you content forces on them). The point is for them to find all that they need in one place, or in the smallest and most obvious set of jumps. This is a goal that can be accomplished by redundancy or by linking, but the point, the thing that should be the KPI, is the ease with which any reader can locate the set of content they need.
Reuse numbers are a measure of behind-the-scenes efficiency. That’s fine, but any measurement of behind-the-scenes efficiency has to be balanced against a set of measurement of efficacy for the user.
This is not to say that you can’t ever reduce efficacy for improved efficiency. Sometimes that is a reasonable business decision. But to optimize efficiency without regard to its effects on efficacy can have very bad results.
Thanks for your thoughtful comment, Mark, and sorry for my late response.
Your point is well taken. I was thinking of a scenario where we have a lot of redundant files (which are impossible to maintain by writers), not redundant content (which can well be helpful for users as you’re right to point out).
In the end, I think we should be aiming for the balance you describe where we equally look at efficient documentation efforts AND user efficacy.
[…] Does this make sense? Can tech comm benefit from NPS and improvement initiatives? Or is that a hare-brained idea, and we should really stick to key performance indicators suitable for tech comm? […]
[…] The best KPIs support your Tech Comm strategy […]