Lee Romero

Archive for the ‘Knowledge Management’ Category

In my last post, I wrote about a particular process for capturing “knowledge nuggets” from a community’s on-going discussions and toward the end of that write up, I described some ideas for the motivation for members to be involved in this knowledge capture process and how it might translate to an enterprise. All of the ideas I wrote about were pretty general and as I considered it, it occurred to me that another topic is – what are the kinds of specific objectives an employee could be given that would (hopefully) increase knowledge sharing in an enterprise? What can a manager (or, more generally, a company) do to give employees an incentive to share knowledge?

Instead of approaching this from the perspective of what motivates participants, I am going to write about some concrete ideas that can be used to measure how much knowledge sharing is going on in your organization. Ultimately, a company needs to build into its culture and values an expectation of knowledge sharing and management in order to have a long-lasting impact. I would think of the more tactical and concrete ideas here as a way to bootstrap an organization into the mindset of knowledge sharing.

A few caveats: First – Given that these are concrete and measurable, they can be “gamed” like anything else that can be measured. I’ve always thought measures like this need to be part of an overall discussion between a manager and an employee about what the employee is doing to share knowledge and not (necessarily) used as absolute truth.

Second – A knowledge sharing culture is much more than numbers – it’s a set of expectations that employees hold of themselves and others; it’s a set of norms that people follow. That being said, I do believe that it is possible to use some aspects of concrete numbers to understand impacts of knowledge management initiatives and to understand how much the expectations and norms are “taking hold” in the culture of your organization. Said another way – measurement is not the goal but if you can not measure something, how do you know its value?

Third – I, again, need to reference the excellent guide, “How to use KPIs in Knowledge Management” by Patrick Lambe. He provides a very exhaustive list of things to measure, but his guide is primarily written as ways to measure the KM program. Here I am trying to personalize it down to an individual employee and setting that employee’s objectives related to knowledge sharing.

In the rest of this post, I’ll make the assumption that your organization has a performance management program and that that program includes the definition for employees of objectives they need to complete during a specific time period. The ideas below are applicable in that context.

• Community membership – Assuming your community program has a way to track community membership, being a member of relative communities can be a simple objective to accomplish.
• Community activity – Assuming you have tools to track activity by members of communities, this can give you a way to set objectives related to being active within a community (which I think is much more valuable than simply being a member). It’s hard to set specific objectives for this type of thing, but the objective could simply be – “Be an active member of relevant communities”. Some examples
  • If your communities use mailing lists, you can measure posts to community mailing lists.
  • If your communities use an collaboration tool, such as a wiki or blog or perhaps shared spaces, measure contributions to those tools.
  • If your communities manage community-based projects, measure involvement in those projects – tasks,deliverables, etc.
  • Assuming your communities hold events (in-person meetings, webcasts, etc.), measure participation in those events.
• Contribution in a corporate knowledge base – An obvious suggestion. Assuming your organization has a knowledge base (perhaps multiples?), you can set expectations for your employee’s contributions to these.
  • Measure contributions to a document management system. More specifically, measure usage of contributions as well.
  • If your organization provides product support of any sort, measure contributions to your product support knowledge base
  • If you have a corporate wiki, measure contributions to the corporate wiki
  • If you have a corporate blog, measure posts and comments on the corporate blog
  • Measure publications to the corporate intranet
  • In your services organization (if you have one), measure contributions of deliverables to your clients. Especially ones of high re-use value.
  • Measure relevance or currency of previously contributed content – Does an employee keep their contributions up to date?
• A much different aspect of a knowledge sharing culture is to also capture a when employees look for knowledge contributed by others – that is, the focus can not simply be on how much output an employee generates but also on how effective an employee is in re-using the knowledge of others.
  • This one is harder for me to get my head around because, as hard as it can be to assign any credible value to the measurements listed above, it’s harder to measure the value someone gets out of received knowledge.
  • Some ideas…
  • Include a specific objective related to receiving formalized training – while a KM program might focus on less formal ways to share knowledge, there’s nothing wrong with this simple idea.
  • If your knowledge management tools support it, measure usage by each employee of knowledge assets – do they download relevant documents? Read relevant wiki articles or blog posts?
  • Measure individual usage of search tools – at least get an indication of when an employee first looks for assets instead of re-inventing the wheel.

Not all of these will apply to all employees and some employees may not have any specific, measurable knowledge sharing objectives (though that seems hard to imagine regardless of the job). An organization should look at what they want to accomplish, what their tool set will support (or what they’re willing to enhance to get their tool set to support what they want) and then be specific with each employee. This is meant only as a set of ideas or suggestions to consider in making knowledge sharing an explicit, concrete and measurable activity for your employees.

Rolling Up Objectives

Given some concrete objectives to measure employees with, it seems relatively simply to roll those objectives up to management to measure (and set expectations for up front) knowledge sharing by a team of employees, not just individual employees. On the other hand, a forward-thinking organization will define group-level objectives which can be cascaded down to individual employees.

Given either of these approaches, a manager (or director, VP, etc.) may then have both an organizational level objective and their own individual objectives related to knowledge sharing.

Knowledge Sharing Index

Lastly – while I’ve never explored this, several years ago, a vice president at my company asked for a single index of knowledge sharing. I would make the analogy of some like a stock index – a mathematical combination of measuring different aspects of knowledge sharing within the company. A single number that somehow denotes how much knowledge sharing is going on.

I don’t seriously think this could be meaningful but it’s an interesting idea to explore. Here are some definitions I’ll use to do so:

• You would need to identify your set of knowledge sharing activities to measure – Call these A₁, … , A_n. Note that these measurements do not need to really measure “activity”. Some might measure, say, the number of members in your communities at a particular time or the number of users of a particular knowledge base during a time period.
• Define how you measure knowledge sharing for A₁, … , A_n – for a given time t, the measurement of activity A_i is M_t,i
• You then need to define a starting point for measurement – perhaps a specific date (or week or month or whatever is appropriate) whose level of activity represents the baseline for measurement. Call these B₁, …, B_n – basically, B_i is M_0,i
• Assuming you have multiple types of activity to measure, you need to assign a weight to each type of activity that is measured – how much impact does change in each type of activity have on the overall measurement? Call these W₁, …. W_n.

Given the above, you could imagine the “knowledge sharing index” at any moment in time could be computed as (for – I don’t know how to make this look like a “real” formula!):

Knowledge index at time t = Sum (i=1…N) of W_i * ( M_t,i / B_i )

A specific example:

1. Let’s say you have three sources of “knowledge sharing” – a corporate wiki, a mailing list server and a corporate knowledge base
2. For the wiki, you’ll measure total edits every week, for the list server, you’ll measure total posts to all mailing lists on it and for the knowledge base, you’ll measure contributions and downloads (as two measures).
3. In terms of weights, you want to give the mailing lists the least weight, the wiki an intermediate weight and the combined knowledge base the most weight. Let’s say the weights are 15 for the mailing lists, 25 for the wiki, 25 for the downloads from the knowledge base and 35 for contributions to the knowledge base. (So the weights total to 100!)
4. Your baseline for future measurement is 200 edits in the wiki, 150 posts to the list server, 25 contributions to the knowledge base and downloads of 2,000 from the knowledge base
5. At some week after the start, you take a measurement and find 180 wiki edits, 160 posts to the list server, 22 knowledge base contributions and 2200 downloads from the knowledge base.
6. The knowledge sharing index for that week would be 95.8. This is “down” even though most measures are up (which simply reflects the relative importance of one factor, which is down).

If I were to actually try something like this, I would pick the values of W_i so that the baseline measurement (when t= 0) comes to a nice round value – 100 or something. You can then imagine reporting something like, “Well, knowledge sharing for this month is at 110!” Or, “Knowledge sharing for this month has fallen from 108 to 92”. If nothing else, I find it amusing to think so concretely in terms of “how much” knowledge sharing is going on in an organization.

There are some obvious complexities in this idea that I don’t have good answers for:

1. How to manage a new means to measure activity becoming available? For example, your company implements a new collaboration solution. Do you add it in as a new factor with its weight and just have to know that at some point there’s a step function of change in the measure that doesn’t mean anything except for this new addition? Do you try to retroactively adjust weights of sources already included to keep the metrics “smooth”?
2. How to handle retiring a source of activity? For example, you retire that aging (but maybe still used extensively) mailing list server. Same question as above, though perhaps simpler – you could just retroactively remove measurements from the now-retired source to keep a smooth picture.
3. How to handle (or do you care to handle?) a growing or shrinking population of knowledge workers? Do you care if your metric goes up because you acquired a new company (for example) or do you need to normalize it to be independent of the number of workers involved?

In any event – I think this is an interesting, if academic, discussion and would be interested in others’ thoughts on either individual performance management or the idea of a knowledge sharing index.

A few weeks back, I was asked by Stan Garfield via email about how I might go about measuring if “knowledge specialization” is increasing – it was a question originally raised by Arnold Kling and Arnold had the hypothesis that increasing knowledge specialization in organizations was making management of those organizations more difficult.

Seth Earley was included on the email thread as well, and, while I replied (only on email – I didn’t post my reply here, though I could if anyone’s interested), I was sure Seth would have some good insights about how to go about grappling with the question.

Yesterday, Seth posted his reply on his blog, which I think highlight a good point about the initial theory – that even trying to analyze the level of specialization in knowledge is tricky because knowledge is fractal – no matter how detailed a look you take at it, there are always levels of detail below that.  To quote Seth:

[Knowledge] “is endlessly complex and classification depends on scale and perspective. It’s not a matter of “there should be more categories… “; there are more. It simply depends on where you look and your perspective.”

In my own reply, I had a vague feeling of unease about the idea of measuring increased knowledge specialization but did not think through what it meant, I tried to come up with ways one might try to discern a hypothetical increase in knowledge specialization. I’m glad to see Seth managed to more concisely crystalize the vague unease I had with the question.

I also really liked Seth’s summarization:

“The bottom line is that economic value is created not by understanding where all the knowledge is and micromanaging activities, but by providing broad constraints on targets, problems to solve, competitive differentiation, values, and resources and then creating the right circumstances that allow teams of people to focus knowledge and expertise on solving problems. Knowledge classifications are part of the tools for communicating value and telling the organization when trial and error has produced something that can be reused and applied to solving other problems.”