Can computers really extract knowlege?

Knowledge management is theoretically impossible. Real knowledge sits between your ears, unseen until it is needed. As happened today. Someone mentioned Battenburg cake to me and all sorts of long forgotten knowledge about tea parties at my grandma’s surfaced.

Not exactly a momentous bit of knowledge, but I joined a conversation on the subject on Facebook of all places. (The dyes in the cake are, apparently dangerous.)

Recently, I visited a company that specialises in testing staff knowledge through questionnaires. The idea is to find out what an employee knows about their job and to determine whether there are any gaps that need filling or good results that need exploiting.

Boards of very large companies have rather taken to this system, a sort of asset register of the staff and their expected performance on the job. They can use it to correct weaknesses or develop strengths. And, should a crisis occurs in a particular department, they can quickly pull up staff information to help them figure out what went wrong.

Test results can also be measured against averaged results for other organisations in the same industry – a sort of performance benchmark.

It all sounds terrific in theory. The underpinning technology is fundamentally sound. But, as always, the acid test is in the implementation. And that involves humans.

By the time the strategy and raw information has found its way to the question designers, all intimacy with the subject matter will have been squeezed out. It’s like speaking a foreign language. It doesn’t matter how perfect your accent, a native will know you are a foreigner within a very short time.

I’ve just read a blog post by a member of staff at the receiving end of an assessment run by this particular system. Slightly tidied up and anonymised, he said, “The people who designed the questions and answers knew nothing about my line of work. The end result has been questions that don’t make sense or which are so ambiguous that one needs to be a professor of English to understand them”.

You can see why I’ve not mentioned the company name. I will return to it when I’ve tried the system myself and dug a little deeper into the particular circumstances around the above comment. But it seems clear that one important step was forgotten – did they try the questionnaires out on people who understood the subject before letting it out in the wild?