We have a deep taxonomy of CS concepts – at the deepest level of the tree there are seven levels. The most useful concepts for precision search are of course the most granular concepts represented by the leaves of the tree.
However, concepts can be multi-parented, so the accurate application of a concept to a text requires that the context within the tree, i.e., the correct branch, be understood.
While expert authors who apply the terms to their articles have varying degrees of interest and attention to this indexing task, our experience shows that they rarely misapply terms – sometimes they appear lazy and are happy to assign only high-level concepts such as “Software” which is not too useful.
However, our experience with an auto-tagger shows that a huge amount of “noise” is created. We consider the noise unacceptable – presenting it to users will create distrust in the taxonomy itself.
We have been expanding the logical rules of the auto-tagger in an effort to reduce the noise to an acceptable level. So far, without success.
I have been trying to understand why.
So far, the best explanation I can come up with is that while hierarchical context is readily understood by the human brain, auto-taggers based on statistical occurrences of a concept and within proximity of other words and concepts, cannot accurately reproduce hierarchical context.
Any advice would be appreciated.