A framework for evaluating automatic indexing or classification in the context of retrieval

Koraljka Golub*, Dagobert Soergel, George Buchanan, Douglas Tudhope, Marianne Lykke, Debra Hiom

*Awdur cyfatebol y gwaith hwn

Allbwn ymchwil: Cyfraniad at gyfnodolynErthygladolygiad gan gymheiriaid

131 Wedi eu Llwytho i Lawr (Pure)


Tools for automatic subject assignment help deal with scale and sustainability in creating and enriching metadata, establishing more connections across and between resources and enhancing consistency. Although some software vendors and experimental researchers claim the tools can replace manual subject indexing, hard scientific evidence of their performance in operating information environments is scarce. A major reason for this is that research is usually conducted in laboratory conditions, excluding the complexities of real-life systems and situations. The article reviews and discusses issues with existing evaluation approaches such as problems of aboutness and relevance assessments, implying the need to use more than a single gold standard method when evaluating indexing and retrieval, and proposes a comprehensive evaluation framework. The framework is informed by a systematic review of the literature on evaluation approaches: evaluating indexing quality directly through assessment by an evaluator or through comparison with a gold standard, evaluating the quality of computer-assisted indexing directly in the context of an indexing workflow, and evaluating indexing quality indirectly through analyzing retrieval performance.

Iaith wreiddiolSaesneg
Tudalennau (o-i)3-16
Nifer y tudalennau14
CyfnodolynJournal of the Association for Information Science and Technology
Rhif cyhoeddi1
Dyddiad ar-lein cynnar22 Hyd 2015
Dynodwyr Gwrthrych Digidol (DOIs)
StatwsCyhoeddwyd - Ion 2016

Ôl bys

Gweld gwybodaeth am bynciau ymchwil 'A framework for evaluating automatic indexing or classification in the context of retrieval'. Gyda’i gilydd, maen nhw’n ffurfio ôl bys unigryw.

Dyfynnu hyn