摘要： The semantic gap problem in image retrieval has motivated much work focusing on automatic image annotation, aimed at facilitating computers to automatically assign keywords to images. The basic measure for evaluating the annotation performance is usually to examine the annotation accuracy. To do this, the fraction of the relevant images, which have been correctly classified by a specific classifier or image annotation system, is measured. Consequently, the evaluation result can be thought of as a surrogate for the judgment of real users. However, the ability of this kind of quantitative evaluation measure to fully evaluate the performance and value of image annotation systems is limited. This paper introduces two complementary metrics related to the rates of annotation accuracy, which can help to further assess the robustness and stability of image annotation systems. They are: (i) the number of annotated keywords with zero-rate accuracy and (ii) the coefficient of variation of annotation accuracy. The evaluation results based on three datasets show that these two metrics are very useful to make a more reliable conclusion for image annotation systems. 出版者： Taylor & Francis 出版日期： 2016-02-17 出處： The imaging science journal, 2016-02, Vol.64 (2), p.94-99 資源來源： EBSCOhost Academic Search Premier 版權： 2016 The Royal Photographic Society 2016 識別號： ISSN: 1368-2199 識別號： EISSN: 1743-131X 識別號： DOI: 10.1080/13682199.2016.1139290
