Disambiguating image descriptions is an important task for supporting users' efficient information access to images for exploratory purposes. For example, in a museum exhibition, information terminals could offer users the opportunity to study further about unfamiliar concepts if the terms in short image descriptions were properly interpreted by the system. With this type of application in mind, we modeled disambiguation as the task of linking the descriptions to entities in a knowledge base, Wikipedia. Concept disambiguation has been extensively studied in the context of Wikipedia linking, but the focus of most prior work has been on linking relatively long text (e.g., newspapers articles) consisting of up to a few hundred words, whereas image descriptions often contain just a few words. Because of this difference, existing systems do not work well in this setting. We
therefore propose to also leverage additional features that can be
extracted from the image itself using image classifiers that have been recently developed (by others) that are based on deep learning. The experiment results show that the linking accuracy of our method, with a mean reciprocal rank near 0.6, outperforms current methods that use only the description text. In this talk I will describe the methods we have tried, the test collections on which we have performed the evaluation, and the results we have obtained.