A Large-Scale Study of User Image Search Behavior on the Web

User interfaces for web image search engine results have a number of different, richer features beyond those available for traditional (text) web search results. In particular, users can see an enlarged image preview by hovering over a result image, and an `image preview' page (separate from the search results page) allows users to browse even further enlarged versions of the results, and to click through to the referral page where the image is embedded. No existing work investigates the utility of these interactions as features for improving search ranking, beyond simply using the click through rate on images displayed in the search results page. In this paper we propose a number of user feedback features based on these additional interactions: hover through rate, 'converted-hover' rate, referral page click through, and a number of dwell time features based on interactions in the image preview page. Also, since images are never self-contained, but always embedded in a referral page, we posit that click through to other images on the same referral page can carry useful relevance information. We also posit that query-independent versions of features extracted from user logs, while not expected to capture topic relevance, will capture feedback about the quality or popularity of images, which is an important dimension of relevance for web image search. We conduct an extensive set of ranking experiments in a learning to rank framework, using a large annotated web image corpus, to evaluate our proposed features. Our results show that the proposed features give statistically significant gains of over 2% compared with a state of the art baseline that uses standard click features.