The problem of ranking, in which the goal is to learn an ordering or ranking over objects, has recently gained much attention in machine learning. Progress has been made in formulating different forms of the ranking problem, proposing and analyzing algorithms for these forms, and developing theory for them. However, a multitude of basic questions remain unanswered:
  • Ranking problems may differ in many ways: in the form of the training examples, in the form of the desired output, and in the performance measure used to evaluate success. What are the consequences of each of these factors on the design of ranking algorithms and on their theoretical guarantees?

  • The relationships between ranking and other classical learning problems such as classification and regression are still under-explored. Is any of these problems inherently harder or easier than another?

  • Although ranking is studied mainly as a supervised learning problem, it can have important consequences for other forms of learning; for example, in semi-supervised learning, one often ranks unlabeled examples so as to assign labels to the ones ranked at the top, and in reinforcement learning, one often learns a policy that ranks actions for each state. To what extent can these connections be explored and exploited?

  • There is a large variety of applications in which ranking is required, ranging from information retrieval to collaborative filtering to computational biology. What forms of ranking are most suited to different applications? What are novel applications that can benefit from ranking, and what other forms of ranking do these applications point us to?
This workshop aims to provide a forum for discussion and debate among researchers interested in the topic of ranking, with a focus on the basic questions above. The goal is not to find immediate answers, but rather to discuss possible methods and applications, develop intuition, brainstorm on possible directions and, in the process, encourage dialogue and collaboration among researchers with complementary ideas.