Lectures in Statistical Learning Theory
August 2014
Department of Computer Science & Automation
Indian Institute of Science
Introduction to Statistical Consistency in Machine Learning
Statistical consistency is a fundamental notion for learning algorithms
that asks the following simple question: As a learning algorithm is
supplied with more and more training data, does the prediction model
learned by it approach an ideal or optimal prediction model for the given
learning problem? The last several years have seen significant progress in
development of tools that help us understand and characterize consistency
of learning algorithms for various types of prediction problems. This
series of two 2hour lectures will give a guided tour of recent
developments in the area, including selfcontained introductions to the
central notions of loss functions and loss matrices, surrogate loss
functions, calibration, and surrogate regret bounds.
Lecture 1: Statistical consistency of learning algorithms for binary
classification problems
Fri Aug 22, 2014
3:00  5:00 PM
CSA 252
Lecture 2: Statistical consistency of learning algorithms for multiclass
prediction problems
Mon Aug 25, 2014
3:30  5:30 PM
CSA 252
References
 Lecture notes from E0 370, AugDec 2013 (these are scribed so somewhat rough, but should be possible to follow the main ideas):

Tutorial lecture slides, IndoUS lectures week, Jan 2014.
 Papers on proper CPE losses/proper composite surrogates:
 Papers on calibrated surrogates for binary classification:
 Peter L. Bartlett, Michael I. Jordan, and Jon D. McAuliffe.
Convexity, classification, and risk bounds.
Journal of the American Statistical Association, 101(473):138156, 2006.
 Clayton Scott.
Calibrated asymmetric surrogate losses.
Electronic Journal of Statistics, 6:958992, 2012.
 Nagarajan Natarajan, Inderjit S. Dhillon, Pradeep Ravikumar and Ambuj Tewari.
Learning with noisy labels.
NIPS 2013.
 Papers on calibrated surrogates for multiclass 01 classification:
 Papers on calibrated surrogates for general multiclass losses: