Lectures in Statistical Learning Theory

August 2014

Department of Computer Science & Automation
Indian Institute of Science



Introduction to Statistical Consistency in Machine Learning

Statistical consistency is a fundamental notion for learning algorithms that asks the following simple question: As a learning algorithm is supplied with more and more training data, does the prediction model learned by it approach an ideal or optimal prediction model for the given learning problem? The last several years have seen significant progress in development of tools that help us understand and characterize consistency of learning algorithms for various types of prediction problems. This series of two 2-hour lectures will give a guided tour of recent developments in the area, including self-contained introductions to the central notions of loss functions and loss matrices, surrogate loss functions, calibration, and surrogate regret bounds.

Lecture 1: Statistical consistency of learning algorithms for binary classification problems

Fri Aug 22, 2014
3:00 - 5:00 PM
CSA 252

Lecture 2: Statistical consistency of learning algorithms for multiclass prediction problems

Mon Aug 25, 2014
3:30 - 5:30 PM
CSA 252

References

  1. Lecture notes from E0 370, Aug-Dec 2013 (these are scribed so somewhat rough, but should be possible to follow the main ideas):


  2. Tutorial lecture slides, Indo-US lectures week, Jan 2014.

  3. Papers on proper CPE losses/proper composite surrogates:


  4. Papers on calibrated surrogates for binary classification:


  5. Papers on calibrated surrogates for multiclass 0-1 classification:


  6. Papers on calibrated surrogates for general multiclass losses: