E0 371: Topics in Machine Learning

January - April 2011

Department of Computer Science & Automation
Indian Institute of Science



[Course Description]  [Guidelines]  [Academic Honesty]  [Schedule]  [References] 

Course Information


Course Description

Goals of the Course

This is a research-oriented course in machine learning that will help students achieve three goals:

Syllabus

Selected topics of current interest in machine learning and statistical learning theory. Examples include learning and regularization on graphs, structured prediction, learning of sparse models, low-rank matrix approximations, and learning to rank. Other topics may be selected based on class interest. References will include (but are not limited to) recent proceedings of COLT, ICML and NIPS conferences.

Course Structure

The course will be based primarily on seminars and research projects. By popular request, the first week will feature a statistical learning theory primer by the instructor. Following this, in the first half of the course, students will pick 1-2 papers of interest (see suggestions below) and give seminars on these to the class. Each seminar presenter will receive personalized feedback on his/her presentation from the instructor in an individual half-hour session the afternoon of the presentation. Everyone will also be required to read the papers being presented each week, critically analyze them, and submit brief (1-2 page) reports/reviews on the papers at the start of the week; this will facilitate active discussion on the papers in class. Again, reviews/reports will be reviewed and feedback will be provided.

During the second half of the course, students will form small teams and work on research projects identified during the first half of the term, with guidance from the instructor. Projects will be iteratively refined over the remaining weeks via reading and feedback sessions, with a final project paper and presentation due at the end of the term.

The best project papers will receive additional mentoring after the course for possible submission to NIPS in June.

Grading

Class grades will be based on paper reviews, seminar presentations, class participation, and research projects, divided roughly as follows:

Paper reviews20%
Seminar presentation    20%
Class participation20%
Research project40%

For paper reviews, you'll be allowed to skip 1-2 weeks without penalty (for most people, this will be 1 week; if you're scribing a lecture, you may skip 2 weeks). For research projects, grades will be based on the initial progress in terms of literature survey and refining the problem, progress updates, and final presentations and reports.

Guidelines on giving presentations and writing paper reviews/project reports, as well as notes on the academic honesty policy of the class, can be found below.

Who Should Take This Course?

This course is designed primarily for advanced research students interested in machine learning. If you are a PhD student with research interests related to machine learning, or a second-year Masters' student interested in exploring machine learning as a research area, this course is for you. If you are a beginning Masters' student with some research maturity and interests in machine learning, this course may also be suitable for you (discuss this with the instructor). While there are no formal prerequisites, a basic introductory-level course related to machine learning (or equivalent research exposure) will be assumed.

If you are considering registering for the course and would like more information, feel free to send an email to the instructor or drop by during office hours on Wed Jan 5 (2-5pm, CSA Room 201).


Guidelines

Guidelines for Paper Reviews

Use the following template for writing your reviews: pdf | tex

This is simply meant to serve as a rough guide; feel free to modify it as necessary. Reviews should be submitted by email (to the instructor, with cc to the TA) before the start of the class on the date on which they're due (see schedule below). The only exception is for those presenting a paper, who can submit their reviews for that week by Friday noon.

Also see the academic honesty policy below.

Guidelines for Seminar Presentations

Aim for a roughly 45 minutes presentation; the remaining time will be taken up by discussion. You can choose to make a slide presentation, a whiteboard presentation, or a combination of both, whichever enables you to communicate the results of the paper to the class most clearly (in most cases, having at least some slides will allow you to give a more effective presentation in the given time). In your presentation, attempt to do the following: Finally, remember that you are presenting not the authors' work, but your understanding of their work. Feel free to add your own examples and opinions, and try to make connections with other work if possible.

Guidelines for Project Proposals/Reports

You are encouraged to discuss your project plans with the instructor early on (some project suggestions will also be provided by the instructor). Further details will be discussed in the class and will be made available here in due course.

Project proposals can be prepared using the same template as those used for your reviews; the proposals should be 4-8 pages in length, and should provide motivation/background, describe the problem(s) you're considering and any relevant prior work, and possibly outline any ideas you may already have for approaching the problem.

Final project reports can be submitted in a format of your choice, e.g. the format used for your reviews, or any other format that works for you . The report should include a detailed introduction section that provides motivation/background, a self-contained description of relevant prior work, any necessary preliminaries, followed by your results/observations/evaluations, and a section on conclusions/future work. The expected length would be roughly 15-25 pages in the format used for your reviews (single-column, full-page), but this is only a rough guide and will vary with the format you use.


Academic Honesty

Paper Reviews

You may discuss the papers with your colleagues, but your reviews must be written on your own and must provide a summary and judgment of the papers in your own words. If you discuss a paper with someone and this influences your understanding/judgment of it, please mention this in your review; this will not adversely affect your score for the review, but rather will be treated positively. If your review is found to be highly similar to another review, both will automatically receive a zero score; any repeat instance will automatically result in a failing grade.

Project Proposals/Reports

You can develop a project proposal on your own, or in a small team (ideal size for such a team is two persons). Project proposals and reports are to be submitted jointly by each team.


Tentative Schedule

Date Topic Presenter Notes
Week 0: Registration
W Jan 5 No class (office hours 2-5pm).
You're strongly encouraged to go to the MSR Approximability School talks on campus instead!
   
Week 1: Statistical Learning Theory Primer
M Jan 10 Lecture: Introduction to Generalization Bounds Shivani Agarwal  
W Jan 12 Lecture: Introduction to Statistical Consistency Shivani Agarwal  
Weeks 2-7: Paper Reading/Presentations
M Jan 17 [G7] Kernels and Regularization on Graphs.
Alexander J. Smola and Risi Kondor.
COLT 2003.
Arun Rajkumar Reviews for papers [G7] and [M3] due
W Jan 19 [M3] Matrix Completion from Power-Law Distributed Samples.
Raghu Meka, Prateek Jain, Inderjit Dhillon.
NIPS 2009.
Adway Mitra  
M Jan 24 [R6] On the Consistency of Ranking Algorithms.
John Duchi, Lester Mackey, and Michael I. Jordan.
ICML 2010.
Harish Guruprasad Reviews for paper [R6] due
W Jan 26 Holiday (Republic Day)    
M Jan 31 No class.
Submit your ICML papers!
   
W Feb 2 Project meetings with instructor.    
M Feb 7 [ME4] Learning From Crowds.
Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, Linda Moy.
Journal of Machine Learning Research, 11:1297-1322, 2010.
Harikrishna Narasimhan Reviews for papers [ME4] and [ME5] due
W Feb 9 [ME5] Modeling annotator expertise: Learning when everybody knows a bit of something.
Yan Yan, Romer Rosales, Glenn Fung, Mark Schmidt, Gerardo Hermosillo, Luca Bogoni, Linda Moy, and Jennifer G. Dy.
AISTATS 2010.
Priyanka Agrawal  
M Feb 14 [S11] A Family of Penalty Functions for Structured Sparsity.
Charles Micchelli, Jean Morales, Massimiliano Pontil.
NIPS 2010.
Achintya Kundu Reviews for paper [S11] due
W Feb 16 Holiday (Id-E-Milad)    
M Feb 21 [G5] An Analysis of the Convergence of Graph Laplacians.
Daniel Ting, Ling Huang, Michael I. Jordan.
ICML 2010.
Mustafa Khandwawala Project proposals due;
Reviews for papers [G5] and [O3] due
W Feb 23 [O3] Robustness and Generalization.
Huan Xu, Shie Mannor.
COLT 2010.
Sahely Bhadra
Weeks 8-9: Project Team Meetings
M Feb 28 Project meeting    
W Mar 2 Project meeting    
M Mar 7 Project meeting    
W Mar 9 Project meeting    
Weeks 10-14: Paper Reading/Presentations and Project Progress Updates
M Mar 14 Lecture: Introduction to Online Learning [slides] Vikram Tankasali  
W Mar 16 [O5] Regularization Techniques for Learning with Matrices.
Sham M. Kakade, Shai Shalev-Shwartz, and Ambuj Tewari.
Preprint, arXiv:0910.0610, 2010.
Vikram Tankasali Reviews for paper [O5] due
M Mar 21 Class canceled.  
W Mar 23 How to Give a Good and a Bad Presentation. Shivani Agarwal Project progress papers due
M Mar 28 Project progress updates Adway;
Harikrishna & Priyanka
 
W Mar 30 Project progress updates Arun & Harish  
M Apr 4 Holiday (Chandramana Ugadi)    
W Apr 6 Project progress updates Adway  
M Apr 11 Project progress updates Harikrishna & Priyanka  
W Apr 13 Project progress updates Arun & Harish  
Week 15: Final Project Presentations/Papers
M Apr 18 Project presentations (CSA Room 101, 3-4:30pm) All  
M Apr 25 Final project papers due   Final project papers due



Suggested Papers for Reading/Presentation

Learning and Regularization on Graphs

Structured Prediction

Learning of Sparse Models

Low-Rank Matrix Approximations

Learning to Rank

Learning from Multiple Experts

Others