Statistical Machine Learning from Data


prepared by Samy Bengio, IDIAP


This series of lectures were given at EPFL during the winter 2005-2006 session, in the I&C Computer, Communication and Information Sciences Doctoral Program

Course Description:

The aim of statistical machine learning is to construct systems able to learn to solve tasks given a set of examples of those tasks and some prior knowledge about them. This includes tasks such as handwritten or speech recognition, time series prediction, image classification, etc. The goal of this course is thus to present the main concepts of statistical machine learning, including theoretical aspects such as the generalization properties (how will the model work on unseen data), and more practical aspects, such as several state-of-the-art models for static and dynamic problems, for either classification, regression or density estimation. The course will furthermore use several real-life applications to illustrate the interest of statistical machine learning.

Lectures and tentative schedule:

Lecture or Lab Slides Schedule
Introduction to Machine Learning pdf 26 October 2005
Statistical Machine Learning Theory pdf 26 October 2005
LAB: Statistical Machine Learning Theory (lab material) pdf 2 November 2005
Classical Models pdf 9 November 2005
Multi Layer Perceptrons and Gradient Descent pdf 16 November 2005
LAB: Classical Models (UCI databases, lab material) pdf 23 November 2005
More Artificial Neural Networks pdf 30 November 2005
LAB: Neural Network Models (lab material) pdf 7 December 2005
Gaussian Mixture Models and Expectation-Maximization pdf 14 December 2005
Hidden Markov Models pdf 21 December 2005
LAB: GMMs and HMMs (lab material) pdf 11 January 2006
Support Vector Machines pdf 18 January 2006
Decision Trees pdf 25 January 2006
Ensembles pdf 25 January 2006
LAB: Large Margin Models (lab material) pdf 1 February 2006
Feature Selection pdf 8 February 2006
PAC Learning (Francois Fleuret) pdf 8 February 2006
EXAM TBD 22 February 2006

General references on each topic pdf html
Quick summaries pdf

Practical Informations: