CS 8751: Advanced Machine Learning

Fall 2003

Lecturer: Rich Maclin (rmaclin)



Class References:

Class Materials:

Program Assignments:

  1. Program 1: Create Your Own Dataset and Dataset Class
  2. Program 2: Learning a Decision Tree for Discrete Features
  3. Program 3: Decision Trees - Adding Capabilities to Program 2
  4. Program 4: Unsupervised Learning - K-Means Clustering
  5. Program 5: Reinforcement Learning - Q Learning for a simple game
  6. Extra Credit Program: Genetic Algorithms - A GABIL like learner

Homework Assignments:

  1. Homework 1
  2. Homework 2
  3. Homework 3
  4. Homework 4

Sample Exam Questions:

Talk Stuff:

Talk Schedule:

DateSpeakerPaperPaper PDFPresentation PDFComment 1 PDFComment 2 PDF
11/20 1: Jason Michellizi Vinokourov et al. PDF talk01-mich0212.pdf comment01-parl0020.pdf comment01-khan0178.pdf
11/20 2: Nagendra Doddapaneni Ben-Hur et al. PDF talk02-dodd0036.pdf comment02-data0003.pdf comment02-rave0029.pdf
11/25 3: Hemal Lal Wang and Dietterich PDF talk03-lalx0004.pdf comment03-khan0178.pdf comment03-salu0005.pdf
11/25 4: Yuhu Yan Kondor and Jebara PDF talk04-yanx0057.pdf comment04-rave0029.pdf comment04-potn0001.pdf
12/2 5: Ashraful Alam Suykens et al. PDF talk05-alam0026.pdf comment05-salu0005.pdf comment05-koda0012.pdf
12/2 6: Archna Yadav Valentini and Dietterich PDF talk06-yada0009.pdf comment06-potn0001.pdf comment06-kapo0020.pdf
12/2 7: Ravindra Bharadia Jaakkola et al. PDF talk07-bhar0022.pdf comment07-koda0012.pdf comment07-mich0212.pdf
12/4 8: Anoop Reddy Parlapalli Engel et al. PDF talk08-parl0020.pdf comment08-dodd0036.pdf comment08-dodd0036.pdf
12/4 9: Ajit Datar Platt PDF talk09-data0003.pdf comment09-mich0212.pdf comment09-lalx0004.pdf
12/4 10: Sudip Khanna Platt PDF talk10-khan0178.pdf comment10-dodd0036.pdf comment10-yanx0057.pdf
12/9-12/11 Hierarchical RL:
Main paper: 11: Pratheepan Raveendranathan
Options: 12: Sampanna Salunke
HAM: 13: Poorva Potnis
MAXQ: 14: Varsha Kodali
Multi-Agent Learning: 15: Tarun Kapoor
Barto and Mahadevan
Sutton et al.
Parr and Russell
Makar et al.
Barto and Mahadevan
Sutton et al.
Parr and Russell
Makar et al.
comment11-lalx0004.pdf comment12-yanx0057.pdf comment13-alam0026.pdf comment14-yada0009.pdf comment15-bhar0022.pdf comment11-alam0026.pdf comment12-yada0009.pdf comment13-bhar0022.pdf comment14-parl0020.pdf comment15-data0003.pdf

Papers to Present:

Support Vector Machines:
  1. Support Vector Clustering, Ben-Hur, Horn, Siegelmann, and Vapnik, JMLR, 2, 2001, Citeseer PDF
  2. A discriminative framework for detecting remote protein homologies, Jaakkola, Diekhans, and Haussler, TR, 1999, Citeseer PDF
  3. A Kernel Between Sets of Vectors, Kondor and Jebara, ICML 2003, PDF
  4. Fast Training of Support Vector Machines using Sequential Minimal Optimization, Platt, in Advances in Kernel Methods - Support Vector Learning, 1998, Citeseer PDF (NOTE: this should be done as a joint presentation by two students)
  5. A Support Vector Machine Formulation to PCA Analysis and its Kernel Version, Suykens, Van Gestel, Vandewalle, and De Moor, TR, 2002, Citeseer PDF
  6. Bias Variance Analysis and Ensembles of SVM, Valentini, and Dietterich, Third International Workshop on Multiple Classifier Systems, 2002, Citeseer PDF
  7. Inferring a Semantic Representation of Text via Cross-Language Correlation Analysis, Vinokourov, Shawe-Taylor, and Cristianini, NIPS, 2002, Citeseer PDF
Reinforcement Learning:
  1. The following paper is a survey paper discussing several useful ideas, I would like to see a 5 person presentation of this paper with the appropriate supporting papers shown below for each section.

    The paper: Recent Advances in Hierarchical Reinforcement Learning, Barto and Mahadevan, Discrete Event Systems 13, 2003, Citeseer PDF

  2. Topics:

    1. The main paper
    2. Options

      Based on: Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning, Sutton, Precup, and Singh, AIJ, 1999, Citeseer PDF

    3. Hierarchies of Abstract Machines

      Based on: Reinforcement Learning with Hierarchies of Machines, Parr, and Russell, NIPS, 1998, Citeseer PDF

    4. MAXQ

      Based on: Hierarchical reinforcement learning with the MAXQ value function decomposition, Dietterich, JAIR, 2000, Citeseer PDF

    5. Multi-agent approaches

      Based on: Hierarchical Multi-Agent Reinforcement Learning, Makar, Mahadevan, and Ghavamzadeh, Fifth International Conference on Autonomous Agents, 2001, Citeseer PDF

  3. Model-based Policy Gradient Reinforcement Learning, Wang, and Dietterich, ICML 2003, PDF
  4. Bayes meets Bellman: The Gaussian Process Approach to Temporal Difference Learning, Engel, Manoor, and Meir, ICML 2003, PDF

Some Useful Links: