Date | Subject | Reading | Questions |

On your own | review probability | Probability Review Bishop 1.1-1.4, slides MIT Probability Open Course | | |

On your own | review linear algebra | Strang’s MIT OpenCourse MLMath | Self Test, Self Test Solutions, |

W/Aug 29 | Intro, (pdf) | | |

F/Aug 31 | Point Estimation: MLE and MAP probability densities (pdf) | supplemental:Bishop 2.1, Appendix B, covariance | quiz |

M/Sep 3 | No class: Labor Day | | |

W/Sep 5 | Local learning Norms (pdf) | Bishop 2.5 | quiz |

F/Sep 7 | Intro to Matlab Tutorial | Coursera octave tutorial | quiz |

M/Sep 10 | Decision Trees and information theory (pdf) | Decision Trees by N. Nilsson, Bishop 1.6 Bishop 14.4 KL and mutual info | |

on your own | Gaussians | Bishop 1.2.4, Bishop 2.3.1-2.3.3 | |

W/Sep 12 | Regression, Regression slides | Bishop 1.1-1.4, Bishop 3.1, 3.1.1, 3.1.4, 3.1.5, 3.2, 3.3, 3.3.1, 3.3.2 | |

F/Sep 14 | RBF, gradient descent | | quiz |

M/Sep 17 | Overfitting and Regularization, Bias Variance Decomposition, Bias/Variance for Regression | Bishop 1.3, 1.5, 3.2 applet | |

W/Sep 19 | regression penalty slides Stepwise, streamwise, stagewise | short video:stepwise regression,Hastie et al. 7.1-7.3 LASSO (supplemental reading) | |

F/Sep 21 | MDL (pdf) | MDL-supplemental MDL-background | |

M/Sep 24 | Classification Logistic Regression, Naive Bayes slides
| Bishop 4.0, 4.2- 4.5 | |

W/Sep 26 | Neural Net slides Supervised Deep Networks | supplemental: deep net tutorial | |

F/Sep 28 | more deep learning Supervised Deep Networks | supplemental: gay learning | |

M/Oct 1 | Constrained Optimization and Lagrangian Duality | | |

W/Oct 3 | Support Vector Machines
| Additional notes, Bishop 7.1 (Max Margin), Hearst 1998 | |

F/Oct 5 | No class: Fall Break | | |

M/Oct 8 | Online Learning and Perceptron | Additional notes | |

W/Oct 10 | Boosting | Additional notes, Bishop 14.3, Schapire’s Tutorial
| |

F/Oct 12 | Review for midterm | Recitation Remix MLE/MAP examples (Multivariate Gaussian, Poisson) Decision Trees, Cross-Validation, Boosting Complexity consistency | |

M/Oct 15 | Midterm | Sample questions and answers. More examples are part of the old final exam below. | |

W/Oct 17 | SVD SVD slides | Kosecka’s review slides | |

F/Oct 19 | Dim. Reduction PCA eigenwords | Bishop 12.1: PCA Bishop Appendix C Properties of Matrices PCR, PLS and CCA | |

M/Oct 22 | Unsupervised Deep Networks | | |

W/Oct 24 | Unsupervised Learning: Clustering, K-means EM | Bishop 9.1-9.3 | |

F/Oct 26 | LDA slides | Supplemental:LDA intro and original LDA paper | |

M/Oct 29 | Performance Measures | | |

W/Oct 31 | Generative PCA, CCA | Bishop 12.1–12.3, supplemental:Neal and Hintons | |

F/Nov 2 | Netflix | supplemental:netflix | |

M/Nov 5 | Project overview and advice and project slides; Real world machine learning | More advice | |

W/Nov 7 | Bayesian Networks | Jordan book draft Ch 2 (Sec 2.1), additional notes, supplemental: Koller et al., Graphical Models in a Nutshell | |

F/Nov 9 | Markov Networks; Variable Elimination | Jordan book draft Ch 2 (Sec 2.2) and Ch 3; supplemental: Koller et al., Graphical Models in a Nutshell | |

M/Nov 12 | Hidden Markov Models | supplemental: Rabiner’s HMM Tutorial, Bishop 13.1–2 | |

F/Nov 16 | More deep learning, Dynamic Nets | Killian’s lecture | |

M/Nov 19 | Sum-Product, Max-Sum | Jordan book draft Ch 4 (Sec 4.1, 4.3), supplemental: Jordan book draft Ch 17, Koller et al., Graphical Models in a Nutshell | |

M/Nov 19 | Semi-Supervised Learning, Active Learning | supplemental: Nigam et al., 2000 (EM for semi-supervised learning), Zhu’s semi-supervised learning survey, Settles’ active learning survey | |

W/Nov 21 | No class | | |

F/Nov 23 | No class — Happy Thanksgiving!! | | |

M/Nov 26 | Reinforcement Learning | supplemental: additional reinforcement learning exercise | |

M/Dec 3 | Reinforcement Learning | supplemental: additional reinforcement learning exercise | |

F/Nov 30 | Structured Prediction | supplemental: Lafferty et al., 2001 (CRFs), McCallum, 2003 (more CRFs), Taskar et al., 2003 (StructSVM), Collins et al., 2008 (EG algorithms for CRFs and StructSVM), Joachims et al., 2009 (Cutting-plane algorithm for StructSVM) | |

M/Dec 3 | Causality | | |

W/Dec 5 | Big Data | Unreasonable effectiveness of data | |

F/Dec 7 | The Future of ML and humanity | supplemental: job futures | |

M/Dec 10 | Final project awards; Other material current deep learning
| LIONbook - a quick review; After CIS520 | |

?T/Dec 11???? | Review Session: 5:00–6:00 pm Building: Towne 100 | Final Review Review Questions | |

F/Dec 14 -but registrar says “tentative” | Final: 9:00am-11:00am TBD | 2016 final and solution | |