Date  Subject  Reading 
On your own  learn linear algebra, basic probability  Self Test, Self Test Solutions, Strang’s Course Probability Review Bishop 1.11.4, slides 
W/Aug 30  Intro, intro_slides  
F/Sep 1  Intro to Matlab Tutorial  Coursera octave tutorial 
M/Sep 4  No class: Labor Day  
W/Sep 6  Local learning Norm slides  Bishop 2.5 
F/Sep 8  Decision Trees (and information theory) slides  Decision Trees by N. Nilsson, Bishop 1.6 Bishop 14.4 KL and mutual info 
M/Sep 11  Basic Point Estimation (MLE and MAP) PDF slides  Bishop 2.1, Appendix B, MLMath covariance 
background  Gaussians  Bishop 1.2.4, Bishop 2.3.12.3.3 
W/Sep 13  Regression, Regression slides  Bishop 1.11.4, Bishop 3.1, 3.1.1, 3.1.4, 3.1.5, 3.2, 3.3, 3.3.1, 3.3.2 
F/Sep 15  Classification Logistic Regression, Naive Bayes slides
 Bishop 4.0, 4.2 4.5 
M/Sep 18  NB vs. LR, Basis Functions, RBFs  Naive Bayes vs Logistic Regression, short video: RBFs 
W/Sep 20  Overfitting and Regularization, Bias Variance Decomposition, Bias/Variance for Regression  Bishop 1.3, 1.5, 3.2 applet 
F/Sep 22  regression penalty slides Stepwise, streamwise, stagewise  short video:stepwise regression,Hastie et al. 7.17.3 LASSO (supplemental reading) 
M/Sep 25  MDL slides, MDL feature selection review  MDLsupplemental MDLbackground 
W/Sep 27  Neural Net slides Supervised Deep Networks  supplemental: deep net tutorial 
F/Sep 29  more deep learning Supervised Deep Networks  supplemental: gay learning 
M/Oct 2  Constrained Optimization and Lagrangian Duality  
W/Oct 4  Support Vector Machines
 Additional notes, Bishop 7.1 (Max Margin), Hearst 1998 
F/Oct 6  No class: Fall Break  
M/Oct 9  Kernel Methods
 Additional notes, Bishop 6.1,6.2 (Kernels) Supplemental: more on kernels 
W/Oct 11  Online Learning and Perceptron  Additional notes 
F/Oct 13  Boosting  Additional notes, Bishop 14.3, Schapire’s Tutorial

M/Oct 16  Performance Measures  
T/Oct 17  review session 5:00 p.m. ANNS Room: 110  Recitation Remix MLE/MAP examples (Multivariate Gaussian, Poisson) Decision Trees, CrossValidation, Boosting Complexity consistency 
W/Oct 18  Midterm recitation recording  sample exam questions and answers more examples are part of the old final exam below 
F/Oct 20  No class  
M/Oct 23  Learning Theory: A Brief Primer  
W/Oct 25  Vectors, Matrices, SVD SVD slides  Kosecka’s review slides 
F/Oct 27  Dim. Reduction PCA eigenwords  Bishop 12.1: PCA Bishop Appendix C Properties of Matrices PCR, PLS and CCA 
M/Oct 30  Unsupervised Deep Networks more courses 
W/Nov 1  Unsupervised Learning: Clustering, Kmeans EM  Bishop 9.19.3 
F/Nov 3  EM Generative PCA,LDA slides  Bishop 12.1–12.3, supplemental:Neal and Hintonsupplemental:LDA intro and original LDA paper 
M/Nov 6  Netflix  supplemental:netflix 
W/Nov 8  Project overview and advice and project slides; Real world machine learning  More advice and python for converting text to ints 
F/Nov 10  Hidden Markov Models  supplemental: Rabiner’s HMM Tutorial, Bishop 13.1–2 
M/Nov 13  Hidden Markov Models  supplemental: Rabiner’s HMM Tutorial, Bishop 13.1–2 
W/Nov 15  Bayesian Networks  Jordan book draft Ch 2 (Sec 2.1), additional notes, supplemental: Koller et al., Graphical Models in a Nutshell 
F/Nov 17  Markov Networks; Variable Elimination  Jordan book draft Ch 2 (Sec 2.2) and Ch 3; supplemental: Koller et al., Graphical Models in a Nutshell 
M/Nov 20  SumProduct, MaxSum  Jordan book draft Ch 4 (Sec 4.1, 4.3), supplemental: Jordan book draft Ch 17, Koller et al., Graphical Models in a Nutshell 
W/Nov 22  No class  
F/Nov 24  No class — Happy Thanksgiving!!  
M/Nov 27  Structured Prediction  supplemental: Lafferty et al., 2001 (CRFs), McCallum, 2003 (more CRFs), Taskar et al., 2003 (StructSVM), Collins et al., 2008 (EG algorithms for CRFs and StructSVM), Joachims et al., 2009 (Cuttingplane algorithm for StructSVM) 
W/Nov 29  SemiSupervised Learning, Active Learning  supplemental: Nigam et al., 2000 (EM for semisupervised learning), Zhu’s semisupervised learning survey, Settles’ active learning survey 
F/Dec 1  More deep learning, Killian’s lecture  
M/Dec 4  Reinforcement Learning  supplemental: additional reinforcement learning exercise 
W/Dec 6  Big Data  Unreasonable effectiveness of data 
F/Dec 8  The Future of ML and humanity  supplemental: job futures 
M/Dec 11  Final project awards; Other material Dynamic Nets current deep learning
 LIONbook  a quick review; After CIS520 
W/Dec 13  Review Session: 4:00–5:00 pm Building: ANNS 110  Final Review Review Questions audio recording (under canvas announcements) 
T/Dec 19  Final: 9:00am11:00am CHEM 102  2014 final and solution and 2016 final and solution 