|
This
course is intended for beginning
graduate students and advanced undergraduates. We assume students have
a rudimentary understanding of linear algebra, calculus, and are able
to program in some type of structured language. There will be four
homework assignments and a final project. Grading will be
approximately 60% on the homework assignments (15% each) and
40% on the final project. Secondary: 5-187 EE/CS
Building
Meetings:
Tuesdays and Thursdays , 12:45pm-2pm, Professor:
Paul Schrater E-mail:
schrater@umn.edu
Office: Primary:
211 Elliott Hall, Office
hours: 2pm-3:15pm Tues, or by appointment
Teaching Assistants:
Daniel Acuña acuna002@umn.edu
Office Hours:
(Daniel) 2 pm-3 pm
T/TH
Office:
2-209 EE/CS
Final Project Assignment: Your final project will involve one of the following
1) Simulation or experiments. For example, implement a pattern recognition system for a particular application, e.g. digit classification, document clustering, etc.
2) Literature survey (with critical evaluation) on a given topic.
3) Theoretical work (detailed derivations, extensions of existing work, etc)
In all cases, the work should be written up as a 12-15 page paper (single space). More difficult projects will get better grades if sucessfully completed. You may work in groups of 2-4. However, the content must be sufficient for the size of the group. You will be evaluated in terms of the care with which you set up and thought through the goals and implementation, and in terms of the competence of the execution. Regardless of form the write up should include a survey of related literature results.
The project schedule is:
Sept. 28: Topic selection. One or two pages explaining the project with
a list of references.
Nov. 7: Partial report (3 to 5 pages).
Dec 20: Final report (10 to 15 pages).
Graduate students should give a short presentation of their project
towards the last weeks of the semester.
This presentation will count for 5% of the total class grade (this
grade will be counted as part of the project
grade).
Example Projects (from previous classes):
Cheating and Plagiarism
The homework and programming assignments
must not be the result of cooperative work. Each student
must work individually in order to understand the material in depth.
You may discuss the issues but by
no means, copy the homework or the programming assignment of somebody
else. All work in the projects
and the programming assignment must properly cite sources. For example,
if you quote a source in your
project, you must include the quotation in quotation marks and clearly
indicate the source of the quotation.
Any student caught cheating will receive an F as a
class grade
and the University policies for cheating
and plagiarism will be followed.
Primary:
Pattern Classification, 2nd Ed. Duda, Hart & Stork,Wiley, 2002.
Secondary
(select
chapters from)
Statistical
Pattern Recognition, 2nd
Ed. Andrew Webb,Wiley, 2002.
Neural Networks for Pattern Recognition, Christopher Bishop, Oxford University Press,
1995.
Elements
of Statistical Learning: Data Mining, Inference, Prediction, Trevor Hastie, Robert Tibshirani, and Jerome
Friedman. Springer, 2001.
Gaussian
Processes for Machine Learning, Carl Rasmussen and Christopher K.I.
Williams, MIT Press, 2006.
| Week | Tuesday | Thursday | Suggested Readings | Lecture Notes | Assignment |
| 1 (9/4-9/6) | Introduction | Probability and Linear Alg. Review | Course Syllabus, Matlab Tutorials(1,2), | Lec1 Lec2 | |
| 2 (9/11-9/13) |
Elementary
Decision Theory | Elementary Decision Theory | DudaHartStork: Chap 1 Webb: Chap 1; | Lec3&4 | |
| 3 (9/18-9/20) | CATEGORICAL
INFERENCE Classification/Inference
with
Parametric models | Classification/Inference with Parametric models (Example) | DudaHartStork: Chap 2 Webb: Chap 2.1,2.2.1 | Lec5 Lec6 | |
| 4 (9/25-9/27) | Classification/Inference with Parametric models: Bayesian Disc via Sampling | Non-Parametric models: k Nearest Neighbor and kernel density estimation | DudaHartStork: Chap 3 Webb: Chap 2 Webb: Chap 3 |
Lec6&7 Lec8 Sampling.pdf | |
| 5 (10/2-10/4) | Non-Parametric
models:continued Linear Discriminant analysis | Linear Discriminant analysis | Webb: Chap 3.2.5 DudaHartStork: Chap 4 Webb: Chap 2.3; Bishop 2.6 | Homework
2 Due 10/11 | |
| 6 (10/9-10/11) | Patt Rec
example: Handwritten digit classification Matab for Pattern Rec. | Non-linear Disc Analysis: Support vector machines | DudaHartStork: Chap 5 Webb: Chap 4.1-4.3.3 | Examples.m | |
| 7 (10/16-10/18) | SVM | SVM | Webb: Chap 5.4 Burgestutorial | ||
| 8 (10/23-10/25) | Multi-layer NN | Multi-layer NN | DudaHartStork:
Chap 6 Webb: Chap 6 | Lec13 | |
| 9 (10/30-11/1) | LEARNING FUNCTIONAL MAPPINGS Regression and Basis functions | Regression and Basis functions | Hastie et al. Chap 3, Chap 5 | Regression | Homework 3 Due 11/27/07, |
| 10 (11/6-11/8) | SVM regression | SVM
regression | SV regression tutorial | RegressionII | |
| 11 (11/13-11/15) | Radial Basis function networks | Bayesian Regression | Webb: Chap 5.1-5.3Bishop:
Chap 5 RegressionBayesianRegression GP Tutorial | RegressionIII BayesRegress | |
| 12 (11/20-11/22) | Gaussian Processes | Thanksgiving | Hastie et al, Chap 9 | GP |
|
| 13 (11/27-11/29) | Gaussian Processes | Gaussian Processes | DudaHartStork:
Chap 10.13-10.14 Webb: Chap 10 DudaHartStork: Chap 9.3-9.6 Webb:Chap 8 Webb: Chap 9 DudaHartStork: Chap 9.3-9.6 | ||
| 14 (12/4-12/6) | PERFORMANCE
ISSUES Assessing/Improving Performance Boosting | Dimensionality Reduction | Boosting OriginalPaper Chapter for Lecture DudaHartStork: Chap 10 Webb: Chap 11 | Performance DimRed | Homework 4 is posted |
| 15 (12/11-12/13) | Unsupervised Learning | Clustering ProbClustering | FINAL PROJECT due Dec. 19th |