Multivariate data are abundant and are found in all areas of application of statistics and machine learning. Multivariate analysis is concerned with the simultaneous statistical analysis of multiple variables. This is an advanced course intended for doctoral students in statistics and related fields. The course will introduce students to methodology, theoretical foundations, and computational aspects of multivariate data analysis from a modern perspective. Theoretical derivations will be presented together with practical aspects and intuition. The topics of the course include:
- Random vectors and matrices
- Summarization, display, and geometry of multivariate data
- Multivariate Gaussian theory
- Core techniques
- Principal components analysis (PCA)
- Linear discriminant analysis (LDA)
- Canonical correlation analysis (CCA)
- Cluster analysis and Gaussian mixture models
- Kernel methods
- Reproducing kernels
- Kernel PCA/CCA
- High-dimensional statistics and regularized procedures
- Sparse covariance estimation and Gaussian graphical model selection
- Sparse PCA/LDA/CCA
This is a 3 credit hour course in lecture format.
Stat 6802, or permission of the instructor. Students are expected to be able to read and write mathematical proofs. Preparation in multivariate calculus, linear algebra, and mathematical statistics is absolutely necessary for this course. Familiarity with the statistical computing environment R or languages such as Matlab is expected. Some of the concepts the instructor assumes that students are familiar with include:
- Convergence in probability and convergence in distribution
- Maximum likelihood, Fisher information
- Loss function, risk of an estimator
- Bias and variance
- Trace, determinants, eigenvalues, and eigenvectors
- Gradients and Hessians
The first homework will have a few review problems that indicate the level of preparation this course requires. If you find them too difficult, then you will probably have difficulty with the rest of the course.
There are no required books, but the following book is recommended for supplemental reading and freely available on the OSU network:
- Izenman, A. J. (2008): Modern Multivariate Statistical Techniques.
It provides modern and broad coverage of multivariate methods. It is similar in breadth and style to The Elements of Statistical Learning by Hastie, Tisbhirani, and Friedman. The instructor will occasionally recommend reading from Izenman (2008) and other references.
A free reference for matrix algebra and calculus is:
- Petersen, K.B. and Pedersen, M. S. (2012): The Matrix Cookbook. Available at
Additional reading material will be posted to the course website during the course.
Coursework & Grading
Evaluation will be based on the following components:
- 25% Homework
- 25% Exam
- 50% project
There will be occasional homework assignments. They will be posted on Carmen and collected in lecture on the due date.
There will be one in-class exam on March 24, 2017.
Students will work on a project through the semester, write a short (8 page, conference-style) paper, and deliver a presentation in the later part of the course. A list of potential topics will be provided by the instructor. The topic selected by the student must be approved in advance by the instructor.
It is the responsibility of the Committee on Academic Misconduct to investigate or establish procedures for the investigation of all reported cases of student academic misconduct. The term "academic misconduct" includes all forms of student academic misconduct wherever committed; illustrated by, but not limited to, cases of plagiarism and dishonest practices in connection with examinations. Instructors shall report all instances of alleged academic misconduct to the committee (Faculty Rule 3335-5-487). For additional information, see the Code of Student Conduct http://studentlife.osu.edu/csc/.
Students with disabilities that have been certified by the Office for Disability Services will be appropriately accommodated and should inform the instructor as soon as possible of their needs. The Office for Disability Services is located in 150 Pomerene Hall, 1760 Neil Avenue; telephone 292-3307, TDD 292-0901; http://www.ods.ohio-state.edu/.
This syllabus is a approximate guide to the course content and dates, however the instructor reserves the right to deviate from the syllabus. Official announcements of changes will always be made in class and updated version of the syllabus will be maintained on the course webpage.